feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
//+------------------------------------------------------------------+
2026-07-14 22:36:27 -04:00
//| Warrior_EA |
//| AnimateDread |
//| |
//+------------------------------------------------------------------+
# include "ExpertSignalCustom.mqh"
# include "..\AI\Network.mqh"
feat: add unified MA type support to indicator tuner
Add `MA_TYPE_PRESETS` enum covering advanced (ALMA, DEMA, ZLEMA, T3, Kalman) and standard (SMA, EMA, SMMA, LWMA) moving averages. Integrate `maType` and `bestMaType` into `CADIndicatorTuner` struct, update flatten/unflatten routines, and bump `AD_TUNE_PARAM_COUNT` to 33. This allows the auto-tuner to search over MA type alongside period, improving feature discovery.
2026-07-23 15:02:09 -04:00
# include "..\Variables\IndicatorResources.mqh"
2026-07-14 22:36:27 -04:00
# include "..\Variables\IndicatorTuneRanges.mqh"
2026-07-17 21:28:59 -04:00
# include "..\System\StatusLabel.mqh"
2026-08-23 21:18:32 -04:00
# include "..\System\SharedFileCopy.mqh"
feat: add configurable news event proximity/impact as an NN input feature
Price, time, volume, and volatility were already trained-model input
features; the real economic calendar (already used for the live
NewsFilter veto) is now an optional one too, reusing
System/NewsRelevance.mqh's symbol-relevance logic from the prior fix.
New EnableNews/NewsFeatureWindowMinutes inputs gate two features per
bar: minutes-since and minutes-until the nearest symbol-relevant
calendar event, impact-weighted. Deliberately limited to proximity +
impact, not actual-vs-forecast deviation - release schedules are
public knowledge ahead of time (not lookahead bias to use for a
historical training bar), but a release's actual outcome is not.
Wired identically to the existing EnableVolume/EnableTime/EnableATR
toggles: InitIndicators() accounts for the +2 neuron count,
BufferTempDataCompute() appends the two feature values, PAI/CONV/LSTM
all wired in Warrior_EA.mq5. Compiled clean (MetaEditor, 0 errors/0
warnings).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-18 17:26:04 -04:00
# include "..\System\NewsRelevance.mqh"
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
# include "..\System\CrossAsset.mqh"
2026-08-16 13:39:00 -04:00
# include "..\System\AltData.mqh"
refactor(ExpertSignalAIBase): extract AutoTune param state into CADIndicatorTuner
CExpertSignalAIBase (4,326 lines, one class) carried 5 struct
definitions, 10 member fields, and 3 methods (Flatten/Unflatten/
PerturbRandom) purely for the AutoTuneIndicators search-space state -
entirely self-contained (never touches Net, Train()'s resumable state
machine, or anything else in the class). Extracted into a new
Expert/ADIndicatorTuner.mqh (CADIndicatorTuner), held as a single
m_indicatorTuner member.
TuneIndicatorsAndTrain() itself - the outer loop that actually
orchestrates Train()/Net/checkpointing around this tuner - turned out
to be exactly as tightly coupled to Train()'s resumable state machine
as Train() itself, so per the same caution already applied to Train()
in this refactor pass, it stays in CExpertSignalAIBase rather than
being pulled into the collaborator; it now calls the tuner's public
Flatten()/Unflatten()/PerturbRandom()/SaveAsBest()/RestoreBest()
instead of manipulating the structs inline.
All internal field-access renames (m_adCumDeltaParams.lookback ->
m_indicatorTuner.adCumDelta.lookback, etc., ~40 sites across the 5
InitAD*() indicator-setup methods) verified against a full grep sweep
- no leftover references to the old field/method names. Compiled
clean (MetaEditor, 0 errors/0 warnings).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-18 16:22:09 -04:00
# include "ADIndicatorTuner.mqh"
refactor(dry): one binomial arithmetic for every "is this edge real" test
The formula p(1-p)/n was transcribed nine times across six files - the two
deploy gates, the two edge floors, the collapse recall floor, the barrier
rung ladder, the inference bin SE, the pooled inverse-variance weights and
both detectability reports. System\BinomialStats.mqh now holds it once, as
free functions with no class dependency, so the god-class declaration does
not grow to host pure math.
BinomialVar(p, n) p(1-p)/n
BinomialSEPct(p, n) 100*sqrt(p(1-p)/n)
BinomialCallsForEdge(p, edge, sigmas) the same, solved for n
NormalUpperTailQ(z) Q(z), via Math\Stat\Normal.mqh
SidakFamilyP(z, N) 1-(1-Q(z))^N
Value-preserving by construction: rates go in as probabilities so no call
site gained a *100/100 round-trip, and BinomialSEPct is written through
BinomialVar so the multiply order is the one it replaced. Every degenerate
guard each site carried (p<=0, p>=1, n<=0) now lives in one place and
returns the 0 those sites already treated as "no bar to clear".
CExpertSignalAIBase::NormalUpperTail is gone; NormalUpperTailQ replaces it.
What consolidating SURFACED, and is deliberately NOT changed here: the two
Sidak selection gates compute their SE on the RAW call count, while every
other SE in the project deflates by EffectiveSampleSize() for triple-
barrier label overlap. That makes them the most permissive test in the
codebase, by ~sqrt(mean label lifespan). Correcting it tightens a live
deploy bar, which is a policy decision, not a refactor - flagged in the
code at both sites.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:50:24 -04:00
# include "..\System\BinomialStats.mqh"
refactor(arch): a read-only training-data view, so modules stop being #included code
The AIBase\*.mqh files are not modules. They are method bodies of one
3,400-line class, textually #included after its declaration. Every one
of them can touch every member of every other, which is why "move this
out" has so far meant "move the whole class".
Introduce the seam that ends that:
CTrainingDataView abstract - the ONLY thing a training-side
collaborator may see: a feature row, a label, an
outcome, an excursion, the shape they share, and
the identity to log under.
CAIBaseTrainingData the adapter. MQL5 gives a class exactly one base
and CExpertSignalAIBase is already a
CExpertSignalCustom, so it cannot implement the
view itself. It owns one of these instead.
Data*() on the the published read API the adapter forwards to.
signal MQL5 has no `friend`, so reaching in from outside
was never an option - and making it explicit is
the point rather than a workaround.
Every row accessor OWNS ITS BOUNDS TEST and answers false for a bar it
has nothing for. Thirty-odd call sites currently carry their own
ArraySize() guard; one that forgets reads past a cache that is shorter
than the bar count for the whole warm-up. The -2.0 "never scored"
sentinel on the arrow cache is folded in the same way, so it can no
longer be mistaken for a small confidence.
Nothing uses it yet - this is the seam only, kept as its own commit so
the pattern compiles before 951 lines of Baselines move onto it. The
pattern is the stdlib's own: abstract base with =0 (Canvas\DX\DXObject),
concrete override, forward-declared owner pointer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:53:00 -04:00
//--- The read-only view training-side collaborators depend on, and the adapter that lets this
//--- class satisfy it without inheriting it (MQL5 gives a class exactly one base).
# include "Training\ITrainingData.mqh"
# include "Training\AIBaseTrainingData.mqh"
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
//--- Same shape, chart-side: the read-only view CChartUI depends on for arrows, the status panel
//--- and the HUD line. CChartUI itself (Chart\ChartUI.mqh) is included further down, next to
//--- Training\BaselineComparator.mqh - it needs MAX_PERSISTED_ARROWS/ARROW_RESTORE_BUDGET_MS etc.,
//--- which are #defined later in this file, before it can be parsed.
# include "Chart\IChartView.mqh"
# include "Chart\AIBaseChartView.mqh"
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
//--- Same shape again, persistence-side: the read+write view CModelPersistence depends on for the
//--- .cfg/.stats sidecars, CPU-inference validation and net-load retry. CModelPersistence itself
//--- (Persistence\ModelPersistence.mqh) is included further down, next to Chart\ChartUI.mqh - it
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- needs CPU_INFERENCE_MAX_DIFF, #defined later in this file.
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
# include "Persistence\IPersistenceView.mqh"
# include "Persistence\AIBasePersistenceView.mqh"
refactor(stdlib): one quantile definition, from Math\Stat
The codebase had THREE conventions for the same statistic. AltData took a
true median; the barrier horizon and the derived input window took the
upper of the two middle values; the MI terciles and the barrier stop
ladder used nearest-rank indexing. All four now go through MathMedian /
MathQuantile, which is R's type 7 and the library's one answer.
System\AltData.mqh column median -> MathMedian (exact, no change)
AIBase\Labels.mqh swing median -> MathMedian
leg-range med -> MathMedian
stop ladder -> MathQuantile, read in one call
AIBase\Topology.mqh window median -> MathMedian
AIBase\AutoTune.mqh MI terciles -> MathQuantile + MathMin/MathMax
Signals\SignalSessionFilter DST last Sunday-> CDateTime::DaysInMonth()
gaps[]/legs[] change from int to double so MathMedian can read them; the
values are bar counts either way.
VALUES MOVE. Even-sample medians shift by half a bin and the quantile
reads interpolate, so the barrier geometry and the derived input window
can land on different rungs - re-keying fingerprints and forcing a
retrain. Accepted deliberately: stdlib consistency was the ask, and three
private conventions for one statistic is what it buys out.
Two YAGNI finds fell out of the ladder rewrite. MathQuantile sorts its own
copy, so DeriveBarrierGeometry no longer sorts up[]/dn[] in place - which
means upUnsorted[], a full array copy kept only to undo that sort, is
gone. ArraySort(up) had no consumer needing order at all; it was pure
work. The library call also gets a failure guard the hand-rolled indexing
never needed but the ladder read does.
Verified while here: Math\Stat\Math.mqh's MathAbs/MathMax/MathSqrt/MathPow
and friends are ARRAY overloads, not scalar redefinitions, so pulling it
into the translation unit shadows no builtin.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 20:16:03 -04:00
# include <Math\Stat\Math.mqh>
refactor(baselines): the first real module - a class, not an #included partial
Baselines was 951 lines of CExpertSignalAIBase method bodies in a file
that only looked like a module. It is now CBaselineComparator: a class
the signal OWNS, which reads a CTrainingDataView and prints. It does
not name the signal anywhere in its code.
What the seam forced out into the open:
- Thirty-odd ArraySize() bounds tests, each carried by its caller, are
now one test per accessor next to the data. The two `hasValueN` and
one `arrowN` locals are gone with them.
- The -2.0 "never scored" sentinel on the arrow cache was tested at the
call site. It is now inside DataDirectionalCall, where it cannot be
read as a small confidence.
- DoubleToSignal needs m_outputNeuronsCount, so a raw double could not
be turned into a side by any reader. The view answers
DirectionalCall(bar, isBuy, magnitude) instead - the conversion
happens where the head width lives, and the module no longer needs
ENUM_SIGNAL at all.
- m_baselineDone was a latch on the signal for a decision only this
module makes. It is m_done, private, where it belongs.
Correction to my own earlier claim: I said Baselines had nine exclusive
members "polluting the signal class". It had none. m_x, m_f, m_ngrad,
m_AvgCE and the rest are FIELDS OF ALGLIB REPORT OBJECTS (state.m_x,
mrep.m_AvgCE) that my `\bm_\w+` scan matched after the dot. The module
needs no private state but its view pointer and that latch - which is
why it came out this cleanly.
The include sits below the g_ens* vote globals and the Alglib headers
it reads, because unlike the AIBase\*.mqh partials this is a real class
declaration compiled where it stands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 12:00:18 -04:00
//--- Alglib forest / least-squares for Training\BaselineComparator.mqh. Also pulls in statistics.mqh, where the
2026-08-20 09:49:33 -04:00
//--- signal-database ranking's significance tests come from - one include serves both.
feat(baselines): Alglib forest + linear on the NN's own matrix
Every direction verdict so far was measured through one architecture
family, so "flat" has two readings that no topology tuning can separate:
the net is the wrong learner, or the matrix carries no directional
information.
Two learners with completely different inductive biases - Alglib's
random decision forest and an ordinary least-squares fit - now train on
the SAME feature windows (BuildFeatureWindow, the net's own function, so
there is no second feature implementation to drift), the SAME labels,
the SAME IS/OOS split with both purges, and are scored through the SAME
precision-against-always-one-direction comparison and the same Sidak
family-wise arithmetic the deploy gate uses. If both also land at
chance, the matrix is the limit.
Deliberate choices, each of which could have made the comparison a
different question wearing this one's name:
- LRBuild, not LRBuildZ: the intercept absorbs the class imbalance, and
a baseline handicapped by a forced zero intercept would flatter the
net for the wrong reason.
- Raw call counts in the SE, matching the live gate's known-permissive
test rather than correcting it here - both sides must face the same
bar.
- No threshold sweep on the linear fit: a threshold fitted on the slice
being scored is the calibration leak the purged band exists to avoid.
- Uniform stride when a cap bites, not the newest N rows, so a score
difference cannot be a regime difference. What was dropped is logged.
Ships off (Run_Alglib_Baselines = false): it is a measurement, not a
trading feature, nothing trades on the answer and no model is saved.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 23:45:03 -04:00
# include <Math\Alglib\dataanalysis.mqh>
2026-08-20 09:49:33 -04:00
//--- FFT cross-correlation for the all-lags profile. Not reached by dataanalysis.mqh.
feat(baselines): combining-weight fit, MLP cross-validation, all-lags correlation
Three ALGLIB additions, all measurement-only and all under the existing
Run_Alglib_Baselines switch.
MinBLEIC COMBINING WEIGHTS. The live ensemble weights each member by its
own pooled holdout win rate - a defensible prior, but not a fit, and
nothing has ever asked what mixture minimises error on the bars the
members disagreed about. Two individually-mediocre members wrong in
different places can beat one individually better, and a per-member win
rate cannot express that because it never looks at them jointly.
Solved on the simplex (w >= 0, sum w = 1), which is exactly what
MinBLEIC is for. Non-negative because a negative weight asserts "trade
the opposite of this member", a claim ~60 effective observations cannot
support. Least squares on the signed outcome rather than precision:
precision is a STEP function of the threshold that no gradient method
can walk, and optimising a smooth proxy for a step decision is how
c3daded put every operating point 14pp underwater - so the result is
reported in BOTH currencies, the SSE it minimised and the directional
hit rate the mixture would actually have scored against the equal mix.
If the second does not improve, the first is noise.
This needed data that did not exist: g_ensVoteSum accumulates member
contributions and the sum destroys the decomposition, while
g_ensVoteVoterMask records only WHETHER a member voted, never what.
g_ensVoteMember[] keeps them unsummed. The live arithmetic is untouched.
ENS_MAX_MEMBERS is 8 and deliberately larger than MAX_AI_SIGNALS (5):
independent caps, over-allocating is free, and matching them would make
this array silently short the day the registry grows - a cap that has
already dropped a member once without saying so.
MLPKFoldCVLBFGS. Every baseline row carries a binomial SE, which is the
sampling error of SCORING a fixed model and says nothing about how much
the FIT moves. One LBFGS run from one random start can land anywhere,
and a baseline that cleared or missed the bar on luck of initialisation
reads exactly like one that did it on merit. 3 folds, because each is a
full retrain. LBFGS not LM - LM builds a Hessian over ~7,700 weights.
CCorr ALL-LAGS PROFILE. Added BESIDE the MI lag profile, not instead:
MI catches nonlinear dependence and is the stronger negative, which is
why it settled the verdict - but its per-lag permutation null limits it
to ~20 lags. FFT correlation gets every lag in one O(n log n) pass, so
linear structure parked at lag 300 would surface for free. Different
question, not a replacement. Walks CONTIGUOUS bars, unlike everything
else in this file, because a lag index is meaningless otherwise; both
series are mean-centred first since CorrR1D is a raw sum of products;
and the max over columns x lags is judged against a Sidak family of
exactly that size, not a bare 2-sigma line.
fasttransforms.mqh needed its own include - verified that none of
ap/optimization/statistics/solvers/linalg reaches it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 07:44:36 -04:00
# include <Math\Alglib\fasttransforms.mqh>
2026-08-20 09:49:33 -04:00
//--- Soft one-hot targets for the 3-neuron head. The per-neuron SIGMOID forward pass can saturate before
//--- the softmax normalises, and a literal 1.0/0.0 target it only approaches asymptotically grows
//--- weights toward the MAX_WEIGHT clamp. The excursion head trains on hard 1/0 instead.
2026-07-28 10:49:53 -04:00
# define LABEL_SMOOTH_HIGH 0.9
# define LABEL_SMOOTH_LOW 0.05
2026-08-20 09:49:33 -04:00
//--- Control-panel object namespace. CAppDialog names every control from the dialog name, so one prefix
//--- covers the tree. Declared here so it can appear in the chart-prefix sweep list below.
//--- (SIG_ARROW_PREFIX moved to ExpertSignalCustom.mqh when the classic signals started drawing too.)
fix: purge every EA object namespace on init and after deinit teardown
Leftover objects survived deinit because the cleanup list had drifted.
PurgeChart()'s own comment said it removed "our namespaced signal arrows
plus the status-label objects" while the code removed arrows ONLY, and
the panel prefix was swept at OnInit and nowhere else - so an ordinary
deinit left the status line, and any panel straggler, on the chart.
Three scattered call sites and a comment cannot be kept in step. There is
now ONE list - WarriorChartPrefixes() - covering arrows, status label and
panel, and one sweep, WarriorPurgeChartObjects(), used by every path.
Add a prefix there when a new object family appears and every cleanup
picks it up.
Two call sites added:
OnInit, before ANYTHING is drawn (including the status label it would
otherwise delete). Chart objects live in the chart PROFILE, not in the
EA, so they outlive the process: a deinit force-terminated at
MetaTrader's ~4,500 ms budget, a crash, a terminal kill, or an .ex5
replaced while attached all strand objects no later deinit will ever
own - and deleting the EA's files does not remove them, which is why
they read as corruption. Arrows are included: LoadChartSignals restores
them from their sidecar moments later and already opens with its own
arrow sweep, so this only removes orphans the sidecar does not account
for - the ones SaveChartSignals would otherwise ADOPT, since it rebuilds
that sidecar by scanning the chart.
OnDeinit, after ExtPanel.Destroy. Destroy walks an unbounded control
tree and ClearStatusLabel clears text rather than guaranteeing object
removal; either can leave a straggler and nothing looked afterwards.
Bounded work - three prefix deletes and one object-list scan - so it
respects the ordering rule that keeps the cheap visible cleanup ahead
of the heavy save. Arrows excluded: ShutdownChartCleanup already
persisted and removed them and re-deleting would race that write.
The two are complementary: the deinit sweep closes the ordinary case, the
OnInit purge closes the case where MetaTrader never let us finish. Only
the second can help after a starved shutdown.
Both sweeps rescan by name across EVERY object type and delete what the
bulk call missed. ObjectsDeleteAll's return has already been observed
disagreeing with a by-name scan of the same chart microseconds apart, and
object commands are queued on the chart rather than applied inline, so a
returned count is not evidence the objects are gone.
Panel create site now uses WARRIOR_PANEL_PREFIX instead of a literal, so
the name cannot drift away from the list that cleans it up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 16:19:26 -04:00
# define WARRIOR_PANEL_PREFIX " WarriorCP "
2026-08-20 09:49:33 -04:00
//--- Base of the PER-INSTANCE custom event id space for the training "study" event. Per-instance because
//--- a shared id made every member run a train chunk for every other member's event - N*N chunks, and a
//--- completely dead control panel - and because id 1 is the Controls library's own ON_DBL_CLICK.
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
# define STUDY_EVENT_ID_BASE 500
2026-08-20 09:49:33 -04:00
//--- An armed study event this old that never arrived is declared lost and re-armed. Generous: a queued
//--- event can legitimately wait tens of seconds behind a sibling's warm-up diagnostics.
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
# define STUDY_EVENT_LOST_MS 60000
2026-08-20 09:49:33 -04:00
//--- Next unassigned study-event id, claimed in the constructor - numbers this chart's members 0..N-1.
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
int g_warriorStudyEventSeq = 0 ;
//+------------------------------------------------------------------+
feat(ensemble): per-NN inputs replace the preset selector - the meta head becomes the vote's gate
User design (2026-08-19): 'remove the enum menu that selects neural networks... individual
inputs for every NN just like classic signals... the META NN should be integrated into the
voting decision pipeline when enabled... as a bonus meta labelling is applied to enabled NNs.'
- AI_CHOICE is GONE (tombstoned per the stale-.set doctrine). Use_MLP/Use_CONV/Use_LSTM/
Use_CONVLSTM are ordinary bools like the classic votes; the ensemble arithmetic adapts to
any subset because the consensus divisor is the enabled capable weight. Two or more
enabled = ensemble (|ENS1 token + joint gate, exactly the old AI_HYBRID fingerprints, so
existing weight files keep loading); one = the old solo preset; none = classic-only.
- Use_MetaLabeling un-couples META from the direction NNs (the old selector made them
mutually exclusive). S3 ships: CSignalMETA::LiveMetaGate scores each vote-cleared entry
(shared window at bar 1 + proposal descriptor: side, net vote, live geometry, spread/ATR;
pattern one-hot ZEROED - ranking, not calibrated probability, documented in the body) and
vetoes below the cost-adjusted break-even. Entries only; fail-open everywhere, loudly.
- COEXISTENCE HAZARDS closed: VoteCapableWeight()=0 and ProspectiveVote()=false for the
meta target - solo-only until today, a trained META would otherwise sit in the consensus
divisor as a permanent abstainer and shrink every vote by its module weight.
- CERTIFIED == TRADED: the ensemble era verdict replays the identical veto through the same
g_warriorMetaGate pointer over its OOS fired bars (bar re-resolved from the row's own
time; fail-open counted as fires and reported: 'metaGate: N approved, M vetoed, K
unscored'). The overlay deliberately does NOT replay it (veto-filter-in-replay class,
calendar-cliff precedent) - documented at the sweep site. Solo charts' own gate does not
model the veto - the standing solo-gate caveat, documented at the input.
- DB continuity: the pattern/journal DB fingerprint's first slot was (int)AIType;
DbLegacyAiSlot() maps every legacy-expressible config to its OLD value (new 2-3 member
subsets get 100+bitmask, outside the legacy range) so no existing database re-keys.
filterID becomes the enabled roster via one EnabledNNSummary().
- HUD: the meta line shows the gate (armed/(trn), last P vs BE, ok/veto tally); the
armed/disarmed announcement fires on state change via one latch (MetaGateArmedNow), not
only when an entry happens to be proposed.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 13:01:02 -04:00
//| ENSEMBLE CHART-LEVEL SHARED STATE (2+ direction NNs enabled). |
2026-08-22 00:24:45 -04:00
//| Era barrier, combined-vote OOS score and warm-up sharing are |
//| chart-level because they are questions about what gets TRADED. |
2026-08-20 09:49:33 -04:00
//| Solo charts register nothing and none of it runs. |
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
//+------------------------------------------------------------------+
class CExpertSignalAIBase ;
CExpertSignalAIBase * g_warriorEnsemble [ ] ;
2026-08-22 00:24:45 -04:00
//--- Combined-vote OOS rows for the current era; a stale-era contribution resets the buffer.
//--- TWO masks: Mask = who EVALUATED this bar, VoterMask = who cast a NON-ZERO vote. VoterMask is the
//--- divisor, because Direction() skips abstentions in both the sum and the count.
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
datetime g_ensVoteTime [ ] ;
double g_ensVoteSum [ ] ;
2026-08-20 09:49:33 -04:00
//--- The same contributions UNSUMMED, one slot per member per bar - the decomposition g_ensVoteSum
refactor(baselines): the first real module - a class, not an #included partial
Baselines was 951 lines of CExpertSignalAIBase method bodies in a file
that only looked like a module. It is now CBaselineComparator: a class
the signal OWNS, which reads a CTrainingDataView and prints. It does
not name the signal anywhere in its code.
What the seam forced out into the open:
- Thirty-odd ArraySize() bounds tests, each carried by its caller, are
now one test per accessor next to the data. The two `hasValueN` and
one `arrowN` locals are gone with them.
- The -2.0 "never scored" sentinel on the arrow cache was tested at the
call site. It is now inside DataDirectionalCall, where it cannot be
read as a small confidence.
- DoubleToSignal needs m_outputNeuronsCount, so a raw double could not
be turned into a side by any reader. The view answers
DirectionalCall(bar, isBuy, magnitude) instead - the conversion
happens where the head width lives, and the module no longer needs
ENUM_SIGNAL at all.
- m_baselineDone was a latch on the signal for a decision only this
module makes. It is m_done, private, where it belongs.
Correction to my own earlier claim: I said Baselines had nine exclusive
members "polluting the signal class". It had none. m_x, m_f, m_ngrad,
m_AvgCE and the rest are FIELDS OF ALGLIB REPORT OBJECTS (state.m_x,
mrep.m_AvgCE) that my `\bm_\w+` scan matched after the dot. The module
needs no private state but its view pointer and that latch - which is
why it came out this cleanly.
The include sits below the g_ens* vote globals and the Alglib headers
it reads, because unlike the AIBase\*.mqh partials this is a real class
declaration compiled where it stands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 12:00:18 -04:00
//--- destroys. Read by the combining-weight fit in Training\BaselineComparator.mqh.
feat(baselines): combining-weight fit, MLP cross-validation, all-lags correlation
Three ALGLIB additions, all measurement-only and all under the existing
Run_Alglib_Baselines switch.
MinBLEIC COMBINING WEIGHTS. The live ensemble weights each member by its
own pooled holdout win rate - a defensible prior, but not a fit, and
nothing has ever asked what mixture minimises error on the bars the
members disagreed about. Two individually-mediocre members wrong in
different places can beat one individually better, and a per-member win
rate cannot express that because it never looks at them jointly.
Solved on the simplex (w >= 0, sum w = 1), which is exactly what
MinBLEIC is for. Non-negative because a negative weight asserts "trade
the opposite of this member", a claim ~60 effective observations cannot
support. Least squares on the signed outcome rather than precision:
precision is a STEP function of the threshold that no gradient method
can walk, and optimising a smooth proxy for a step decision is how
c3daded put every operating point 14pp underwater - so the result is
reported in BOTH currencies, the SSE it minimised and the directional
hit rate the mixture would actually have scored against the equal mix.
If the second does not improve, the first is noise.
This needed data that did not exist: g_ensVoteSum accumulates member
contributions and the sum destroys the decomposition, while
g_ensVoteVoterMask records only WHETHER a member voted, never what.
g_ensVoteMember[] keeps them unsummed. The live arithmetic is untouched.
ENS_MAX_MEMBERS is 8 and deliberately larger than MAX_AI_SIGNALS (5):
independent caps, over-allocating is free, and matching them would make
this array silently short the day the registry grows - a cap that has
already dropped a member once without saying so.
MLPKFoldCVLBFGS. Every baseline row carries a binomial SE, which is the
sampling error of SCORING a fixed model and says nothing about how much
the FIT moves. One LBFGS run from one random start can land anywhere,
and a baseline that cleared or missed the bar on luck of initialisation
reads exactly like one that did it on merit. 3 folds, because each is a
full retrain. LBFGS not LM - LM builds a Hessian over ~7,700 weights.
CCorr ALL-LAGS PROFILE. Added BESIDE the MI lag profile, not instead:
MI catches nonlinear dependence and is the stronger negative, which is
why it settled the verdict - but its per-lag permutation null limits it
to ~20 lags. FFT correlation gets every lag in one O(n log n) pass, so
linear structure parked at lag 300 would surface for free. Different
question, not a replacement. Walks CONTIGUOUS bars, unlike everything
else in this file, because a lag index is meaningless otherwise; both
series are mean-centred first since CorrR1D is a raw sum of products;
and the max over columns x lags is judged against a Sidak family of
exactly that size, not a bare 2-sigma line.
fasttransforms.mqh needed its own include - verified that none of
ap/optimization/statistics/solvers/linalg reaches it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 07:44:36 -04:00
double g_ensVoteMember [ ] ;
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
int g_ensVoteMask [ ] ;
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
int g_ensVoteVoterMask [ ] ;
2026-08-20 09:49:33 -04:00
//--- The weighted mean's DIVISOR, carried per row rather than recomputed at verdict time: a member's
//--- m_weight can be rewritten by UpdateSignalsWeights() between the scan and the verdict, and the
//--- divisor must be the one in force when the numerator was accumulated.
feat(vote): thresholds become confidence percentages, on ONE scale everywhere
User request: "the entry/exit thresholds are manual numbers, I would like
them to be confidence percentages, so the current 20 would be only 20%
confidence in a profitable trade."
WHY 20 WAS EVER SENSIBLE. Under UseDatabaseRanking both factors of a filter's
contribution are win rates: the pattern weight is that pattern's measured win
rate (UpdateSignalsWeights -> ApplyPatternWeight) and m_weight is the filter's
average win rate over its patterns, /100. Dividing the sum by the VOTER COUNT
therefore produced a mean of PRODUCTS of two win rates - a genuinely
60%-accurate filter firing a 60% pattern scored 0.60 x 60 = 36. The number was
never on a probability scale, so its magnitude meant nothing on its own.
Dividing by Sum(m_weight) instead makes it a weighted MEAN of win rates, which
is a win rate: result = Sum(w_i*p_i)/Sum(w_i). Every voter at 60% now reads 60;
MACD's double-divergence pattern (weight 100) voting alone reads 100. m_weight
stops being a discount on the probability and becomes how much a filter's
opinion COUNTS - which is what a module weight should always have been.
Default Min_Vote_Open 20 -> 50: not a tightening, the same bar re-expressed.
ONE SCALE, EVERYWHERE - the part that made this bigger than a rescale. Three
other places compared against a 0..1 softmax confidence and would each have
become a fresh currency mismatch the moment the input changed meaning:
* the AI early-exit route (LiveSignedConfidence vs m_ai_exit_threshold) now
reads m_lastAiVote - the AI filters' own weighted mean, undiluted by the
classic side, which is the only reason that route exists - against the
same m_threshold_close the averaged vote uses. m_ai_exit_threshold is
retired rather than left dangling.
* m_oosDecisionSeries now carries the vote, not the confidence, so the exit
SIMULATION stops modelling a close rule the EA does not run.
* ExitPolicy() clamped anything > 1.0 to zero. Passing the unscaled input
through that would have silently switched vote exits off in the
simulation while live went on running them - found before it shipped;
the bound now tracks the scale.
LiveSignedConfidence() is deliberately untouched and still 0..1: MM sizing,
SL/TP scaling and the intelligent trailing want a model confidence, not a win
rate.
CALIBRATION CAVEAT, stated in the code where the claim is made: this is only a
real probability to the extent the pattern weights are. A pattern with fewer
than MIN_TRADES_FOR_WIN_RATE journaled trades keeps its DEFAULT weight - a
designed prior (25/50/75/100 for the AI tiers), not a measurement. Until the
signal DB fills, "60" means "the designed conviction of the patterns that
fired". Closing that gap is the next commit.
Also corrects VOTE_CLOSE_PRESETS' comment, which documented the two scales
this removes.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:52:08 -04:00
double g_ensVoteWeightSum [ ] ;
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
bool g_ensVoteLabelBuy [ ] ; // the bar's swing label, per side - the vote's truth and its chance rate
bool g_ensVoteLabelSell [ ] ;
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
bool g_ensVoteDirLabel [ ] ; // bar carried a Buy/Sell label - the coverage floor's base rate
feat(gate): measure what a call was WORTH, not only how often it was right
The ensemble deploy gate certifies PRECISION against a chance rate and has
never known whether a correct call pays for its own spread. Every verdict this
project has recorded - 33% precision against a 14% chance rate, an edge that
clears its exact-binomial bar comfortably - is silent on the one question that
decides whether any of it is tradeable, and the cost boundary is exactly where
several earlier edges died with their precision already believed.
Adds a per-row payoff measurement, taken once per ROW (a chart property, not a
member one) at the same time the label is written:
* forward close move over K = round(SwingLifespanEstimate()) bars,
* the up and down extreme excursions over the same window,
each divided by the bar's own ATR. K is deliberately the label lifespan the
effective-sample-size deflation already uses, so precision and payoff describe
the same window and can be read in one sentence.
POLICY-FREE: no stop, no target, no trailing rule. It measures the SIGNAL, not
a trade-management choice layered on top - exit shaping moves payoff around
without creating any, so mixing the two would hide which was responsible.
Stored unsigned by direction; the sign comes from the vote at verdict time, and
a short's excursions SWAP rather than negate - negating them would report a
short's worst case as a negative best case.
The newest K bars of the OOS slice have no forward window and are dropped from
the tally with their own denominator, never counted as a zero move: that is the
leading-edge trap that made the lag profile's first run a false positive.
The era verdict now prints mean R, MFE and MAE at the certified rung against
the spread in the same ATR units. It GATES NOTHING - wiring a policy to an
unvalidated payoff number is how a measurement becomes a decision before anyone
has checked it.
Build tag payoff-v1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 07:14:15 -04:00
//--- WHAT THE CALL WAS WORTH, in ATR units at the signal bar. The deploy gate certifies PRECISION
//--- against a chance rate and has never known whether a correct call pays for its own spread - and
//--- this project has watched several edges die on exactly that gap, at the point where precision was
//--- already believed. Measured over a FIXED horizon of round(SwingLifespanEstimate()) bars, the same
//--- label lifespan the effective-sample-size deflation uses, so the precision number and the payoff
//--- number describe the SAME window and can be read in one sentence.
//---
//--- POLICY-FREE BY CONSTRUCTION: no stop, no target, no trailing rule. This measures the SIGNAL, not
//--- a trade-management choice layered on top of it - exit shaping moves payoff around without
//--- creating any (see the exit-management verdict), so mixing the two here would only hide which of
//--- them was responsible. Stored unsigned by direction (up and down excursions kept apart); the sign
//--- is applied at verdict time from the vote's own direction, so one row serves a long read and a
//--- short read identically and no row has to be measured twice.
2026-08-27 07:19:40 -04:00
//--- TWO HORIZONS, BECAUSE ONE OF THEM CANNOT ANSWER THE QUESTION. The label fires when a pivot
//--- lands WITHIN PIVOT_LABEL_TOLERANCE_BARS bars - so at that horizon the pivot may only just have
//--- happened, and a perfectly correct call can still show a negative forward move because the turn
//--- it predicted has not had a single bar to run yet. Measuring only there would understate, and
//--- could invert, the payoff of a signal that is working exactly as designed.
//--- SHORT = PIVOT_LABEL_TOLERANCE_BARS: "has the pivot arrived". A control, not the answer.
//--- HOLD = that plus the median ZigZag leg: the pivot, PLUS the leg it opens. What a trade on
//--- this call would actually be held for, and the horizon the payoff belongs to.
//--- Reporting both is also the guard against picking one and calling it the truth - this project
//--- has already had a break-even conclusion overturned purely by getting a horizon wrong.
feat(gate): measure what a call was WORTH, not only how often it was right
The ensemble deploy gate certifies PRECISION against a chance rate and has
never known whether a correct call pays for its own spread. Every verdict this
project has recorded - 33% precision against a 14% chance rate, an edge that
clears its exact-binomial bar comfortably - is silent on the one question that
decides whether any of it is tradeable, and the cost boundary is exactly where
several earlier edges died with their precision already believed.
Adds a per-row payoff measurement, taken once per ROW (a chart property, not a
member one) at the same time the label is written:
* forward close move over K = round(SwingLifespanEstimate()) bars,
* the up and down extreme excursions over the same window,
each divided by the bar's own ATR. K is deliberately the label lifespan the
effective-sample-size deflation already uses, so precision and payoff describe
the same window and can be read in one sentence.
POLICY-FREE: no stop, no target, no trailing rule. It measures the SIGNAL, not
a trade-management choice layered on top - exit shaping moves payoff around
without creating any, so mixing the two would hide which was responsible.
Stored unsigned by direction; the sign comes from the vote at verdict time, and
a short's excursions SWAP rather than negate - negating them would report a
short's worst case as a negative best case.
The newest K bars of the OOS slice have no forward window and are dropped from
the tally with their own denominator, never counted as a zero move: that is the
leading-edge trap that made the lag profile's first run a false positive.
The era verdict now prints mean R, MFE and MAE at the certified rung against
the spread in the same ATR units. It GATES NOTHING - wiring a policy to an
unvalidated payoff number is how a measurement becomes a decision before anyone
has checked it.
Build tag payoff-v1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 07:14:15 -04:00
double g_ensVoteFwdR [ ] ; // (close[bar-K] - close[bar]) / ATR[bar], signed by PRICE not by vote
double g_ensVoteUpR [ ] ; // (max high over the K forward bars - close[bar]) / ATR[bar], >= 0
double g_ensVoteDnR [ ] ; // (close[bar] - min low over the K forward bars) / ATR[bar], >= 0
2026-08-27 07:19:40 -04:00
double g_ensVoteFwdR2 [ ] ; // the same three at the HOLD horizon
double g_ensVoteUpR2 [ ] ;
double g_ensVoteDnR2 [ ] ;
bool g_ensVoteHasR2 [ ] ; // its own flag: the longer window runs off the leading edge sooner
diag(label): expose bars-to-pivot - the confound the payoff test was missing
CORRECTION to what the payoff instrument was measuring. The 5-bar horizon looked
like the powered test and it is confounded.
SwingPivotDirectionLabel returns Buy when a swing LOW lands up to
PIVOT_LABEL_TOLERANCE_BARS bars AHEAD, and says the quiet part itself: gating on
where the pivot sits relative to entry "would drop exactly the bars where the
turn has not finished coming to us", and how much adverse move remains before
the turn "is a trade-management question".
So on a CORRECT Buy call price is often still falling for d more bars. A window
shorter than d measures the APPROACH, not the leg, and its negative contribution
is expected on the calls that are RIGHT. The tight null at 5 bars
(-0.012 +/- 0.074) is therefore not evidence of no payoff. Neither horizon is
both clean and powered: 5 bars is powered and confounded, 18-19 is clean and has
an SE of 0.277.
(idx - P1) was computed in the label and thrown away. Now cached beside
m_labelResolveAge under the same validity flag, and bucketed in the era verdict.
DELIBERATELY NOT USED AS A PER-CALL HORIZON, which is the trap sitting right
next to this: d exists only on bars the label found a pivot for, so a horizon
that varied with d would hand correct and incorrect calls different windows and
bias the comparison outright. The horizon stays fixed; d only buckets.
The bucket for "the label called no pivot here" is reported by name rather than
folded in, because it is the control the others are read against. Buckets 1..N
condition on the label, so they describe the MECHANISM, not what a book earns.
Reads: rising with d means the edge is in EARLY calls and the tolerance window
is spending it - fixable by reweighting the loss, not by a new label. Flat means
that hypothesis dies.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 08:16:47 -04:00
//--- Bars from the row's bar to the pivot its LABEL calls, -1 where it calls none. Used only to
//--- BUCKET the payoff diagnostic - never to choose a per-call horizon, which would be a trap: d
//--- exists only on bars the label got a pivot for, so a horizon that varied with d would give
//--- correct and incorrect calls different windows and bias the comparison outright.
int g_ensVoteD [ ] ;
feat(gate): measure what a call was WORTH, not only how often it was right
The ensemble deploy gate certifies PRECISION against a chance rate and has
never known whether a correct call pays for its own spread. Every verdict this
project has recorded - 33% precision against a 14% chance rate, an edge that
clears its exact-binomial bar comfortably - is silent on the one question that
decides whether any of it is tradeable, and the cost boundary is exactly where
several earlier edges died with their precision already believed.
Adds a per-row payoff measurement, taken once per ROW (a chart property, not a
member one) at the same time the label is written:
* forward close move over K = round(SwingLifespanEstimate()) bars,
* the up and down extreme excursions over the same window,
each divided by the bar's own ATR. K is deliberately the label lifespan the
effective-sample-size deflation already uses, so precision and payoff describe
the same window and can be read in one sentence.
POLICY-FREE: no stop, no target, no trailing rule. It measures the SIGNAL, not
a trade-management choice layered on top - exit shaping moves payoff around
without creating any, so mixing the two would hide which was responsible.
Stored unsigned by direction; the sign comes from the vote at verdict time, and
a short's excursions SWAP rather than negate - negating them would report a
short's worst case as a negative best case.
The newest K bars of the OOS slice have no forward window and are dropped from
the tally with their own denominator, never counted as a zero move: that is the
leading-edge trap that made the lag profile's first run a false positive.
The era verdict now prints mean R, MFE and MAE at the certified rung against
the spread in the same ATR units. It GATES NOTHING - wiring a policy to an
unvalidated payoff number is how a measurement becomes a decision before anyone
has checked it.
Build tag payoff-v1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 07:14:15 -04:00
//--- FALSE at the NEWEST K bars of the OOS slice, which have no forward window yet, and whenever the
//--- ATR or a bar in the window is unusable. Those rows are DROPPED from the payoff tally rather than
//--- counted as a zero move - the same leading-edge trap that made the lag profile's first run a
//--- spectacular false positive (4113afd/bbe26a0).
bool g_ensVoteHasR [ ] ;
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
int g_ensVoteRows = 0 ;
long g_ensVoteEra = -1 ;
int g_ensVoteDoneMask = 0 ;
int g_ensVoteCursor [ 8 ] ; // per-member monotonic row cursor (members scan bars in the same order)
2026-08-20 09:49:33 -04:00
//--- Deliberately LARGER than MAX_AI_SIGNALS (5): independent caps, and over-allocating is free, whereas
//--- matching would make this array silently too small the day the registry grows.
feat(baselines): combining-weight fit, MLP cross-validation, all-lags correlation
Three ALGLIB additions, all measurement-only and all under the existing
Run_Alglib_Baselines switch.
MinBLEIC COMBINING WEIGHTS. The live ensemble weights each member by its
own pooled holdout win rate - a defensible prior, but not a fit, and
nothing has ever asked what mixture minimises error on the bars the
members disagreed about. Two individually-mediocre members wrong in
different places can beat one individually better, and a per-member win
rate cannot express that because it never looks at them jointly.
Solved on the simplex (w >= 0, sum w = 1), which is exactly what
MinBLEIC is for. Non-negative because a negative weight asserts "trade
the opposite of this member", a claim ~60 effective observations cannot
support. Least squares on the signed outcome rather than precision:
precision is a STEP function of the threshold that no gradient method
can walk, and optimising a smooth proxy for a step decision is how
c3daded put every operating point 14pp underwater - so the result is
reported in BOTH currencies, the SSE it minimised and the directional
hit rate the mixture would actually have scored against the equal mix.
If the second does not improve, the first is noise.
This needed data that did not exist: g_ensVoteSum accumulates member
contributions and the sum destroys the decomposition, while
g_ensVoteVoterMask records only WHETHER a member voted, never what.
g_ensVoteMember[] keeps them unsummed. The live arithmetic is untouched.
ENS_MAX_MEMBERS is 8 and deliberately larger than MAX_AI_SIGNALS (5):
independent caps, over-allocating is free, and matching them would make
this array silently short the day the registry grows - a cap that has
already dropped a member once without saying so.
MLPKFoldCVLBFGS. Every baseline row carries a binomial SE, which is the
sampling error of SCORING a fixed model and says nothing about how much
the FIT moves. One LBFGS run from one random start can land anywhere,
and a baseline that cleared or missed the bar on luck of initialisation
reads exactly like one that did it on merit. 3 folds, because each is a
full retrain. LBFGS not LM - LM builds a Hessian over ~7,700 weights.
CCorr ALL-LAGS PROFILE. Added BESIDE the MI lag profile, not instead:
MI catches nonlinear dependence and is the stronger negative, which is
why it settled the verdict - but its per-lag permutation null limits it
to ~20 lags. FFT correlation gets every lag in one O(n log n) pass, so
linear structure parked at lag 300 would surface for free. Different
question, not a replacement. Walks CONTIGUOUS bars, unlike everything
else in this file, because a lag index is meaningless otherwise; both
series are mean-centred first since CorrR1D is a raw sum of products;
and the max over columns x lags is judged against a Sidak family of
exactly that size, not a bare 2-sigma line.
fasttransforms.mqh needed its own include - verified that none of
ap/optimization/statistics/solvers/linalg reaches it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 07:44:36 -04:00
# define ENS_MAX_MEMBERS 8
2026-08-20 09:49:33 -04:00
//+------------------------------------------------------------------+
//| Population count over the member masks. Bounded by the 8-slot |
//| ensemble, so a plain loop is both clearest and fastest. |
//+------------------------------------------------------------------+
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
int EnsembleBitCount ( const int mask )
{
int n = 0 ;
for ( int b = 0 ; b < 8 ; b + + )
if ( ( mask & ( 1 < < b ) ) ! = 0 )
n + + ;
return n ;
}
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
//+------------------------------------------------------------------+
2026-08-22 00:30:14 -04:00
//| ENSEMBLE DEPLOY GATE. In ensemble mode THE UNIT OF EVALUATION IS |
//| THE VOTE: best era, checkpointing, give-up and deploy all move |
//| here, because all four ask what gets TRADED. |
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
//+------------------------------------------------------------------+
double g_ensBestScore = -1.0 ; // best combined-vote selection score (precision x coverage credit)
bool g_ensBestTradeable = false ; // did that era clear the vote's own deployability floor
bool g_ensBestTwoSided = false ; // did it fire both long and short
double g_ensBestPrecPct = -1.0 ; // the winning era's vote win rate, for the family-wise test
double g_ensBestChancePct = -1.0 ; // and its chance reference
int g_ensBestCalls = 0 ; // and the n that sets the standard error
2026-08-26 01:14:51 -04:00
//--- WHICH of the deployability conditions the best era actually failed. The stage-3 refusal used to
//--- say "no era's combined vote ever cleared the deployability floor" and then list all THREE
//--- conditions in one parenthesis without saying which one fired - so an operator reading it could
//--- not tell a coverage problem from a precision problem from a one-sided book, and the three have
//--- nothing in common as fixes. Cost real time to diagnose by hand 2026-08-26, when the answer was
//--- coverage every time. Same doctrine as CTrainPoolReader::Announce's reject list: a refusal that
//--- will not say WHY is the failure this project has already paid for under several other names.
feat(vote): derive the threshold instead of configuring it
Signal_ThresholdOpen becomes a seed. The era verdict now picks the HIGHEST
sweep rung whose vote still clears the whole deploy gate - coverage floor,
exact-binomial precision bar and two-sidedness together - computes the era's
verdict AT that rung, and publishes it to the live signal's m_threshold_open
so the bar the gate certifies is the bar the EA trades.
Measured on 619 era verdicts across all six live charts:
* every era on every symbol had at least one rung clearing the full gate.
At the fixed 25% the fleet was actually running, four of six symbols had
none, ever. The threshold, not the models, was the blocker.
* walk-forward (rung derived on era N, scored on era N+1): 10.2% coverage /
31.8% precision, against an oracle re-picking on N+1 of 10.3% / 31.7%.
Near-zero shrinkage - a measurement, not a fit. It holds because the
binding constraint is COVERAGE, a near-deterministic step function of the
vote distribution, not precision.
* vs a fixed 15% (best global value): +0.6pp precision, 3.4pp less coverage.
vs a fixed 20%: deployable on all six rather than four of six.
Selection on the highest PASSING rung, never on the best-precision rung - that
is a best-of-6 on a noisy statistic and this project has crowned noise that way
four times. The multiplicity that remains is paid for: nTried in
EnsembleSurvivesSelection is now eras x rungs. Costs nothing - all six charts
clear it by 6.5-12 sigma even forming z on effective rather than raw calls.
Also fixes, in the same path: the direction-policy gate is hoisted above the
per-rung tally so every rung is scored on the population the gate certifies.
Retrain-neutral: not in BuildModelFingerprint(), no .nnw re-keyed.
Compiled clean; NOT yet run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 09:02:58 -04:00
//--- THE VOTE THRESHOLD'S CANDIDATE RUNGS. Rungs of PERCENTAGE_PRESETS, so every value here is one
//--- an operator could also have selected by hand. NO LONGER DIAGNOSTIC (2026-08-26): the era verdict
//--- now DERIVES the threshold from these instead of reading Signal_ThresholdOpen - see
//--- the THE DERIVED THRESHOLD block in EnsembleEraVerdict() for the rule and the measurement.
2026-08-26 04:17:27 -04:00
# define ENS_THRESHOLD_SWEEP_N 6
const double g_ensThresholdSweep [ ENS_THRESHOLD_SWEEP_N ] = { 5.0 , 10.0 , 15.0 , 20.0 , 25.0 , 30.0 } ;
feat(vote): derive the threshold instead of configuring it
Signal_ThresholdOpen becomes a seed. The era verdict now picks the HIGHEST
sweep rung whose vote still clears the whole deploy gate - coverage floor,
exact-binomial precision bar and two-sidedness together - computes the era's
verdict AT that rung, and publishes it to the live signal's m_threshold_open
so the bar the gate certifies is the bar the EA trades.
Measured on 619 era verdicts across all six live charts:
* every era on every symbol had at least one rung clearing the full gate.
At the fixed 25% the fleet was actually running, four of six symbols had
none, ever. The threshold, not the models, was the blocker.
* walk-forward (rung derived on era N, scored on era N+1): 10.2% coverage /
31.8% precision, against an oracle re-picking on N+1 of 10.3% / 31.7%.
Near-zero shrinkage - a measurement, not a fit. It holds because the
binding constraint is COVERAGE, a near-deterministic step function of the
vote distribution, not precision.
* vs a fixed 15% (best global value): +0.6pp precision, 3.4pp less coverage.
vs a fixed 20%: deployable on all six rather than four of six.
Selection on the highest PASSING rung, never on the best-precision rung - that
is a best-of-6 on a noisy statistic and this project has crowned noise that way
four times. The multiplicity that remains is paid for: nTried in
EnsembleSurvivesSelection is now eras x rungs. Costs nothing - all six charts
clear it by 6.5-12 sigma even forming z on effective rather than raw calls.
Also fixes, in the same path: the direction-policy gate is hoisted above the
per-rung tally so every rung is scored on the population the gate certifies.
Retrain-neutral: not in BuildModelFingerprint(), no .nnw re-keyed.
Compiled clean; NOT yet run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 09:02:58 -04:00
//--- The rung the era verdict derived, -1 before the first scored era. Published to the LIVE signal's
//--- m_threshold_open by Warrior_EA.mq5 so the bar the gate certifies is the bar the EA trades - the
//--- certified!=traded defect this project has already paid for once (2c443ba).
double g_ensDerivedThreshold = -1.0 ;
fix(chart): stale combined-vote arrows survived every wipe, because two files lived outside Warrior_EA\
Operator report: arrows labelled as restored from a previous session on a
fleet training from era 0. Confirmed - all six charts restored 115-431
combined-vote arrows drawn by models that no longer exist.
TWO INDEPENDENT DEFECTS, either of which alone causes it.
1. CVoteArrowStore::Discard() HAD NO CALLER.
The member-scoped .arrows file is cleared by ClearPersistedChartSignals on a
fresh topology. The CHART-scoped .votearrows store has an equivalent
Discard(), written for exactly this, and nothing ever called it. The store
is keyed on the DB config fingerprint, which does not move when a model is
wiped, so it reloaded across any reset - fresh topology, panel weight reset,
or a model-file wipe.
A vote is a claim made by a specific set of members. If any member rebuilt
from scratch this run, the whole stored history is void, so
g_warriorFreshTopologyThisRun is now raised wherever a member discards
weights or builds a fresh topology, and the store Discards instead of Loads.
2. TWO WARRIOR FILES LIVED OUTSIDE Warrior_EA\.
.sigvis and .votearrows were written to the ROOT of Common\Files, outside
the one directory that "wipe the Warrior EA files" has always meant. Two
consecutive wipes this session left them standing untouched, and neither
wipe was as fresh as reported. Both now live under Warrior_EA\ChartState\.
A wipe that does not remove all of a program's state is not a wipe, and
nothing in the log told the operator which files were missed.
NOTE for anyone re-running the wipe: pre-existing WarriorVote_*.votearrows and
Warrior_EA_*.sigvis in the Common\Files ROOT are orphaned by this change and
should be deleted once.
Build tag -> fleet-pool-v3.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 21:07:52 -04:00
//--- SET BY ANY MEMBER THAT REBUILT FROM SCRATCH THIS RUN. The COMBINED-VOTE arrow store is
//--- chart-scoped and keyed on the DB config fingerprint, which does not move when a model is
//--- wiped - so it happily restored arrows drawn by models that no longer exist (observed
//--- 2026-08-26: six charts restored 115-431 vote arrows onto a fleet training from era 0).
//--- CVoteArrowStore::Discard() existed for exactly this and had NO CALLER. A vote is a claim made
//--- by a specific set of members; if any one of them is fresh, the whole history is void.
bool g_warriorFreshTopologyThisRun = false ;
2026-08-26 01:14:51 -04:00
double g_ensBestCoveragePct = -1.0 ; // what the best era's vote actually fired on
double g_ensBestMinCoverPct = -1.0 ; // the floor it had to clear
double g_ensBestEdgeFloorPct = -1.0 ; // and the precision bar, so all three are reportable
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
long g_ensBestEra = -1 ;
int g_ensCandidateEras = 0 ; // N for the family-wise correction: eras that COULD have won
int g_ensErasSinceBest = 0 ; // shared plateau counter
int g_ensPlateauStage = 0 ; // shared plateau stage
2026-08-20 09:49:33 -04:00
//--- Which best-era the family-wise deploy test has already run against, -1 = none. Without it the
2026-08-22 00:24:45 -04:00
//--- all-members-plateaued shortcut re-ran the gate against an unchanged best every era, incrementing
//--- the candidate count the correction divides by - the run spent its time RAISING ITS OWN BAR.
fix(gate): the plateau shortcut re-ran the deploy test every era, raising its own bar
User report: 'eras since best' in the ensemble line is always 0 (era 147, best at era 90,
'0 eras ago'). That is a control-flow bug wearing a display symptom.
Once every member's in-sample error had plateaued, the shortcut forced the ladder to its
DEPLOY stage on EVERY era. The failed-gate branch resets the stage to 0 so the ladder can
climb again - so the shortcut raised it, the branch cleared it, forever. Three consequences,
only the first of which was visible:
- g_ensErasSinceBest was reset every era, pinning the counter at 0.
- The stage-1/2 boosted warm restarts were never reached, so the one mechanism that can
un-plateau a stuck member never ran. The models sat at a WORSE error than their best
(0.2408 -> 0.3015 on PAI) with no escape.
- Every repetition ran EnsembleSurvivesSelection against an unchanged best and incremented
the candidate-era count the family-wise correction divides by. The run spent its time
RAISING ITS OWN SIDAK BAR - the same waste as the 2026-08-18 inert IS-error stop, one
layer up, and the reason a gate that needed >47.8% saw its bar climb era after era.
Fix: the shortcut fires ONCE PER BEST-ERA (g_ensGateTestedEra, stamped before the outcome
branches because it is the re-running that inflates the family, pass or fail). A refused
gate now falls back to the normal counter-driven ladder - warm restart, anneal, then a
fresh deploy test - which is the escape the shortcut was skipping.
Also, per user: the signal marks were too small to see. Span doubled (2.6 bar widths, so
the overhang either side of the candle is ~0.8 bars) and both layers thickened - 1px dotted
was invisible on a candle chart at any realistic zoom.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 15:40:52 -04:00
long g_ensGateTestedEra = -1 ;
2026-08-20 09:49:33 -04:00
//--- One-shot latch so the collective IS-error plateau announces once per run, not once per era.
fix(plateau): the IS-error early stop was inert for every ensemble member
15 hours of training, and the stop that exists to END a run announced itself
1,299 consecutive times without ending anything:
SP500 ConvLSTM IN-SAMPLE ERROR PLATEAU - not improved in 1297 / 1298 / 1299
eras (best 0.2689, now 0.3269) ... era 1396, 1397, 1398
SP500 LSTM 536 eras SP500 CONV 442 eras SP500 PAI 150 eras
XAUUSD HYB 478 eras XAUUSD LSTM 296 eras XAUUSD CONV 366 eras
CAUSE: it wrote its decision into m_plateauStage, and EnsembleEraVerdict mirrors
the shared ladder onto every member - `mm.m_plateauStage = g_ensPlateauStage` -
on EVERY era, purely so each member's status line shows the collective stage. A
display mirror was silently overwriting a decision, so the stop re-armed and
re-fired the next era, forever.
This is the worst possible direction for this particular bug. Every one of those
1,299 eras was scored out of sample and joined the family the deploy gate
corrects over (Sidak, g_ensCandidateEras). The stop's entire purpose is to make
that family SMALLER; instead the run spent fifteen hours raising its own bar.
- m_isErrorPlateaued: a one-way per-member latch, cleared only by a fresh run.
Nothing in the ladder may reset it. The stop condition and the two solo deploy
conditions read the latch, not the mirrored stage.
- The orchestrator combines: EnsembleEraVerdict requires UNANIMITY across
participating members (same participation test the era barrier uses, so an
excluded or finished member cannot veto). One member still learning can still
move the combined vote, and the vote is what the gate certifies.
- Fed in as `dueStage = PLATEAU_STAGE_DEPLOY`, NOT written to g_ensPlateauStage.
The block that actually ends the run sits under `dueStage > g_ensPlateauStage`,
so assigning the stage directly makes that test false and the deploy never
happens - the same inert-write shape as the bug being fixed. Caught before
committing; raising dueStage carries it through the ladder's own path (warm
restarts skipped, family-wise vote test, measurement screen, joint checkpoint)
unchanged.
- g_ensIsPlateauAnnounced: announce once per run, not once per era.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 13:09:26 -04:00
bool g_ensIsPlateauAnnounced = false ;
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
bool g_ensDeployApproved = false ; // stage 3 reached AND the vote cleared the family-wise gate
long g_ensLastVerdictEra = -1 ; // guards against scoring one era twice
2026-08-20 09:49:33 -04:00
//--- Lifetime combined-vote win rate over every bar the VOTE fired on, in the SAME shape as a solo
//--- model's m_cumOosCorrect/m_cumOosTotal so the panel reads identically. Session-scoped like the rest
//--- of the g_ens* ladder state.
2026-08-16 21:08:41 -04:00
long g_ensCumOosCorrect = 0 ;
long g_ensCumOosTotal = 0 ;
2026-08-20 09:49:33 -04:00
//--- Mirror of Signal_ThresholdOpen, pushed in at registration so the combined-vote scorer fires on the
//--- same criterion the live trade does. UNITS are the 0..100 VOTE scale, not a confidence percentage -
//--- see LiveVoteContribution(). The seed is only read before registration overwrites it.
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
double g_ensembleVoteThreshold = 60.0 ;
fix(panel,arrows): one deploy predicate, a deployed-only readout, and persist the vote arrows
Four reported symptoms, three of them one root cause: the ensemble's
certified record was session-scoped and written ONLY at pass-3
completion. A deployed ensemble runs no further eras, so every restart
lost the aggregate win rate, the aggregate panel line and the overlay
snapshots - and could never regenerate them, because regeneration only
happens at an era end that will never come.
THE SELF-CONTRADICTION. Member rows read "Live - learning from new bars"
(from m_trainingComplete) while the line under them read "training, not
tradable yet" (from `prospective`, which means "this number came from
ProspectiveVote() rather than a real Direction() call" - what happens on
any bar where every member abstains, and which says nothing whatever
about training state). Both now resolve through one predicate:
WarriorChartModelsDeployed(), fed by members publishing their own state
on the same slot and cadence as their vote. Adds a third verdict word,
"armed (bar still open)", for a deployed model on a prospective
recompute - the case that used to claim it was training.
DEPLOYED PANEL. Once every published model is converged the per-member
rows are dropped: what ships is the aggregate vote win rate, the live
vote, and the verdict. While training the rows stay - they are the only
way a collapsed or lagging member is visible, since a collapsed member
abstains and so is invisible in the aggregate by construction.
ACCURACY NOW RESPECTS THE ENTRY THRESHOLD. The panel's "precision 65%"
came from m_cumOosCorrect/m_cumOosTotal, which counts every bar a model
called Buy or Sell - threshold-blind, and per-model rather than
per-vote. The correct number already existed (votePrecPct: bars where
|vote| >= threshold and the direction policy allows) and is now what the
panel shows, with the threshold named in the text because the number is
meaningless without it.
VOTE ARROWS PERSIST. With DrawUnfilteredSignals off - the default - the
chart shows SIG_VOTE_PREFIX arrows, and nothing saved them:
CChartUI's .arrows sidecar is member-scoped and never saw that layer.
New CVoteArrowStore mirrors them to a chart-keyed sidecar and restores
them progressively at init, on the same budgeted non-blocking path.
The header stores the open/close thresholds; a mismatch on load DISCARDS
the arrows rather than redrawing a picture of a strategy no longer
configured - stale arrows are worse than none, because none is visibly
empty and stale is confidently wrong.
Also: .stats bumped to WST7 carrying the ensemble record (guarded on
threshold match, most-complete-copy-wins), and the loader's version
tests collapsed from an or-chain to ">=" - the magics are ASCII 'WST1'..
'WST7' so they are already ordered, and a missed arm in that chain reads
the NEXT field's bytes into this one, which fails as plausible numbers
rather than as an error.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 13:00:15 -04:00
//+------------------------------------------------------------------+
//| THE ENSEMBLE'S HEADLINE NUMBER, in one place. |
//| |
//| It used to be built inline at pass-3 completion and nowhere else, |
//| which is why a restarted deployed chart had no aggregate line at |
//| all: that code runs once per era, and a deployed ensemble runs no |
//| further eras. Now it is a function, called from two places - the |
//| era end (with this era's figure) and init (without one, from the |
//| record restored out of .stats). |
//| |
//| WHAT THE NUMBER IS, and why it is the only one that belongs on a |
//| deployed panel: the win rate of the COMBINED VOTE over the bars |
//| the vote actually fired on - i.e. bars whose |vote| cleared |
//| Signal_ThresholdOpen and whose side the direction policy allows. |
//| A member's own precision counts every bar that member called Buy |
//| or Sell, threshold or no threshold, which is not a quantity |
//| anyone can trade. The threshold is NAMED in the text for the same |
//| reason it is stored in the file: the number is meaningless |
//| without it. |
//+------------------------------------------------------------------+
void PublishEnsembleAccuracyLine ( const double thisEraPrecPct , const int thisEraFired )
{
if ( g_ensCumOosTotal < = 0 )
{
fix(persist): adopt the pinned threshold on load; trim the accuracy label
THE REGRESSION, mine, from c6eb908. LoadModelStats() dropped the whole ensemble
record unless the stored threshold EQUALLED the live one. That was right while
the threshold was an operator input - a record built at 25% says nothing about a
chart now running 15%. Once the threshold became derived and pinned the
comparison inverted its own meaning: at load time g_ensembleVoteThreshold is
still the Signal_ThresholdOpen SEED, so the stored derived value never matches
and the record is ALWAYS dropped. Two things died with it, silently:
* g_ensDeployApproved - a DEPLOYED ensemble came back as a training one on
every restart, discarding the family-wise deploy it had earned.
* the pinned threshold itself - PublishVoteThreshold() only fires on a positive
g_ensDerivedThreshold, so a deployed chart would have traded the .chr seed
instead of the rung its deploy was certified at. certified != traded, the
defect 2c443ba fixed, reintroduced three commits later.
Not yet observed live only because SP500 deployed at 10:20, after the last
restart at 09:54, so no restart has crossed a deployed state.
Now ADOPTED, not compared: threshold, counts and deploy flag restore together,
the only coherent state - the counts were conditional on that threshold, which is
why it is stored beside them. Same doctrine as the .cfg topology: adopt what the
model was certified with, never re-derive it underneath a checkpoint. The
most-complete-copy guard is unchanged. It now logs what it restored.
THE PANEL LABEL. "Vote win rate: 34% (338 calls at or above the 15% threshold,
this era 31%)" -> "Accuracy: 34%". The call count, threshold and this-era figure
are diagnostics, all present in the era log line, and on a panel they buried the
one number anyone reads. The threshold no longer needs naming either: it is
derived and pinned rather than an operator's choice, so it is not a caveat on the
percentage. The era/models/deployable suffix appended at era end goes with them.
Compiled clean; NOT yet run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 11:59:25 -04:00
g_ensembleVoteLine = ( g_ensCandidateEras > 0 ) ? " Accuracy: no calls yet "
: " Accuracy: measuring... " ;
fix(panel,arrows): one deploy predicate, a deployed-only readout, and persist the vote arrows
Four reported symptoms, three of them one root cause: the ensemble's
certified record was session-scoped and written ONLY at pass-3
completion. A deployed ensemble runs no further eras, so every restart
lost the aggregate win rate, the aggregate panel line and the overlay
snapshots - and could never regenerate them, because regeneration only
happens at an era end that will never come.
THE SELF-CONTRADICTION. Member rows read "Live - learning from new bars"
(from m_trainingComplete) while the line under them read "training, not
tradable yet" (from `prospective`, which means "this number came from
ProspectiveVote() rather than a real Direction() call" - what happens on
any bar where every member abstains, and which says nothing whatever
about training state). Both now resolve through one predicate:
WarriorChartModelsDeployed(), fed by members publishing their own state
on the same slot and cadence as their vote. Adds a third verdict word,
"armed (bar still open)", for a deployed model on a prospective
recompute - the case that used to claim it was training.
DEPLOYED PANEL. Once every published model is converged the per-member
rows are dropped: what ships is the aggregate vote win rate, the live
vote, and the verdict. While training the rows stay - they are the only
way a collapsed or lagging member is visible, since a collapsed member
abstains and so is invisible in the aggregate by construction.
ACCURACY NOW RESPECTS THE ENTRY THRESHOLD. The panel's "precision 65%"
came from m_cumOosCorrect/m_cumOosTotal, which counts every bar a model
called Buy or Sell - threshold-blind, and per-model rather than
per-vote. The correct number already existed (votePrecPct: bars where
|vote| >= threshold and the direction policy allows) and is now what the
panel shows, with the threshold named in the text because the number is
meaningless without it.
VOTE ARROWS PERSIST. With DrawUnfilteredSignals off - the default - the
chart shows SIG_VOTE_PREFIX arrows, and nothing saved them:
CChartUI's .arrows sidecar is member-scoped and never saw that layer.
New CVoteArrowStore mirrors them to a chart-keyed sidecar and restores
them progressively at init, on the same budgeted non-blocking path.
The header stores the open/close thresholds; a mismatch on load DISCARDS
the arrows rather than redrawing a picture of a strategy no longer
configured - stale arrows are worse than none, because none is visibly
empty and stale is confidently wrong.
Also: .stats bumped to WST7 carrying the ensemble record (guarded on
threshold match, most-complete-copy-wins), and the loader's version
tests collapsed from an or-chain to ">=" - the magics are ASCII 'WST1'..
'WST7' so they are already ordered, and a missed arm in that chain reads
the NEXT field's bytes into this one, which fails as plausible numbers
rather than as an error.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 13:00:15 -04:00
return ;
}
int winPct = ( int ) MathRound ( g_ensCumOosCorrect * 100.0 / g_ensCumOosTotal ) ;
//--- THIS ERA alongside the lifetime figure - same reason as the solo panel's
//--- ComputeCompoundedAccuracyLine: the lifetime average is diluted by every fired bar from every
//--- prior era, so a real swing this era barely moves it. Omitted entirely at init, where there is
//--- no "this era" and a stale one would read as live.
fix(persist): adopt the pinned threshold on load; trim the accuracy label
THE REGRESSION, mine, from c6eb908. LoadModelStats() dropped the whole ensemble
record unless the stored threshold EQUALLED the live one. That was right while
the threshold was an operator input - a record built at 25% says nothing about a
chart now running 15%. Once the threshold became derived and pinned the
comparison inverted its own meaning: at load time g_ensembleVoteThreshold is
still the Signal_ThresholdOpen SEED, so the stored derived value never matches
and the record is ALWAYS dropped. Two things died with it, silently:
* g_ensDeployApproved - a DEPLOYED ensemble came back as a training one on
every restart, discarding the family-wise deploy it had earned.
* the pinned threshold itself - PublishVoteThreshold() only fires on a positive
g_ensDerivedThreshold, so a deployed chart would have traded the .chr seed
instead of the rung its deploy was certified at. certified != traded, the
defect 2c443ba fixed, reintroduced three commits later.
Not yet observed live only because SP500 deployed at 10:20, after the last
restart at 09:54, so no restart has crossed a deployed state.
Now ADOPTED, not compared: threshold, counts and deploy flag restore together,
the only coherent state - the counts were conditional on that threshold, which is
why it is stored beside them. Same doctrine as the .cfg topology: adopt what the
model was certified with, never re-derive it underneath a checkpoint. The
most-complete-copy guard is unchanged. It now logs what it restored.
THE PANEL LABEL. "Vote win rate: 34% (338 calls at or above the 15% threshold,
this era 31%)" -> "Accuracy: 34%". The call count, threshold and this-era figure
are diagnostics, all present in the era log line, and on a panel they buried the
one number anyone reads. The threshold no longer needs naming either: it is
derived and pinned rather than an operator's choice, so it is not a caveat on the
percentage. The era/models/deployable suffix appended at era end goes with them.
Compiled clean; NOT yet run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 11:59:25 -04:00
//--- JUST THE NUMBER (2026-08-26). The call count, the threshold and the this-era figure all rode
//--- along here; they are diagnostics, every one of them is in the era log line, and on a panel
//--- they buried the one number anyone actually reads. The threshold in particular no longer needs
//--- naming: it is derived and pinned rather than an operator's choice, so it is not a caveat on
//--- the percentage any more.
g_ensembleVoteLine = StringFormat ( " Accuracy: %d%% " , winPct ) ;
fix(panel,arrows): one deploy predicate, a deployed-only readout, and persist the vote arrows
Four reported symptoms, three of them one root cause: the ensemble's
certified record was session-scoped and written ONLY at pass-3
completion. A deployed ensemble runs no further eras, so every restart
lost the aggregate win rate, the aggregate panel line and the overlay
snapshots - and could never regenerate them, because regeneration only
happens at an era end that will never come.
THE SELF-CONTRADICTION. Member rows read "Live - learning from new bars"
(from m_trainingComplete) while the line under them read "training, not
tradable yet" (from `prospective`, which means "this number came from
ProspectiveVote() rather than a real Direction() call" - what happens on
any bar where every member abstains, and which says nothing whatever
about training state). Both now resolve through one predicate:
WarriorChartModelsDeployed(), fed by members publishing their own state
on the same slot and cadence as their vote. Adds a third verdict word,
"armed (bar still open)", for a deployed model on a prospective
recompute - the case that used to claim it was training.
DEPLOYED PANEL. Once every published model is converged the per-member
rows are dropped: what ships is the aggregate vote win rate, the live
vote, and the verdict. While training the rows stay - they are the only
way a collapsed or lagging member is visible, since a collapsed member
abstains and so is invisible in the aggregate by construction.
ACCURACY NOW RESPECTS THE ENTRY THRESHOLD. The panel's "precision 65%"
came from m_cumOosCorrect/m_cumOosTotal, which counts every bar a model
called Buy or Sell - threshold-blind, and per-model rather than
per-vote. The correct number already existed (votePrecPct: bars where
|vote| >= threshold and the direction policy allows) and is now what the
panel shows, with the threshold named in the text because the number is
meaningless without it.
VOTE ARROWS PERSIST. With DrawUnfilteredSignals off - the default - the
chart shows SIG_VOTE_PREFIX arrows, and nothing saved them:
CChartUI's .arrows sidecar is member-scoped and never saw that layer.
New CVoteArrowStore mirrors them to a chart-keyed sidecar and restores
them progressively at init, on the same budgeted non-blocking path.
The header stores the open/close thresholds; a mismatch on load DISCARDS
the arrows rather than redrawing a picture of a strategy no longer
configured - stale arrows are worse than none, because none is visibly
empty and stale is confidently wrong.
Also: .stats bumped to WST7 carrying the ensemble record (guarded on
threshold match, most-complete-copy-wins), and the loader's version
tests collapsed from an or-chain to ">=" - the magics are ASCII 'WST1'..
'WST7' so they are already ordered, and a missed arm in that chain reads
the NEXT field's bytes into this one, which fails as plausible numbers
rather than as an error.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 13:00:15 -04:00
}
2026-08-20 09:49:33 -04:00
//--- Set by AdvancePatternDatabaseBackfill() on completion; lets OnTimer bypass the hourly DB-ranking
//--- throttle ONCE so weights refresh from the fresh rows immediately.
2026-08-16 21:08:41 -04:00
bool g_forcePatternWeightsRefresh = false ;
2026-08-20 09:49:33 -04:00
//--- Set by RankTiersFromOos() at every pass-3 completion; (re)arms the filtered-view overlay sweep.
//--- A flag rather than an era comparison, because what the sweep needs is "a snapshot just got
//--- fresher" - approximating that with era counters is how it used to re-arm against half-built caches.
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era
Full-pipeline analysis after "threshold 30, attained often, nothing drawn,
still glued to buy". The log falsified the premise before any code did:
21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0%
21:42:07 swept 4999, 0 voters, drew 0
21:51:30 swept 4999, 0 voters, drew 0
21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0%
The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root
cause, three symptoms: every display path read m_arrowSignalCache, which is
wiped to sentinel at each era start and only complete again when pass 3
finishes. With eras at ~30s and a sweep at ~17s:
* ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its
else-branch deleted the arrow on every voteless bar - erasing the previous
sweep's entire output. The chart cycled populated -> blank -> populated;
the user kept catching the blank phase.
* READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every
era and fell through to dPrevSignal - the frozen purge-band edge bar that
reads Buy. 659638e fixed which bar was frozen, not the freezing.
* VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a
different fraction of half-rebuilt caches.
THE FIX, structural rather than another patch:
1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one
moment the cache is complete - and now copies it (raw signals, newest
LOOKBACK+16 bars) into member-owned snapshot state, unconditionally,
BEFORE its early return: an all-Neutral era is a snapshot worth showing,
not an absence of one. Raw signals rather than votes, so a tier re-rank
between eras reprices them at read time via LiveVoteContribution for free.
2. The sweep (SnapshotVoteAt) and the prospective readout both read
snapshots; the readout's fallback chain is live-cache -> snapshot ->
dPrevSignal, and the snapshot leg is the one that fires most of the time.
3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual
sub-threshold vote takes an arrow down. This alone ends the wipe half of
the flicker even where snapshots are missing (before the first era).
4. Arming moved from an era-counter diff (which fires at era BOUNDARIES,
i.e. precisely when caches are about to be wiped) to
g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's
snapshot just got fresher", the only event a redraw can act on. 60s rate
limit collapses the four members' burst into one sweep. Classic-only
charts arm once at start.
5. Census now reports the direction split - "922 had a voter (610 buy / 312
sell)" - so "the vote leans buy" is checkable from the log instead of
inferred from arrow colours.
Also visible in the log and worth knowing: the threshold flip-flopped
30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51
ran at 40), so part of the observed blankness was configuration, not code.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 22:03:20 -04:00
bool g_warriorOverlayArmRequest = false ;
fix(vote): unranked members voted with the stock 25/50/75/100 ladder
Two defects behind "arrows drawn while members are still mid-era".
1. THE DRAW. The filtered overlay armed on the FIRST member to finish
pass 3 and leaned on a 60 s rate limit to "collapse the burst",
assuming members finish seconds apart. They do not - on USDJPY one
member was at sample 10496 of pass 2 while another was at 2304,
minutes apart. A member with no era-end snapshot returns false from
SnapshotVoteAt, and the sweep's `if(!hasData) continue;` skips it
BEFORE `den += ModuleWeight()`, so the one finished model's tier
weight became the entire vote and was drawn as a consensus arrow.
An abstention is a member that looked at the bar and said nothing; a
missing snapshot is a member that has not looked. The first must
dilute the vote, the second must suppress the draw. The arm is now a
readiness MASK - one bit per m_ensembleIndex, set at that member's
pass-3 completion, cleared when a sweep arms - and a sweep waits for
every enrolled member. Bounded at 10 minutes so a member that stops
cannot freeze the chart, and the partial draw PRINTS which members
were missing: the be39674 lesson is that a hold must never silence
the thing that reports it.
2. THE VOTE ITSELF, which is the worse half and is not display-only.
Tier weights are not persisted in the .nnw - they exist only as the
output of a completed pass 3 - so before a member's first
RankTiersFromOos() it holds the constructor's stock 25/50/75/100.
Since 4858507 the vote currency is a WIN RATE, so an unranked tier-3
call enters the capability-weighted mean claiming a 100% win rate
beside ranked members contributing ~25. Not a strong opinion: the
wrong unit. One unranked member drags the ensemble over any
threshold, on every fresh deploy and every resume. USDJPY has a
measured ceiling of ~19 and was firing anyway.
LiveVoteContribution() now abstains until self-ranked, which drops
the member from the sum AND the divisor. One function, so live and
the gate move together (2c443ba).
Era 0 will therefore report 0 coverage until each member completes one
era. The ensemble line says so explicitly rather than leaving it to look
like the USDJPY unreachable-threshold case - the two are identical in
the coverage number and completely different problems.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 08:55:04 -04:00
//--- OVERLAY READINESS, one bit per ensemble member (the bit index IS m_ensembleIndex, which is that
//--- member's slot in g_warriorEnsemble). Set at that member's pass-3 completion, cleared when a
//--- sweep arms.
//---
//--- WHY A MASK AND NOT A RATE LIMIT (2026-08-23): a member with no era-end snapshot returns false
//--- from SnapshotVoteAt, and the sweep's `if(!hasData) continue;` skips it BEFORE the divisor - so
//--- one finished model's tier weight becomes the WHOLE vote and gets drawn as a consensus arrow.
//--- The old code armed on the first member to finish and leaned on a 60 s limit to "collapse the
//--- burst", on the assumption that members finish seconds apart. They do not: on USDJPY one member
//--- was at sample 10496 while another was at 2304 of the same pass, minutes apart, so the sweep ran
//--- with one voter and three silent members and put arrows on the chart for a consensus that did
//--- not exist. AN ABSTENTION IS A MEMBER THAT LOOKED AND SAID NOTHING; A MISSING SNAPSHOT IS A
//--- MEMBER THAT HAS NOT LOOKED. The first must dilute the vote, the second must suppress the draw.
uint g_warriorOverlayReadyMask = 0 ;
//--- First tick a sweep was wanted but held for a missing member; 0 = not waiting. Bounds the hold,
//--- because a member that stops (converged, stopped, error) would otherwise freeze the chart
//--- forever - the be39674 lesson: a barrier must never silence the thing that reports it.
uint g_warriorOverlayArmSince = 0 ;
fix: purge every EA object namespace on init and after deinit teardown
Leftover objects survived deinit because the cleanup list had drifted.
PurgeChart()'s own comment said it removed "our namespaced signal arrows
plus the status-label objects" while the code removed arrows ONLY, and
the panel prefix was swept at OnInit and nowhere else - so an ordinary
deinit left the status line, and any panel straggler, on the chart.
Three scattered call sites and a comment cannot be kept in step. There is
now ONE list - WarriorChartPrefixes() - covering arrows, status label and
panel, and one sweep, WarriorPurgeChartObjects(), used by every path.
Add a prefix there when a new object family appears and every cleanup
picks it up.
Two call sites added:
OnInit, before ANYTHING is drawn (including the status label it would
otherwise delete). Chart objects live in the chart PROFILE, not in the
EA, so they outlive the process: a deinit force-terminated at
MetaTrader's ~4,500 ms budget, a crash, a terminal kill, or an .ex5
replaced while attached all strand objects no later deinit will ever
own - and deleting the EA's files does not remove them, which is why
they read as corruption. Arrows are included: LoadChartSignals restores
them from their sidecar moments later and already opens with its own
arrow sweep, so this only removes orphans the sidecar does not account
for - the ones SaveChartSignals would otherwise ADOPT, since it rebuilds
that sidecar by scanning the chart.
OnDeinit, after ExtPanel.Destroy. Destroy walks an unbounded control
tree and ClearStatusLabel clears text rather than guaranteeing object
removal; either can leave a straggler and nothing looked afterwards.
Bounded work - three prefix deletes and one object-list scan - so it
respects the ordering rule that keeps the cheap visible cleanup ahead
of the heavy save. Arrows excluded: ShutdownChartCleanup already
persisted and removed them and re-deleting would race that write.
The two are complementary: the deinit sweep closes the ordinary case, the
OnInit purge closes the case where MetaTrader never let us finish. Only
the second can help after a starved shutdown.
Both sweeps rescan by name across EVERY object type and delete what the
bulk call missed. ObjectsDeleteAll's return has already been observed
disagreeing with a by-name scan of the same chart microseconds apart, and
object commands are queued on the chart rather than applied inline, so a
returned count is not evidence the objects are gone.
Panel create site now uses WARRIOR_PANEL_PREFIX instead of a literal, so
the name cannot drift away from the list that cleans it up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 16:19:26 -04:00
//+------------------------------------------------------------------+
2026-08-22 00:24:45 -04:00
//| EVERY chart-object namespace this EA creates, in ONE list - the |
//| scattered call sites drifted and left stragglers behind. Add a |
//| prefix here the moment a new object family appears. |
//| Delete BY PREFIX, never ObjectsDeleteAll: a blanket wipe also |
//| removes the user's own drawings. |
fix: purge every EA object namespace on init and after deinit teardown
Leftover objects survived deinit because the cleanup list had drifted.
PurgeChart()'s own comment said it removed "our namespaced signal arrows
plus the status-label objects" while the code removed arrows ONLY, and
the panel prefix was swept at OnInit and nowhere else - so an ordinary
deinit left the status line, and any panel straggler, on the chart.
Three scattered call sites and a comment cannot be kept in step. There is
now ONE list - WarriorChartPrefixes() - covering arrows, status label and
panel, and one sweep, WarriorPurgeChartObjects(), used by every path.
Add a prefix there when a new object family appears and every cleanup
picks it up.
Two call sites added:
OnInit, before ANYTHING is drawn (including the status label it would
otherwise delete). Chart objects live in the chart PROFILE, not in the
EA, so they outlive the process: a deinit force-terminated at
MetaTrader's ~4,500 ms budget, a crash, a terminal kill, or an .ex5
replaced while attached all strand objects no later deinit will ever
own - and deleting the EA's files does not remove them, which is why
they read as corruption. Arrows are included: LoadChartSignals restores
them from their sidecar moments later and already opens with its own
arrow sweep, so this only removes orphans the sidecar does not account
for - the ones SaveChartSignals would otherwise ADOPT, since it rebuilds
that sidecar by scanning the chart.
OnDeinit, after ExtPanel.Destroy. Destroy walks an unbounded control
tree and ClearStatusLabel clears text rather than guaranteeing object
removal; either can leave a straggler and nothing looked afterwards.
Bounded work - three prefix deletes and one object-list scan - so it
respects the ordering rule that keeps the cheap visible cleanup ahead
of the heavy save. Arrows excluded: ShutdownChartCleanup already
persisted and removed them and re-deleting would race that write.
The two are complementary: the deinit sweep closes the ordinary case, the
OnInit purge closes the case where MetaTrader never let us finish. Only
the second can help after a starved shutdown.
Both sweeps rescan by name across EVERY object type and delete what the
bulk call missed. ObjectsDeleteAll's return has already been observed
disagreeing with a by-name scan of the same chart microseconds apart, and
object commands are queued on the chart rather than applied inline, so a
returned count is not evidence the objects are gone.
Panel create site now uses WARRIOR_PANEL_PREFIX instead of a literal, so
the name cannot drift away from the list that cleans it up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 16:19:26 -04:00
//+------------------------------------------------------------------+
int WarriorChartPrefixes ( string & out [ ] )
{
feat(chart): on-chart vote readout, and Min_Vote_Open 50 -> 40
THRESHOLD. 40 is a measured correction, not a preference. Once
RankTiersFromOos() replaced the designed tier priors with each model's real
held-out win rate, the vote converges on that win rate - logged 2026-08-18 as
pooled 23-36% across four members on three symbols - so a 50% bar could not be
reached by anything on offer and the ensemble gate fired on 0 of 4,865 OOS
bars. 40 clears the ~34% break-even those same lines report without being
unreachable. The comment says plainly not to copy the number: break-even is a
function of the barrier geometry, so read the gate's own "needs >N%" for the
config in front of you.
READOUT. One line, top-right:
VOTE SELL 37.2% peak 44.1% need 40% 3 voter(s) -> no trade
Every other number on the chart is downstream of the weighted mean the open
threshold is compared against, and that was the one quantity never displayed.
A chart with no arrows could mean the models abstained, the vote was diluted,
or the threshold is unreachable - and telling those apart meant waiting for an
era to end and reading the gate line, which is how the last two sessions went.
PEAK is the part that earns its space. A threshold above what the vote ever
attains can never fire, and that is not knowable from a single bar - it is
precisely the "unreachable gate vs merely unmet gate" confusion this project
has paid for twice. Colour carries the verdict rather than the direction:
green/red ONLY when the vote would actually place an order, grey otherwise.
Green-for-buy would make a below-threshold buy look like a trade, which is the
specific misreading the display exists to prevent.
Guarded on `total > 0` for the same reason the normalization is: Direction()
is inherited as-is by every leaf filter, so without it each filter would write
its own opinion into the one shared label and the last to run would win - the
reader would be looking at an arbitrary member's number believing it was the
vote. Drawn after the +-100 range check, so it shows what the threshold is
actually tested against.
CORNER_RIGHT_UPPER: the status lines, control panel and ensemble panel all
live on the left. Registered in WarriorChartPrefixes() explicitly even though
the "Warrior" catch-all already reaches it - that catch-all exists because the
list has drifted twice, not to make entries optional.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 18:22:45 -04:00
ArrayResize ( out , 7 ) ;
fix(ensemble): per-member arrow namespaces; ConvLSTM rename; dialog in purge list
The ensemble chart UI had a shared-namespace defect that answered the user
question "what do the arrows represent?" with "a bug": all four members drew
arrows under the same WarSig_<bartime> object names, so the chart showed
whichever member rendered LAST, one member Neutral deleted another member Buy
at the same bar, each member init sweep wiped the arrows the previous member
had just restored, and SaveChartSignals - which rebuilds the sidecar by
SCANNING the chart - persisted every other member arrows into its own history
(the exact cross-model laundering its own header warns about, now happening
BETWEEN ensemble members).
Arrows are now namespaced per member (WarSig_PAI_, WarSig_CONV_, WarSig_LSTM_,
WarSig_HYB_): draw, delete, restore, prune, member init sweep, destructor
purge and the sidecar scan are all member-scoped, and the tooltip names the
model. Global purges keep matching the bare WarSig_ prefix, which covers all
member namespaces plus old-format leftovers from earlier builds.
Labels: the ensemble panel header no longer says "HYBRID ensemble" (HYBRID is
one member; the header is the ensemble) and the CONVLSTM member displays as
ConvLSTM instead of Hybrid. Its SHORT id stays HYB deliberately - it names the
model folder and changing it would orphan every model trained under that path.
Deinit: the alt-data mapping dialog namespace (WarriorAltMap_) joins
WarriorChartPrefixes, so both the OnInit purge and the deinit final sweep now
cover it - it was in neither list, so a dialog starved of its own Destroy()
left its controls on the chart permanently.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 18:26:55 -04:00
out [ 0 ] = SIG_ARROW_PREFIX ; // directional signal arrows - bare prefix, so it also
// matches every per-member namespace (WarSig_PAI_ ...)
fix: purge every EA object namespace on init and after deinit teardown
Leftover objects survived deinit because the cleanup list had drifted.
PurgeChart()'s own comment said it removed "our namespaced signal arrows
plus the status-label objects" while the code removed arrows ONLY, and
the panel prefix was swept at OnInit and nowhere else - so an ordinary
deinit left the status line, and any panel straggler, on the chart.
Three scattered call sites and a comment cannot be kept in step. There is
now ONE list - WarriorChartPrefixes() - covering arrows, status label and
panel, and one sweep, WarriorPurgeChartObjects(), used by every path.
Add a prefix there when a new object family appears and every cleanup
picks it up.
Two call sites added:
OnInit, before ANYTHING is drawn (including the status label it would
otherwise delete). Chart objects live in the chart PROFILE, not in the
EA, so they outlive the process: a deinit force-terminated at
MetaTrader's ~4,500 ms budget, a crash, a terminal kill, or an .ex5
replaced while attached all strand objects no later deinit will ever
own - and deleting the EA's files does not remove them, which is why
they read as corruption. Arrows are included: LoadChartSignals restores
them from their sidecar moments later and already opens with its own
arrow sweep, so this only removes orphans the sidecar does not account
for - the ones SaveChartSignals would otherwise ADOPT, since it rebuilds
that sidecar by scanning the chart.
OnDeinit, after ExtPanel.Destroy. Destroy walks an unbounded control
tree and ClearStatusLabel clears text rather than guaranteeing object
removal; either can leave a straggler and nothing looked afterwards.
Bounded work - three prefix deletes and one object-list scan - so it
respects the ordering rule that keeps the cheap visible cleanup ahead
of the heavy save. Arrows excluded: ShutdownChartCleanup already
persisted and removed them and re-deleting would race that write.
The two are complementary: the deinit sweep closes the ordinary case, the
OnInit purge closes the case where MetaTrader never let us finish. Only
the second can help after a starved shutdown.
Both sweeps rescan by name across EVERY object type and delete what the
bulk call missed. ObjectsDeleteAll's return has already been observed
disagreeing with a by-name scan of the same chart microseconds apart, and
object commands are queued on the chart rather than applied inline, so a
returned count is not evidence the objects are gone.
Panel create site now uses WARRIOR_PANEL_PREFIX instead of a literal, so
the name cannot drift away from the list that cleans it up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 16:19:26 -04:00
out [ 1 ] = STATUS_LABEL_PREFIX ; // status line background + text (System\StatusLabel.mqh)
out [ 2 ] = WARRIOR_PANEL_PREFIX ; // control panel and its whole control tree
fix(ensemble): per-member arrow namespaces; ConvLSTM rename; dialog in purge list
The ensemble chart UI had a shared-namespace defect that answered the user
question "what do the arrows represent?" with "a bug": all four members drew
arrows under the same WarSig_<bartime> object names, so the chart showed
whichever member rendered LAST, one member Neutral deleted another member Buy
at the same bar, each member init sweep wiped the arrows the previous member
had just restored, and SaveChartSignals - which rebuilds the sidecar by
SCANNING the chart - persisted every other member arrows into its own history
(the exact cross-model laundering its own header warns about, now happening
BETWEEN ensemble members).
Arrows are now namespaced per member (WarSig_PAI_, WarSig_CONV_, WarSig_LSTM_,
WarSig_HYB_): draw, delete, restore, prune, member init sweep, destructor
purge and the sidecar scan are all member-scoped, and the tooltip names the
model. Global purges keep matching the bare WarSig_ prefix, which covers all
member namespaces plus old-format leftovers from earlier builds.
Labels: the ensemble panel header no longer says "HYBRID ensemble" (HYBRID is
one member; the header is the ensemble) and the CONVLSTM member displays as
ConvLSTM instead of Hybrid. Its SHORT id stays HYB deliberately - it names the
model folder and changing it would orphan every model trained under that path.
Deinit: the alt-data mapping dialog namespace (WarriorAltMap_) joins
WarriorChartPrefixes, so both the OnInit purge and the deinit final sweep now
cover it - it was in neither list, so a dialog starved of its own Destroy()
left its controls on the chart permanently.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 18:26:55 -04:00
out [ 3 ] = " WarriorAltMap_ " ; // alt-data symbol-mapping dialog (ADM_PREFIX in
// Panel\AltDataMapDialog.mqh - literal here because that
// header is included later in the build order)
2026-08-22 00:24:45 -04:00
//--- CATCH-ALL. "Nothing matching our prefixes" and "the chart is clean" are different statements,
//--- and only the first was checked - charts came up with duplicated panels after a purge reported
//--- zero leftovers. Does NOT defeat skipArrows: "WarSig_" does not start with "Warrior".
fix(chart): a purge that reports "zero leftovers" was only ever checking its own list
2026-08-17 21:58: all three charts hit "Abnormal termination" ~5.3 s into
OnDeinit with NO cleanup-timings line - the teardown was starved again. The
22:00 init purge then removed 993 / 1373 / 1557 stranded objects and reported
ZERO by-name leftovers on every chart, and the charts still came up with
duplicated panels. "Nothing matching our prefixes remains" and "the chart is
clean" are different statements and only the first was being made.
Three changes, in the order they matter:
1. WHY the teardown starved, and it is a gap in ad80e0b. StartLabelCachePrebuild
runs ResizeBuffers + RefreshData over the FULL study window (33,984 bars on
XAUUSD), unchunked, and OnDeinit cannot begin until it returns. Normally a
once-per-run cost. That night SP500 and XAUUSD LSTM were wedged in the "cache
invalidated at era start" loop, which calls it on EVERY Train() call - two
members re-preparing tens of thousands of bars indefinitely. The terminal
closed into that. Guarded now, plus a resumable guard in the prebuild chunk
loop (the tally pass after it is not chunked).
2. Catch-all "Warrior" prefix in WarriorChartPrefixes. Every family this EA
creates is named Warrior* except the arrows (WarSig_), so one bare prefix
covers the three named entries AND anything a rename or a stale .ex5 left
under a name nobody remembers. Still a prefix delete, never
ObjectsDeleteAll(chart) - the user's own drawings are not ours to remove. Does
not defeat skipArrows: "WarSig_" does not start with "Warrior".
3. The init purge now REPORTS the residue it did not claim, by name (up to 12).
Not deleted - an unmatched object may belong to the user or another indicator.
If a Warrior panel is visible and appears in neither the removed count nor
this list, the prefix list has drifted a third time and the name is in the
journal instead of being inferred from a screenshot.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:07:09 -04:00
out [ 4 ] = " Warrior " ;
2026-08-22 00:24:45 -04:00
//--- Vote arrows, listed SEPARATELY from SIG_ARROW_PREFIX even though the name matches it:
//--- skipArrows protects the per-model arrows because their sidecar is rebuilt by SCANNING them
//--- off the chart.
fix(deinit): vote arrows survived the cheap sweep, and 5 long loops ignored the stop
Leftover chart objects on long-history charts. Two causes, one of them
introduced by 07aa017.
THE ONE I ADDED. The filtered view's overlay draws up to
SIGNAL_RESCAN_LOOKBACK_BARS vote arrows. OnDeinit's EARLY VISIBLE-UI SWEEP
runs with skipArrows=true, which skips any prefix equal to SIG_ARROW_PREFIX -
and "WarSig_VOTE_..." starts with "WarSig_", so every one of them was skipped
by the one sweep that is cheap enough to always complete. They then sat in the
object list while the two expensive scans that follow walked it: a per-member
SaveChartSignals O(total) scan, then the by-name rescan. On a chart with years
of history that is thousands of extra objects walked twice, inside a teardown
budget measured from the stop REQUEST rather than from OnDeinit's first line.
skipArrows exists because the per-model arrows' sidecar is rebuilt by SCANNING
them off the chart, so they cannot be deleted before that write. Vote arrows
have no sidecar - they are a reconstruction, rebuilt on the next attach - so
nothing is preserving them and they now get their own prefix slot, deleted by
one native call in the first few milliseconds.
THE FIVE LOOPS. A time budget bounds THROUGHPUT, not latency to an unload, and
OnDeinit cannot begin until whatever is in flight returns. These all scaled
with history and none of them checked:
* Training passes 2, 2.5 and 3 yielded only on TRAIN_TIME_BUDGET_MS. Pass 1
has checked IsStopped() all along; the other three never have, and they
are the ones that grow with the bar count. Free to fix - the resume state
is written either way, so a stopped chunk simply is not re-entered.
* PruneDirectionalClusters: the one UNCHUNKED sweep left, once per era over
every bar, with its own header noting that raising the training budget
cannot help its cost. Now bails outright.
* AdvanceChartSignalRestore / AdvanceChartSignalRescan: chunked, but the
rescan runs a full feedForward per bar over up to 5000 bars and the
restore can hold MAX_RESTORED_ARROWS entries. Checked on the same
64-object stride as the clock read, since the check is not free either.
* AdvanceFilteredOverlay (mine, 07aa017) replays Direction() on every
classic filter per bar and had no check at all. Now per bar.
ChartUI.mqh had ZERO shutdown checks across six loops before this.
Nothing was added to the purge path itself: that is the work that must
complete, and an IsStopped() check inside it would abort unconditionally -
IsStopped() is already true by the time OnDeinit runs.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 17:13:50 -04:00
out [ 5 ] = SIG_VOTE_PREFIX ;
2026-08-20 09:49:33 -04:00
//--- The vote readout. Already covered by the catch-all, and listed anyway: the catch-all exists
//--- because this list has drifted twice, not to make entries optional.
feat(chart): on-chart vote readout, and Min_Vote_Open 50 -> 40
THRESHOLD. 40 is a measured correction, not a preference. Once
RankTiersFromOos() replaced the designed tier priors with each model's real
held-out win rate, the vote converges on that win rate - logged 2026-08-18 as
pooled 23-36% across four members on three symbols - so a 50% bar could not be
reached by anything on offer and the ensemble gate fired on 0 of 4,865 OOS
bars. 40 clears the ~34% break-even those same lines report without being
unreachable. The comment says plainly not to copy the number: break-even is a
function of the barrier geometry, so read the gate's own "needs >N%" for the
config in front of you.
READOUT. One line, top-right:
VOTE SELL 37.2% peak 44.1% need 40% 3 voter(s) -> no trade
Every other number on the chart is downstream of the weighted mean the open
threshold is compared against, and that was the one quantity never displayed.
A chart with no arrows could mean the models abstained, the vote was diluted,
or the threshold is unreachable - and telling those apart meant waiting for an
era to end and reading the gate line, which is how the last two sessions went.
PEAK is the part that earns its space. A threshold above what the vote ever
attains can never fire, and that is not knowable from a single bar - it is
precisely the "unreachable gate vs merely unmet gate" confusion this project
has paid for twice. Colour carries the verdict rather than the direction:
green/red ONLY when the vote would actually place an order, grey otherwise.
Green-for-buy would make a below-threshold buy look like a trade, which is the
specific misreading the display exists to prevent.
Guarded on `total > 0` for the same reason the normalization is: Direction()
is inherited as-is by every leaf filter, so without it each filter would write
its own opinion into the one shared label and the last to run would win - the
reader would be looking at an arbitrary member's number believing it was the
vote. Drawn after the +-100 range check, so it shows what the threshold is
actually tested against.
CORNER_RIGHT_UPPER: the status lines, control panel and ensemble panel all
live on the left. Registered in WarriorChartPrefixes() explicitly even though
the "Warrior" catch-all already reaches it - that catch-all exists because the
list has drifted twice, not to make entries optional.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 18:22:45 -04:00
out [ 6 ] = VOTE_HUD_PREFIX ;
return 7 ;
fix: purge every EA object namespace on init and after deinit teardown
Leftover objects survived deinit because the cleanup list had drifted.
PurgeChart()'s own comment said it removed "our namespaced signal arrows
plus the status-label objects" while the code removed arrows ONLY, and
the panel prefix was swept at OnInit and nowhere else - so an ordinary
deinit left the status line, and any panel straggler, on the chart.
Three scattered call sites and a comment cannot be kept in step. There is
now ONE list - WarriorChartPrefixes() - covering arrows, status label and
panel, and one sweep, WarriorPurgeChartObjects(), used by every path.
Add a prefix there when a new object family appears and every cleanup
picks it up.
Two call sites added:
OnInit, before ANYTHING is drawn (including the status label it would
otherwise delete). Chart objects live in the chart PROFILE, not in the
EA, so they outlive the process: a deinit force-terminated at
MetaTrader's ~4,500 ms budget, a crash, a terminal kill, or an .ex5
replaced while attached all strand objects no later deinit will ever
own - and deleting the EA's files does not remove them, which is why
they read as corruption. Arrows are included: LoadChartSignals restores
them from their sidecar moments later and already opens with its own
arrow sweep, so this only removes orphans the sidecar does not account
for - the ones SaveChartSignals would otherwise ADOPT, since it rebuilds
that sidecar by scanning the chart.
OnDeinit, after ExtPanel.Destroy. Destroy walks an unbounded control
tree and ClearStatusLabel clears text rather than guaranteeing object
removal; either can leave a straggler and nothing looked afterwards.
Bounded work - three prefix deletes and one object-list scan - so it
respects the ordering rule that keeps the cheap visible cleanup ahead
of the heavy save. Arrows excluded: ShutdownChartCleanup already
persisted and removed them and re-deleting would race that write.
The two are complementary: the deinit sweep closes the ordinary case, the
OnInit purge closes the case where MetaTrader never let us finish. Only
the second can help after a starved shutdown.
Both sweeps rescan by name across EVERY object type and delete what the
bulk call missed. ObjectsDeleteAll's return has already been observed
disagreeing with a by-name scan of the same chart microseconds apart, and
object commands are queued on the chart rather than applied inline, so a
returned count is not evidence the objects are gone.
Panel create site now uses WARRIOR_PANEL_PREFIX instead of a literal, so
the name cannot drift away from the list that cleans it up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 16:19:26 -04:00
}
//+------------------------------------------------------------------+
//| Delete every object in those namespaces from a chart, and verify. |
2026-08-22 00:24:45 -04:00
//| skipArrows spares the arrows for the one caller that must: a |
//| re-init restores them from their sidecar, so wiping them flickers.|
//| The rescan is required - object commands are QUEUED, so a bulk |
//| call's return value is not evidence they are gone. Names are |
//| collected before deleting: deleting while enumerating renumbers |
//| the list being walked. |
fix: purge every EA object namespace on init and after deinit teardown
Leftover objects survived deinit because the cleanup list had drifted.
PurgeChart()'s own comment said it removed "our namespaced signal arrows
plus the status-label objects" while the code removed arrows ONLY, and
the panel prefix was swept at OnInit and nowhere else - so an ordinary
deinit left the status line, and any panel straggler, on the chart.
Three scattered call sites and a comment cannot be kept in step. There is
now ONE list - WarriorChartPrefixes() - covering arrows, status label and
panel, and one sweep, WarriorPurgeChartObjects(), used by every path.
Add a prefix there when a new object family appears and every cleanup
picks it up.
Two call sites added:
OnInit, before ANYTHING is drawn (including the status label it would
otherwise delete). Chart objects live in the chart PROFILE, not in the
EA, so they outlive the process: a deinit force-terminated at
MetaTrader's ~4,500 ms budget, a crash, a terminal kill, or an .ex5
replaced while attached all strand objects no later deinit will ever
own - and deleting the EA's files does not remove them, which is why
they read as corruption. Arrows are included: LoadChartSignals restores
them from their sidecar moments later and already opens with its own
arrow sweep, so this only removes orphans the sidecar does not account
for - the ones SaveChartSignals would otherwise ADOPT, since it rebuilds
that sidecar by scanning the chart.
OnDeinit, after ExtPanel.Destroy. Destroy walks an unbounded control
tree and ClearStatusLabel clears text rather than guaranteeing object
removal; either can leave a straggler and nothing looked afterwards.
Bounded work - three prefix deletes and one object-list scan - so it
respects the ordering rule that keeps the cheap visible cleanup ahead
of the heavy save. Arrows excluded: ShutdownChartCleanup already
persisted and removed them and re-deleting would race that write.
The two are complementary: the deinit sweep closes the ordinary case, the
OnInit purge closes the case where MetaTrader never let us finish. Only
the second can help after a starved shutdown.
Both sweeps rescan by name across EVERY object type and delete what the
bulk call missed. ObjectsDeleteAll's return has already been observed
disagreeing with a by-name scan of the same chart microseconds apart, and
object commands are queued on the chart rather than applied inline, so a
returned count is not evidence the objects are gone.
Panel create site now uses WARRIOR_PANEL_PREFIX instead of a literal, so
the name cannot drift away from the list that cleans it up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 16:19:26 -04:00
//+------------------------------------------------------------------+
int WarriorPurgeChartObjects ( long chartID , bool skipArrows , int & leftoverCount )
{
string prefixes [ ] ;
int n = WarriorChartPrefixes ( prefixes ) ;
int removed = 0 ;
leftoverCount = 0 ;
for ( int p = 0 ; p < n ; p + + )
{
if ( skipArrows & & prefixes [ p ] = = SIG_ARROW_PREFIX )
continue ;
int r = ObjectsDeleteAll ( chartID , prefixes [ p ] ) ;
if ( r > 0 )
removed + = r ;
}
2026-08-20 09:49:33 -04:00
//--- Typed-blind rescan across EVERY object type: filtering on OBJ_ARROW made this blind in the same
//--- way the bulk delete was, which is how two scans of one chart disagreed for three sessions.
fix: purge every EA object namespace on init and after deinit teardown
Leftover objects survived deinit because the cleanup list had drifted.
PurgeChart()'s own comment said it removed "our namespaced signal arrows
plus the status-label objects" while the code removed arrows ONLY, and
the panel prefix was swept at OnInit and nowhere else - so an ordinary
deinit left the status line, and any panel straggler, on the chart.
Three scattered call sites and a comment cannot be kept in step. There is
now ONE list - WarriorChartPrefixes() - covering arrows, status label and
panel, and one sweep, WarriorPurgeChartObjects(), used by every path.
Add a prefix there when a new object family appears and every cleanup
picks it up.
Two call sites added:
OnInit, before ANYTHING is drawn (including the status label it would
otherwise delete). Chart objects live in the chart PROFILE, not in the
EA, so they outlive the process: a deinit force-terminated at
MetaTrader's ~4,500 ms budget, a crash, a terminal kill, or an .ex5
replaced while attached all strand objects no later deinit will ever
own - and deleting the EA's files does not remove them, which is why
they read as corruption. Arrows are included: LoadChartSignals restores
them from their sidecar moments later and already opens with its own
arrow sweep, so this only removes orphans the sidecar does not account
for - the ones SaveChartSignals would otherwise ADOPT, since it rebuilds
that sidecar by scanning the chart.
OnDeinit, after ExtPanel.Destroy. Destroy walks an unbounded control
tree and ClearStatusLabel clears text rather than guaranteeing object
removal; either can leave a straggler and nothing looked afterwards.
Bounded work - three prefix deletes and one object-list scan - so it
respects the ordering rule that keeps the cheap visible cleanup ahead
of the heavy save. Arrows excluded: ShutdownChartCleanup already
persisted and removed them and re-deleting would race that write.
The two are complementary: the deinit sweep closes the ordinary case, the
OnInit purge closes the case where MetaTrader never let us finish. Only
the second can help after a starved shutdown.
Both sweeps rescan by name across EVERY object type and delete what the
bulk call missed. ObjectsDeleteAll's return has already been observed
disagreeing with a by-name scan of the same chart microseconds apart, and
object commands are queued on the chart rather than applied inline, so a
returned count is not evidence the objects are gone.
Panel create site now uses WARRIOR_PANEL_PREFIX instead of a literal, so
the name cannot drift away from the list that cleans it up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 16:19:26 -04:00
int total = ObjectsTotal ( chartID , -1 , -1 ) ;
string leftovers [ ] ;
int found = 0 ;
if ( total > 0 )
{
ArrayResize ( leftovers , total ) ;
for ( int i = 0 ; i < total ; i + + )
{
string nm = ObjectName ( chartID , i , -1 , -1 ) ;
for ( int p = 0 ; p < n ; p + + )
{
if ( skipArrows & & prefixes [ p ] = = SIG_ARROW_PREFIX )
continue ;
if ( StringFind ( nm , prefixes [ p ] ) = = 0 )
{
leftovers [ found + + ] = nm ;
break ;
}
}
}
}
for ( int i = 0 ; i < found ; i + + )
ObjectDelete ( chartID , leftovers [ i ] ) ;
leftoverCount = found ;
return removed + found ;
}
2026-08-20 09:49:33 -04:00
//--- Guard against a corrupt .arrows header declaring a garbage count. Restoring is chunked across timer
//--- calls regardless, so a large-but-valid count costs progressive fill-in, never a frozen OnInit.
2026-07-26 11:17:55 -04:00
# define MAX_RESTORED_ARROWS 50000
2026-08-20 09:49:33 -04:00
//--- How many of the MOST RECENT arrows stay on the chart and in the sidecar. Both save and load select
//--- by TIME, not scan order - ObjectsTotal() order is arbitrary, so "the last N scanned" would keep a
2026-07-26 11:17:55 -04:00
//--- random subset rather than the newest.
# define MAX_PERSISTED_ARROWS 1000
2026-08-22 00:24:45 -04:00
//--- Prior strength RankTiersFromOos() shrinks each tier toward the pooled holdout win rate,
//--- counted in EFFECTIVE observations. A tier carries ~8-15 of those per era, so at 10 it sits
//--- about half on its own evidence.
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
# define TIER_PRIOR_EFF_N 10.0
fix(geometry): a free zero made "never resolve" the winning geometry
The CANDIDATE GEOMETRY line shipped in 05f1a53 said per-candidate
geometry beats the global pair on every SP500 member at 2-3 sigma. It
does not. It said so because a bar that reached neither barrier scored
0 R, and the incumbent's mean is NEGATIVE (-0.07 to -0.21 R). Against a
losing baseline a free zero is a win, so the widest candidate always
came out ahead - and the reported gain ordered itself by timeout share,
not by skill:
PAI 95.1% timed out -> +0.189 R (head measured -2.42 sigma, HARMFUL)
HYB 73.8% -> +0.182 R (head at chance, +0.68 sigma)
CONV 61.8% -> +0.163 R (head measured -2.47 sigma, HARMFUL)
LSTM 27.1% -> +0.158 R (head +1.67 sigma)
Monotone in the timeout share and inverted against the sigma gate. The
acceptance test written when this was built - "the sigma gate predicts
LSTM helps and CONV hurts; if the R difference does not reproduce that
ordering, something is wrong" - is what caught it.
A trade that reaches neither barrier is not worth zero. It is closed at
the horizon, which is what the scheduled close-all does live and what
SimulateTradeOutcome's timeout path already charges. So mark it there:
TripleBarrierLabel now publishes the signed close-to-close travel at the
last bar it actually visited (m_termTravelCache, same validity flag as
the excursion and ladder caches), and LadderOutcomeR prices a timeout
off it instead of returning false. A bar that cannot be evaluated under
BOTH pairs is now dropped whole - scoring one leg and defaulting the
other is the same bug in a smaller costume.
Second defect, same function: CandidateGeometryFor applied neither of
the floors the global derivation applies, so on USDJPY it chose stop
2.00 / target 1.00 - a 67% break-even, forbidden by the 1:2 policy
floor. c3daded in miniature: a selector optimising its own criterion
with no reference to the decision criterion. Both floors now apply, and
the ratio is re-checked AFTER the per-leg rung snap, which can lose it.
Also: the module weight was an unshrunk pooled win rate. USDJPY ConvLSTM
fired 19 times (2.0 effective), won 36.8%, and took module weight 0.37 -
41% of the ensemble's capable weight and the loudest voice on the chart,
off two effective observations. It also lifted the computed vote ceiling
to 26.3 against a 25 threshold, which is why THRESHOLD UNREACHABLE never
printed on a chart whose peak vote is 14 and whose practical ceiling
without that member is 18.8. The pooled rate is now shrunk toward the
coin-flip rate on the era's own OOS bars over 30 prior-equivalent calls,
and the tiers shrink toward the shrunk value rather than the raw one. A
member with ~300 effective calls moves by ~0.4pp; the 19-fire member
goes 0.37 -> ~0.15.
MEASUREMENT ONLY still - no order reads any of this.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:30:54 -04:00
//--- Prior strength for the MODULE weight - how loudly this member speaks in the ensemble mean.
//--- Deliberately far heavier than the tier prior because it is shrunk toward CHANCE, not toward
//--- the member's own pooled rate: a member with almost no held-out fires must not be trusted at
//--- whatever those few fires happened to show. Measured 2026-08-22, USDJPY: ConvLSTM fired 19
//--- times (2.0 effective), won 36.8%, and took module weight 0.37 - 41% of the whole ensemble's
//--- capable weight, off two effective observations, and the loudest voice on the chart. At 30 it
//--- pulls that to ~0.15 while leaving a member with 300 effective calls essentially untouched.
# define MODULE_PRIOR_EFF_N 30.0
2026-08-20 09:49:33 -04:00
//--- Wall-clock budget per chunk of the deferred arrow restore. MQL5 gives a chart ONE thread, so "async"
//--- means small time-boxed slices, never one long blocking pass. 50ms sits between the training chunk
//--- (120ms) and the 500ms timer period.
2026-07-26 11:17:55 -04:00
# define ARROW_RESTORE_BUDGET_MS 50
2026-08-20 09:49:33 -04:00
//--- Max |diff| between the compute backend and the pure-MQL5 path for a model to be marked
//--- MQL5-inference-safe. Summation-order noise is ~1e-6; a genuine port bug shows up as >0.01.
2026-07-24 11:52:19 -04:00
# define CPU_INFERENCE_MAX_DIFF 1.0e-3
2026-08-20 09:49:33 -04:00
//--- Equal-frequency bins the feature column is discretised into. MI is biased upward as bins increase,
//--- and 8 against MI_SAMPLE_BARS keeps that bias small and EQUAL across candidates - equal is what
//--- matters, since this score is only ever used to RANK.
feat(features): per-column MI keep-screen (report only)
Step 1 of the prune, stopping deliberately short of pruning - two blockers make
an immediate mask the wrong move, and this is the measurement that decides
whether pruning is worth doing at all.
WHY NOT PRUNE YET:
* the screen runs with cross-asset ABSENT - its own log line says the numbers
"describe a NARROWER vector than training will use". A mask built from it
would have no evidence either way about the cross-asset block.
* a per-chart mask FRAGMENTS THE POOL. The mask must participate in the
fingerprint, and the pool only accepts peers with an identical feature
layout. Pooling is currently the only thing keeping the FX trio off the
capacity floor - the three pool-poor charts (SP500, XAUUSD, XTIUSD) are
exactly the three still floored. Six per-chart masks = six pool groups of
one, and pruning could cost more capacity than it buys.
WHAT THIS ADDS: the per-column MI was always computed inside ScoreMiSample and
thrown away except for the sum and the max. It is retained now, and the same
permutation draws that build the headline null also accumulate a PER-COLUMN null,
which is what a per-column p-value needs - distinct from the null-of-the-max,
which answers the single family-wise question "is the strongest column real".
Selection uses Benjamini-Hochberg at q=0.10, NOT the family-wise bar. FWER
controls the chance of one false positive, which is right for a verdict and far
too conservative for selection - it would discard every genuinely weak-but-useful
feature. BH bounds the expected SHARE of kept columns that are noise, which is
what a feature set cares about.
The report prints the decision in capacity units: columns kept, the resulting
input width, and the first-layer budget before and after against the 16-wide
floor. 3 of 52 is not a feature set; 45 of 52 is not worth a fingerprint re-key.
The cross-asset caveat prints itself when it applies.
Context that makes this worth doing at all: under the pivot-event label the MI
screen now reads "above the noise floor - a real association" - mean 4x the null
(p=0.005), strongest column 7.7x the null-max, excess 0.80% of label entropy,
against 1.3x / 1.15x / ~0.1% under the old label. The noise-floor verdict that
closed several earlier directions was a property of the OLD label.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 15:43:59 -04:00
//--- FALSE DISCOVERY RATE for the per-column keep report. FDR, not the family-wise bar the headline
//--- test uses: FWER asks "is ANY column real" and controls the chance of a single false positive,
//--- which is the right question for a verdict and far too conservative for SELECTION - it would
//--- discard every genuinely weak-but-useful feature to protect against one false one. Benjamini-
//--- Hochberg instead bounds the EXPECTED SHARE of kept columns that are noise, which is what a
//--- feature set actually cares about. 0.10 = at most ~10% of what is kept is expected to be junk.
# define MI_KEEP_FDR_Q 0.10
perf(autotune): replace the genetic search with a filter score - hours to seconds
MEASURED COST OF THE GA, which is what retired it. Per generation:
rung 0: 8 cand x 3 seeds x 3 eras = 72 eras
rung 1: 4 cand x 3 seeds x 8 eras = 96
rung 2: 2 cand x 3 seeds x 20 eras = 120
= 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's
real training began. Against the observed era times on SP500 H1:
PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22)
CONV 41.3 s/era -> 13.2 h
LSTM 150.4 s/era -> 48.1 h
HYBRID 154.6 s/era -> 49.5 h
Two days to tune is not a first-run experience, and it is the phase in
which the panel goes quiet, which is what made it look like a hang.
It also bought nothing. The space is 90 points (10 MA periods x 9 MA
types), so 1152 evaluations revisited each point ~13 times; and rungs of
3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run
proves it: every finalist scored 25.0-25.9% balanced accuracy - below the
33.3% one-class floor, i.e. indistinguishable noise - and the search then
"deployed the winner" of that.
THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full
training run to choose a feature's period is a wrapper method paying
wrapper prices for a decision that does not need one. The reference book
does not do this: ch. 3.3 selects inputs by measuring each candidate
indicator's CORRELATION with the target and dropping the ones with none,
with no network involved.
So: rank candidates by the MUTUAL INFORMATION between the resulting
feature vector and the triple-barrier label. MI rather than correlation
because the label is 3-class categorical and the features are not
monotonically related to it. Equal-FREQUENCY binning (rank-based),
because these features are ATR-normalised and heavy-tailed - fixed-width
bins put nearly everything in one bucket and report ~0 information for a
genuinely useful feature.
Scoring is arithmetic over the feature cache, so it costs seconds and its
cost is independent of topology: LSTM now tunes as fast as the MLP.
Coordinate sweep, not product sweep - cost is the SUM of per-parameter
candidate counts, so enabling every indicator stays affordable - with a
second pass that breaks early once nothing moves.
Sampling is IS-ONLY. Letting the OOS window influence which indicator
settings ship would mean the holdout had been used for selection and had
stopped being a holdout.
HONEST LIMIT, recorded because it is the price: MI is marginal, so a
parameter that only pays off in combination with another can be missed
(Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it
replaces was ranking pure noise at 48 h a run, this is strictly better.
Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/
GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga*
members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget.
AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28
read sites all permanently inert. That is not a tidy-up: the `if
(!m_evalMode)` guard on UpdateClassPriors is exactly what silently
disabled the imbalance correction for entire runs two commits ago. Dead
machinery that still reads like live machinery is this codebase's most
expensive recurring bug, and leaving 28 more instances of it would have
been indefensible.
The panel's tuning-progress state goes too - tuning no longer takes long
enough to need one.
Both builds compile 0 errors / 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
# define MI_BINS 8
2026-08-20 09:49:33 -04:00
//--- Bars sampled per candidate. Tuning cost is candidates x this x features, so it is the one number
//--- that trades accuracy for time.
perf(autotune): replace the genetic search with a filter score - hours to seconds
MEASURED COST OF THE GA, which is what retired it. Per generation:
rung 0: 8 cand x 3 seeds x 3 eras = 72 eras
rung 1: 4 cand x 3 seeds x 8 eras = 96
rung 2: 2 cand x 3 seeds x 20 eras = 120
= 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's
real training began. Against the observed era times on SP500 H1:
PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22)
CONV 41.3 s/era -> 13.2 h
LSTM 150.4 s/era -> 48.1 h
HYBRID 154.6 s/era -> 49.5 h
Two days to tune is not a first-run experience, and it is the phase in
which the panel goes quiet, which is what made it look like a hang.
It also bought nothing. The space is 90 points (10 MA periods x 9 MA
types), so 1152 evaluations revisited each point ~13 times; and rungs of
3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run
proves it: every finalist scored 25.0-25.9% balanced accuracy - below the
33.3% one-class floor, i.e. indistinguishable noise - and the search then
"deployed the winner" of that.
THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full
training run to choose a feature's period is a wrapper method paying
wrapper prices for a decision that does not need one. The reference book
does not do this: ch. 3.3 selects inputs by measuring each candidate
indicator's CORRELATION with the target and dropping the ones with none,
with no network involved.
So: rank candidates by the MUTUAL INFORMATION between the resulting
feature vector and the triple-barrier label. MI rather than correlation
because the label is 3-class categorical and the features are not
monotonically related to it. Equal-FREQUENCY binning (rank-based),
because these features are ATR-normalised and heavy-tailed - fixed-width
bins put nearly everything in one bucket and report ~0 information for a
genuinely useful feature.
Scoring is arithmetic over the feature cache, so it costs seconds and its
cost is independent of topology: LSTM now tunes as fast as the MLP.
Coordinate sweep, not product sweep - cost is the SUM of per-parameter
candidate counts, so enabling every indicator stays affordable - with a
second pass that breaks early once nothing moves.
Sampling is IS-ONLY. Letting the OOS window influence which indicator
settings ship would mean the holdout had been used for selection and had
stopped being a holdout.
HONEST LIMIT, recorded because it is the price: MI is marginal, so a
parameter that only pays off in combination with another can be missed
(Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it
replaces was ranking pure noise at 48 h a run, this is strictly better.
Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/
GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga*
members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget.
AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28
read sites all permanently inert. That is not a tidy-up: the `if
(!m_evalMode)` guard on UpdateClassPriors is exactly what silently
disabled the imbalance correction for entire runs two commits ago. Dead
machinery that still reads like live machinery is this codebase's most
expensive recurring bug, and leaving 28 more instances of it would have
been indefensible.
The panel's tuning-progress state goes too - tuning no longer takes long
enough to need one.
Both builds compile 0 errors / 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
# define MI_SAMPLE_BARS 2000
# define MI_MIN_SAMPLES 200
2026-08-20 09:49:33 -04:00
//--- Eras the MI diagnostics may wait for the cross-asset panel before reporting without it.
2026-08-02 12:25:20 -04:00
# define MI_REPORT_MAX_DEFERRALS 3
feat(pool,mi): one feature layout fleet-wide, and the keep-screen stops self-disabling on a cold start
TWO CHANGES, BOTH RETRAIN-FORCING BY INTENT.
1. SP500 was training alone, and one alt-data column was the reason.
The alt block's width joins the model fingerprint, and the pool reader only
adopts peer rows whose fingerprint and width match. The exporter gives each
instrument the series that apply to it - FX 15 columns, metals/oil 14, SP500
13 - so the fleet ran as three incompatible pools:
EURUSD/USDJPY/USDCAD adopt ~57-60k peer rows each
XAUUSD/XTIUSD adopt 6.4k / 20.3k
SP500 "EVERY peer file was REJECTED, so this chart is
training alone" - 0 rows
SP500 therefore trained on 2279 independent observations against a 600-wide
input with its first layer floored at 16, printing its own "expect
overfitting" warning. It is the one chart with no pool and the worst
capacity ratio in the fleet by a factor of three.
Fresh models now pin ALTDATA_FLEET_COLUMNS - the 12-column intersection -
instead of their own file header. An existing model still adopts its .cfg
pin, so this re-keys nothing that is already trained.
Intersection rather than union: filling an absent series with its median
makes that column constant per instrument, which lets a pooled model
identify the source instrument and stop learning the shared mechanism. It
is also 6 columns narrower. Cost is six columns whose retained information
is UNMEASURED - the keep-screen reports a bitmask nothing has mapped back
to names.
2. The MI keep-screen disabled itself for the whole run on any cold start.
ReportFeatureLabelInformation set m_miReportDone on ENTRY. On a cold start
the label cache is allocated before it is filled, so BuildMiSample finds no
row carrying a resolved label and returns 0 - a sixth exit, and the only
one the 8c1266d instrumentation did not cover, which is why it printed
nothing. observed then stayed -1, the permutation loop never iterated, and
the report emitted "-1.00000 nats over 0 permutations" beside a plausible
"strongest single feature 0.05979" that was a STALE m_miBestColumn from an
earlier scoring call. The first ensemble member propagated the latch to
g_ensembleChartMiReportDone and silenced every member on the chart.
The flag now latches only once a measurement exists. A short sample is
reported as a deferral naming the two numbers that identify it (cached bars
vs bars carrying a resolved label) and retried, up to
MI_REPORT_MAX_ATTEMPTS.
Build tag -> fleet-pool-v1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 17:41:14 -04:00
//--- How many ERAS the screen may find an unusable sample before it gives up for the run. A COLD
//--- start allocates the label cache before it fills it, so BuildMiSample returns 0 usable rows and
//--- the screen measures nothing - see ReportFeatureLabelInformation. That is a transient ordering
//--- condition, not a verdict, so it must be retried rather than latched.
# define MI_REPORT_MAX_ATTEMPTS 8
fix(topology): stop a training-alone size becoming permanent, and stop the keep-screen latching underpowered
1. THE POOL FIX WAS LANDING ON A TOPOLOGY THAT COULD NOT SEE IT.
ComputeFirstLayerWidth budgets against EstimatedInSampleBars, which counts
this chart's own bars PLUS the training pool. On a COLD fleet start every
chart derives and pins its topology BEFORE any chart has published a pool
file - measured on the 18:13 start, model creation at 18:13:21 against a
first publish at 18:13:48. All six sized as if training alone, wrote that
into .cfg, and adopted it back on every later start even with the pool full.
SP500 ran a first layer floored to 16 while adopting 30229 peer rows.
Adopt-don't-compare exists to protect weights shaped by those sizes. It was
also running for a model with NO .nnw, where there is nothing to protect and
the .cfg is just a record of one unlucky moment. The four derived sizes are
now re-measured when no weights exist.
Safe on all three counts that matter: free (nothing to discard), cannot loop
(once weights exist the .cfg is authoritative again), and cannot fragment the
pool - the derived width is NOT in BuildModelFingerprint, which keys only on
the FEATURE layout. Verified: field 2 of the fingerprint is
LEGACY_HISTORY_BARS_SLOT, not the first-layer width.
TO TAKE EFFECT the weights must be wiped while the TrainPool is KEPT - the
census has to be non-empty at derivation time. A full wipe empties the pool
and reproduces the original condition exactly.
2. THE KEEP-SCREEN LATCHED ON AN UNDERPOWERED SAMPLE.
MI_MIN_SAMPLES is a floor for "can this be computed", and it was being used
as the bar for "is this answer final". The screen fired on the first era
clearing 200 rows and latched, measuring at 202-773 samples where a warm
chart gives ~2065. Columns kept then tracked SAMPLE SIZE rather than
information - EURUSD kept 0 of 49 at n=202, SP500 kept 15 at n=773, and the
ordering across all six charts was very nearly monotone in n.
A thin sample is still measured and printed, but it no longer closes the
question: below MI_GOOD_SAMPLE_FRACTION of the target the result is labelled
underpowered and a later era supersedes it, bounded by the same attempt
budget. An underpowered screen that latches is worse than one that waits,
because it looks like a result.
Build tag -> fleet-pool-v2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 19:27:48 -04:00
//--- The sample the screen will WAIT FOR before treating its answer as final, as a fraction of the
//--- MI_SAMPLE_BARS target. MI_MIN_SAMPLES is a floor for "can this be computed at all", and using
//--- it as the bar for "is this worth keeping" cost the 2026-08-26 fresh start: the screen fired on
//--- the first era clearing 200 rows and latched, measuring at 202-773 samples where a warm chart
//--- gives ~2065. Columns kept then tracked SAMPLE SIZE, not information - EURUSD kept 0 of 49 at
//--- n=202 while SP500 kept 15 at n=773. An underpowered screen that latches is worse than one that
//--- waits, because it looks like a result.
# define MI_GOOD_SAMPLE_FRACTION 0.60
feat(topology): re-derive capacity once when the training pool appears
A cold fleet start sizes every model BEFORE any chart has published a pool
file, so the first layer is budgeted as if the chart trains alone and then
pinned to .cfg. This is not a rare race - it is what happens EVERY time the
feature layout changes, because that invalidates the pool and forces a wipe.
Correcting it by hand needs a two-phase start: run the fleet to fill the pool,
stop, wipe the weights while KEEPING the pool, restart so derivation sees it.
That is not something an unattended fleet can do for itself, and getting it
wrong is silent - the models simply stay narrow.
TuneIndicatorsAndTrain now notices that the pool has appeared and re-derives
once, reusing ResetWeights() - the existing tested path that re-measures all
four sizes, rebuilds and rewrites the .cfg. No second copy of that logic.
Bounded on every axis that could make it a loop:
- once per model (the flag is set BEFORE the reset, because ResetWeights
zeroes m_eraCount and the model would otherwise re-qualify forever)
- only while era <= CAPACITY_RESIZE_MAX_ERA, so the discarded eras are worth
nothing
- only on CAPACITY_RESIZE_MIN_GROWTH real growth
- only if the recomputed width actually differs; if it does not, the check
settles itself rather than re-running the census every era
Safe against the one thing that would make it self-defeating: the derived width
is NOT part of BuildModelFingerprint, so a model that resizes does not leave
the pool it resized for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 21:42:35 -04:00
//--- Capacity re-derive: how many eras a FRESH model stays eligible, and how much the pool must have
//--- grown to justify throwing away those eras. The era bound keeps the cost trivial (a re-derive at
//--- era <=8 discards almost nothing) and makes a loop impossible to sustain; the growth factor stops
//--- a trickle of peer rows from triggering a rebuild that changes no width.
# define CAPACITY_RESIZE_MAX_ERA 8
# define CAPACITY_RESIZE_MIN_GROWTH 1.50
2026-08-22 00:24:45 -04:00
//--- Largest |k| the label-alignment scan uses. Padding by |offset| instead shifted the offset
//--- build's starting bar and declared every sound measurement void.
2026-08-02 08:12:47 -04:00
# define MI_ALIGN_MAX_SHIFT 5
2026-08-20 09:49:33 -04:00
//--- Coordinate-descent passes; the loop breaks as soon as a pass changes nothing, so this is a ceiling.
perf(autotune): replace the genetic search with a filter score - hours to seconds
MEASURED COST OF THE GA, which is what retired it. Per generation:
rung 0: 8 cand x 3 seeds x 3 eras = 72 eras
rung 1: 4 cand x 3 seeds x 8 eras = 96
rung 2: 2 cand x 3 seeds x 20 eras = 120
= 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's
real training began. Against the observed era times on SP500 H1:
PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22)
CONV 41.3 s/era -> 13.2 h
LSTM 150.4 s/era -> 48.1 h
HYBRID 154.6 s/era -> 49.5 h
Two days to tune is not a first-run experience, and it is the phase in
which the panel goes quiet, which is what made it look like a hang.
It also bought nothing. The space is 90 points (10 MA periods x 9 MA
types), so 1152 evaluations revisited each point ~13 times; and rungs of
3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run
proves it: every finalist scored 25.0-25.9% balanced accuracy - below the
33.3% one-class floor, i.e. indistinguishable noise - and the search then
"deployed the winner" of that.
THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full
training run to choose a feature's period is a wrapper method paying
wrapper prices for a decision that does not need one. The reference book
does not do this: ch. 3.3 selects inputs by measuring each candidate
indicator's CORRELATION with the target and dropping the ones with none,
with no network involved.
So: rank candidates by the MUTUAL INFORMATION between the resulting
feature vector and the triple-barrier label. MI rather than correlation
because the label is 3-class categorical and the features are not
monotonically related to it. Equal-FREQUENCY binning (rank-based),
because these features are ATR-normalised and heavy-tailed - fixed-width
bins put nearly everything in one bucket and report ~0 information for a
genuinely useful feature.
Scoring is arithmetic over the feature cache, so it costs seconds and its
cost is independent of topology: LSTM now tunes as fast as the MLP.
Coordinate sweep, not product sweep - cost is the SUM of per-parameter
candidate counts, so enabling every indicator stays affordable - with a
second pass that breaks early once nothing moves.
Sampling is IS-ONLY. Letting the OOS window influence which indicator
settings ship would mean the holdout had been used for selection and had
stopped being a holdout.
HONEST LIMIT, recorded because it is the price: MI is marginal, so a
parameter that only pays off in combination with another can be missed
(Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it
replaces was ranking pure noise at 48 h a run, this is strictly better.
Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/
GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga*
members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget.
AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28
read sites all permanently inert. That is not a tidy-up: the `if
(!m_evalMode)` guard on UpdateClassPriors is exactly what silently
disabled the imbalance correction for entire runs two commits ago. Dead
machinery that still reads like live machinery is this codebase's most
expensive recurring bug, and leaving 28 more instances of it would have
been indefensible.
The panel's tuning-progress state goes too - tuning no longer takes long
enough to need one.
Both builds compile 0 errors / 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
# define MI_TUNE_PASSES 2
2026-08-22 00:24:45 -04:00
//--- Null draws for the observed score. Empirical p cannot go below 1/(B+1), so 200 reports
//--- "p<=0.005" and no finer. Cheap: BuildMiSample runs ONCE and every draw reuses it. Counting
//--- ranks estimates no spread.
diag(autotune): five permutations was still a coin flip - use a real test
The 5-draw z-score shipped an hour ago disproved itself on its first run.
All four charts scored the IDENTICAL 0.00401 nats on identical features
and identical labels - and reported z of +1.3, +2.0, +4.0 and +4.7. Two
"AT THE NOISE FLOOR", two "a real association", same data. The entire
swing came from estimating the null's spread from five draws, where the
standard deviation of the standard-deviation estimate is ~35%: the
denominator was noisier than the effect it was judging.
Replaced with an empirical permutation test. 200 draws, p counted by rank
with the +1/(B+1) correction (Phipson & Smyth 2010) so p is never
reported as exactly zero - no normality assumption and no spread to
estimate. The strongest single column is tested against the null
distribution OF THE MAXIMUM, which corrects for scoring 26 features at
once by construction and is far less conservative than Bonferroni.
Affordable because BuildMiSample is now split out of ScoreCurrentParamsByMI
and runs ONCE for the whole test - every draw reuses that sample and costs
a relabel plus 26 histogram passes, not 2000 feature extractions. The
coordinate sweep still calls the combined form, which is correct there:
each candidate changes the indicator settings, so its features really do
have to be re-extracted.
The verdict line keeps both questions apart and prints both answers: the
p-value for "is it real", the excess as a percentage of H(Y) for "is it
big enough to trade". At n=2000 those can disagree, and collapsing them
into one word is how a worthless effect gets called a discovery.
Compiles 0 errors / 0 warnings. Build tag permtest-v1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:45:46 -04:00
# define MI_NOISE_PERMUTATIONS 200
2026-08-20 09:49:33 -04:00
//--- Significance the tuner's winner must reach AFTER correcting for best-of-N. This selector overwrites
//--- the user's indicator settings and forces a fresh topology, so a gate has to exist.
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
# define MI_TUNE_ALPHA 0.05
2026-08-20 09:49:33 -04:00
//--- Standard errors a checkpoint's directional precision must clear chance by to be deployable.
fix(autotune): MI scorer read an array nobody filled; add the permutation floor
THE TUNER WAS A SILENT NO-OP. Every chart logged
auto-tune complete - 17 candidate settings scored in ~139s,
feature/label mutual information 0.0000 -> 0.0000 nats (no improvement)
0.0000 is not a weak result, it is a broken measurement: finite-sample MI
is biased UPWARD, so even pure noise scores above zero. Cause:
ScoreCurrentParamsByMI called BufferTempDataCompute(), which APPENDS the
bar's features to TempData and never touches m_featureCache - only the
caching wrapper BufferTempData() writes that array. It then read
m_featureCache, which ReInitADIndicators had just invalidated. Every
column came back constant, FeatureColumnMI returned 0 for all of them,
and all 17 candidates tied at exactly zero. 139 s per chart to return the
settings it started with.
Now reads the values back out of TempData, where they actually land. And
an exactly-zero best score is called out as a fault rather than reported
as "no improvement", because that is what it is.
ADDED: a PERMUTATION BASELINE, which is the diagnostic this project has
been missing. MI's finite-sample bias is ~(bins-1)(classes-1)/(2n) nats -
at these sample sizes the same order as any real edge in this domain - so
a raw MI figure is uninterpretable on its own. Shuffling the labels
destroys every genuine association while leaving sample size, binning and
class proportions intact, so the score it produces IS this dataset's
noise floor, measured rather than approximated. The log now reads
feature/label information - X nats against a shuffled-label floor of Y
and says outright whether the features carry usable information about the
target. It needs no training, no topology and no convergence, so unlike
every accuracy number in this codebase it cannot be confounded by an
optimizer or an objective. If the score sits on the floor, no change of
architecture can help - which is the question the last three days of
zero-edge results have been circling.
DEPLOY FLOOR: `dirPrecPct > chancePrecPct` passed anything above chance by
any amount. At ~11,000 directional calls the standard error of the
precision estimate is ~0.4pp, so that gate was accepting sub-one-sigma
noise - the perceptron deployed at edge +0pp on 2026-08-01. Now requires
EDGE_MIN_SIGMAS (2.0) standard errors above chance, computed from the
actual call count, so the bar scales with the evidence instead of needing
a hand-picked constant.
Recorded with it, because it is why chance is the right reference at all:
under a driftless random walk P(touch +k*ATR before -m*ATR) = m/(m+k),
and the break-even win rate for a k:m reward:risk trade is ALSO m/(m+k).
The label's own base rate IS the break-even rate, at every SL/TP setting.
So "beats chance" and "is profitable" are the same test, and no choice of
SL/TP can manufacture an edge - only prediction can.
Both builds compile 0 errors / 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:05:50 -04:00
# define EDGE_MIN_SIGMAS 2.0
2026-08-22 00:24:45 -04:00
//--- BOTH-DIRECTIONS FLOOR. The perceptron reported "Sell:0%" in all 41 of its eras, cleared on Buy
//--- alone at 36.6% vs 34%, and deployed.
fix: the Adam second moment was never Adam - all four tiers
Root cause of the B=32 regression, and it predates F4 entirely. Every Adam
kernel stored v already square-rooted and then fed that stored value back in
as if it were the variance:
v_new = sqrt(b2 * v_old + (1 - b2) * g^2)
That recursion has a fixed point at v ~= b2 = 0.999 for ANY gradient below
unit scale, so the denominator stops tracking the gradient and Adam degrades
into plain SGD with lr = lt. Measured against the shipped WarriorCPU.dll
(batch_accum_check.cpp, TestOptimizerScaleInvariance), 4000 steps of a
constant gradient: 3285x less displacement at |g|=1e-5 than at |g|=1, where
a scale-invariant optimizer gives the same distance for both. After the fix
all six magnitudes read 1.199 and v tracks |g| exactly.
It hit conv/LSTM specifically because they sit behind a batch-norm with
running variance ~2.6e+05, so their gradients arrive divided by ~500 - deep
in the degraded regime - while the dense stack near the loss stayed in the
working one. In situ on SP500 H1: lstm1 dW/W 2.62/10.0/7.14% -> 0.024/0.022/
0.003%, conv1 decaying to 0.000% by era 30. NeuronBatchNorm.mqh already
squared v back for gamma/beta and its comment named the kernels as wrong,
which is exactly why gamma/beta kept training while the stages behind froze.
Persisted .nnw needs no migration - v keeps its std-dev meaning.
Also, the two ways F4 exposed it, both mine:
- No LR compensation for B fewer steps per era. sqrt(B) for adaptive methods
(Krizhevsky 2014; Granziol et al. 2022), applied once in
InitialEtaForOptimizer(). Linear scaling (Goyal et al. 2017) is for SGD.
- Plateau patience denominated in eras, so raising B made the ladder 32x more
impatient in its only unit. PAI converged at era 41 on ~49k updates where
the same config had been finding new bests at era 1028.
TrainPlateauPatienceEras() stretches it by the same sqrt(B).
TRAIN_BATCH_SIZE 32 -> 8 so the patience stretch stays affordable (8 -> 23
eras per stage, not 8 -> 45). Both helpers are identities at B=1.
Deploy gate: DEPLOY_MIN_SIDE_RECALL_PCT (10%) folded into tradeableOK. The
perceptron reported Sell:0% recall in all 41 eras, cleared the floor on Buy
alone at 36.6% vs 34% chance, deployed, and sprayed buy arrows. Folded into
the ranking key rather than checked at deploy time so a one-sided era cannot
become best-so-far in the first place.
Deinit: the arrow purge now runs BEFORE ExtPanel.Destroy(), an unbounded
CAppDialog teardown that sat ahead of it - the same ordering inversion the
rule there exists to prevent. CONV was force-terminated 4.8 s into OnDeinit
(vs ~1.1 s for the three that finished) having reached none of its cleanup,
so its arrows stayed on the chart. Steps are now timed in the log.
PurgeChart's verification rescan filtered on OBJ_ARROW, the same blind spot
as the bulk delete, so "persisted 10 ... cleared 0" passed silently. It now
walks every object type and reports the object counts when both are zero.
Both build variants compile 0 errors / 0 warnings; both DLLs rebuilt.
FORCES A RETRAIN (already forced by N1) and both DLLs must ship with the .ex5.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 14:02:35 -04:00
# define DEPLOY_MIN_SIDE_RECALL_PCT 10.0
2026-08-22 00:24:45 -04:00
//--- FAMILY-WISE DEPLOYMENT GATE. EDGE_MIN_SIGMAS is a PER-ERA test and the deployed model is the
//--- MAXIMUM over every era - the one construction this project has repeatedly proven crowns noise.
feat: gate deployment on the null of the MAXIMUM, not the per-era null
EDGE_MIN_SIGMAS is a PER-ERA test and the deployed model is the MAXIMUM
over every era a run ranks. A 2-sigma one-sided test passes on noise with
probability 0.0228 per era, so over N eras the chance at least one clears
it is 1-(1-0.0228)^N: 34% by era 18, 80% by era 70, 93% by era 112. The
gate was near-certain to open on a long run whatever the data held.
It did. HYBRID deployed 2026-08-08 at dir-precision 35.5% vs 34% chance -
+1.5pp, best of 112 eras whose per-era values wandered 30%..35.5%. At the
call counts these runs produce that is p_family 0.92..0.9999.
Every OTHER best-of-N decision here already carries this correction, and
every one REJECTS on this data: the barrier-geometry winner (null of the
maximum over 6, p=0.3902), the indicator tuner (Sidak, p=1.0000), the MI
lag profile (null of the maximum over 21 lags). The one decision that
ships a model to a live account had none.
BestCheckpointSurvivesSelection() re-tests the checkpoint that is about to
deploy:
z = (precision - chance)/SE, SE = sqrt(p0(1-p0)/n)
p_single = P(Z >= z)
p_family = 1 - (1-p_single)^N
against DEPLOY_FAMILY_WISE_ALPHA. It uses the checkpoint's OWN
snapshotted precision/chance/call-count, not the latest era's, because
the model that ships is the one that has to clear the bar.
N counts CANDIDATE eras (coverage measurable, at least one directional
call) - an era that called nothing directional could never have become
the best, so counting it would make the gate stricter than the search
that actually happened.
Conservative on purpose: consecutive eras share OOS bars and differ by
one gradient step, so they are nowhere near N independent draws and the
true family-wise error is below this bound. This gate decides what trades
real money and the house posture is reject-unless-demonstrated.
Effect at 2900 directional calls / N=112: required edge goes 1.76pp ->
2.92pp. A real edge clears it; +1.5pp does not.
Applied to BOTH automatic paths - the plateau ladder's stage-3 deploy and
the m_trainingComplete assignment - which must stay identical or the flag
persisted into the .nnw disagrees with the decision to stop, and a reload
runs inference on a model the ladder refused.
NOT applied to the two operator paths (era-cap deploy, panel Deploy
button). Those stay the operator's call; ReportSelectionGateVerdict()
logs the verdict beside them so an authorised deploy can never later be
misread as a validated one.
NormalUpperTail() is A&S 26.2.17 (|err| < 7.5e-8), self-contained rather
than pulling in Math\Stat. Verified against reference values to 6dp:
Q(1.645)=0.049985, Q(1.96)=0.024998, Q(3.0)=0.001350. Its locals are
ntB1..ntB5 because AI\Network.mqh line 79 does "#define b1 AdamBeta1".
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 18:01:04 -04:00
# define DEPLOY_FAMILY_WISE_ALPHA 0.05
2026-08-20 09:49:33 -04:00
//--- Share of bars one class must hold before the era-0 output-bias seed fires. A +-3.0 bias seed is a
//--- correction at a 94%-Neutral prior and a distortion at a 50% one.
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets
The 31:1 class imbalance was self-inflicted by the TARGET, not a property
of the market. Labelling only the exact bar where a ZigZag pivot confirms
gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism
this codebase accumulated sits downstream of that one choice: the
logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias
seed, balanced-accuracy-then-precision selection with its coverage floor,
the recall floor and its catch-22, the alternation gate, NMS, and the four
oversampling designs that collapsed before them.
The reference this engine is built on (references/neuronetworksbook.pdf
ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT
EXTREMUM on every bar - ~50/50 by construction, with no imbalance to
correct at all. It never had this problem because it never asked "is this
the pivot bar".
Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's
OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its
target before its stop, within a horizon. Buy = long resolves, Sell =
short resolves, Neutral = neither. Consequences:
- dir-precision in the era line stops being a proxy and becomes the win
rate of the strategy under its own exit rules.
- Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e.
~2:1 instead of 31:1. Measured and logged at the end of the prebuild.
- Spread is charged on both legs, so it is a NET win rate.
- Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches
inside one bar and the optimistic reading is how a backtested edge
becomes a live loss.
ZigZag stays as input features (EnableSwingContext) and now also supplies
the vertical barrier: the horizon is the median confirmed leg length,
snapped to a coarse ladder. Derived, not configured, and deliberately kept
out of the filename fingerprint - a filename keyed on a measured quantity
orphans a trained model the moment the measurement moves.
Removed, because the premise died with the old target:
- the alternation gate. Correct for pivot labels (a ZigZag cannot emit two
same-type pivots in a row, so a repeat was provably a false fire), and
wrong for barrier labels, which answer each bar independently. It also
took its worst consequence with it: a one-sided model previously got ONE
trade per backtest, a hard blocker on marketplace validation.
- SignalClusterWindow now defaults off - it de-duplicated repeats that are
now real trades. Kept as an opt-in display control.
- LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel.
- the era-0 output-bias seed now needs a genuinely dominant class (0.70)
rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a
correction.
Also fixed, both found while wiring the above:
1. RefreshConvergedSignal sized its buffers from a date delta
(Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training
watermark; in the tester it is loaded from a live-chart save AHEAD of
the simulated date, so the interval inverted, Bars() returned ~0, and
the buffer came out at exactly m_historyBars - deep enough for the OHLC
window and far too shallow for the Donchian-50 / 20-bar-return / SMA
extension behind it. Inference silently computed DIFFERENT features
from the ones training learned on, live as well as in the tester. Now
sized from what the feature builder actually needs.
2. The barrier horizon is resolved on the deployed path too. A deployed
model never enters Train(), so it never reached the prebuild, and
OnlineLearnStep reads the horizon as its confirmation delay - left at
the fallback it would have backpropped bars whose barriers had not
resolved. Silent lookahead in the one place that writes to a live model.
SL_Mode/TP_Mode join the weights fingerprint: they define the labels now,
so a model trained at 1:3 must never be silently reused at 1:1. This
re-keys every pre-existing model by design - none were trained on this task.
Inference census extended with the vote gate. LongCondition/ShortCondition
open with a readiness check the refresh counters never see; in the tester it
reduces to "the seeded _optcache.nnw must have LOADED", and if it did not,
every vote is hard-zeroed while the model still answers Buy. The old three
counters would have read that as "the model says Neutral" - false, and a
completely different fix. This is the leading candidate for the
zero-direction backtest and the census can now name it in one run.
Both builds compile 0 errors / 0 warnings. Forces a full retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 20:39:49 -04:00
# define COLD_START_SEED_MIN_DOMINANCE 0.70
2026-08-22 00:24:45 -04:00
//--- Reduce-on-regression learning-rate decay. g_eta is read fresh by every weight-update call on
//--- every backend, so shrinking it takes effect on the next backProp() everywhere at once.
2026-07-15 21:47:09 -04:00
# define ETA_DECAY_REGRESSION_PCT 5.0 / / only decay after a real regression , not per - era noise
# define ETA_DECAY_FACTOR 0.7
2026-08-20 09:49:33 -04:00
//--- 1e-5, not 1e-4: against the 3e-4 ceiling the old floor left the schedule a 3x dynamic range, so
//--- three decays pinned it and "reduce LR on regression" could never settle an oscillating run. The
//--- recovery bump still climbs back at 1/0.7 per new best.
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric
Four of the six findings from research/training_pipeline_audit_2026-08-09.md
(F4 mini-batching and F6 feature re-encode deliberately deferred - see the
report's implementation-status section for why):
- F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%,
which is 15-bit - provably non-uniform on every full-history era over 32,768
queued samples. New 30-bit ShuffleRandomIndex().
- F2: plateau warm restarts were a no-op whenever eta already sat at its
ceiling (the normal state of a non-regressing plateau) - the ladder was just
a 24-era countdown. Restarts now overshoot to 5x the ceiling
(PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience
window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has
real range.
- F3: checkpoint restores put weights back but kept the rejected trajectory's
Adam moments, so the optimizer immediately pushed back toward the rolled-back
state (the restore->regress->restore oscillation). CNet::ResetOptimizerState()
zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta
untouched) on every mid-run restore, every boosted restart, and the
deploy-time restore that online learning continues from.
- F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk,
so the selection metric the checkpoint ranking and deploy gate read is a pure
function of the checkpoint instead of partly measuring BN drift. Defensive
unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and
the OOS continual-learning simulation stay adaptive by design.
Compiled clean (0 errors, 0 warnings) via the staged-tree recipe.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 10:54:09 -04:00
# define ETA_MIN 0.00001
2026-08-20 09:49:33 -04:00
//--- ComputeFirstLayerWidth() constants. SECONDS_PER_YEAR is the mean Julian year, MQL5's own
2026-08-22 00:24:45 -04:00
//--- convention. MARKET_OPEN_FRACTION allows for closed hours and weekends - anything in 0.6-0.85
//--- lands on the same ladder rung.
refactor(ai): derive the first dense layer's width instead of asking for it
InitialNeurons was an input whose only defensible value depends on two
things the user cannot see when picking from a dropdown: how wide the input
vector ended up after feature selection, and how much in-sample data the
study period actually yields. Left to a hand-picked constant it was badly
wrong - 500 units against a 420-wide input is 210,500 weights, 72% of a
292,583-weight model, against ~36,500 training bars of which only ~2,236
are directional. That is 6.6 weights per training bar, and it EXPANDS a set
of highly correlated inputs rather than compressing them.
The symptom was already in the logs and had been read as a depth problem:
the shallowest topology consistently beat the deepest (perceptron 52.7%
balanced, hybrid 41.3%). Over-parameterization predicts that ordering just
as well as covariate shift does, and only one of the two had been addressed.
ComputeFirstLayerWidth() budgets roughly one first-layer weight per
in-sample bar. Measured across the configurations in use:
M15 10y -> 256 units, 129,071 weights, 0.73 per bar
H1 10y -> 64 units, 28,727 weights, 0.65 per bar
H4 10y -> 16 units, 7,559 weights, 0.68 per bar
Two design points that matter:
- It estimates in-sample bars from the STUDY PERIOD and timeframe, not
from Bars(). What is downloaded grows over a terminal's lifetime, and a
topology that widened as history filled in would re-key its own weights
file and discard a trained model.
- The result is snapped down to a coarse power-of-two ladder, so the
estimate would have to be wrong by ~2x to change the answer.
Every field it reads is already part of the weights-filename fingerprint,
so the derived value needs no fingerprint entry of its own. The public
setter is removed - it could only have been called after construction, and
would either be ignored or silently re-key the model mid-run.
Where the data cannot support even the floor (D1 over 10 years is under
2,000 bars) it now says so and names the fixes, rather than quietly
training a model with more weights than examples.
The DB config fingerprint drops the term too, which re-keys existing
pattern databases once - correct, since a model an order of magnitude
smaller should not inherit the old one's win-rate history.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 13:01:16 -04:00
# define SECONDS_PER_YEAR 31557600.0
# define MARKET_OPEN_FRACTION 0.72
2026-08-22 00:24:45 -04:00
//--- Ceiling on how much of the head's LOGIT RANGE the logit-adjustment offsets may consume. Menon
//--- et al. assume an UNBOUNDED head.
fix(ai): cap logit-adjustment strength to the head's usable logit range
tau=1.0 inverted the collapse instead of curing it. The head is SIGMOID, so
each output is bounded to [0,1] and the widest logit gap the net can express
between two classes is CLASS_LOGIT_SCALE * (1-0) = 6. The offsets are
tau*log(prior_c), whose spread on this 30:1 imbalance is 3.42 - so tau=1.0
spent 57% of the ENTIRE expressible range on the prior correction.
The network did the only thing available to it: saturate Buy/Sell outputs to
1.0 to overcome a -3.42 training handicap. The offsets are absent at
inference, so that surplus made every bar directional. Measured across all
five still-training charts: Neutral recall 0%, directional calls on ~100% of
bars, win rate 5-7% against a ~6% base rate - no information whatsoever -
while balanced accuracy read a flattering 58-64% because two of its three
terms sat near 95%. OOS accuracy 6%.
Menon et al. assume an unbounded logit head where a 3.42 shift is negligible
against the reachable range. It is not negligible here, so the strength is
now expressed RELATIVE to the range actually available:
tau_eff = min(tau_cfg, LOGIT_ADJUST_MAX_RANGE_FRACTION * SCALE / spread)
At 20% that gives tau 0.35 on this data. Deliberately a fraction rather than
a tau ceiling: it stays correct if CLASS_LOGIT_SCALE changes, if the head
becomes unbounded, or on any symbol whose imbalance differs. The input
remains effective below the cap, so dialling it down needs no rebuild.
Simulated at a signal strength where the task is genuinely learnable, the
precision/recall frontier is monotone: tau 1.0 -> 49.6% call rate at 6.4%
precision (base rate 6.1%, i.e. worthless); tau 0.35 -> 2.0% at 15.5%;
tau 0.15 -> 0.2% at 33.3%. The capped value lands in the same regime the
pre-logit-adjustment run occupied (1-6% of bars at 20-35% win rate).
Also logs the measured priors, the spread, and whether the cap bound.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 23:20:07 -04:00
# define LOGIT_ADJUST_MAX_RANGE_FRACTION 0.20
2026-08-20 09:49:33 -04:00
//--- Minimum directional call rate for deployability, as a FRACTION OF THE TRUE DIRECTIONAL BASE RATE
//--- rather than an absolute percentage - a model calling a direction a quarter as often as one occurs
//--- is sparse but usable; one calling ten times a decade is not, however precise those ten were.
feat(ai): rank checkpoints on directional precision, not balanced accuracy
Balanced accuracy is maximized by exactly the model this system must never
deploy. Measured frontier at fixed signal strength, base rate 6.1%:
tau 0.00 -> calls 0.2% of bars at 27.3% precision, balanced 34.0%
tau 0.35 -> calls 2.0% of bars at 15.5% precision, balanced 36.3%
tau 1.00 -> calls 49.6% of bars at 6.4% precision, balanced 53.5%
It rises monotonically as the model calls MORE and is right LESS, because
two of its three terms are directional recalls that a call-everything model
drives to ~95%, while the Neutral term it sacrifices counts for only a
third. The 2026-07-29 run landed exactly there: balanced 58-64% while
calling a direction on ~100% of bars at a 5-7% win rate against a ~6% base
rate. Only the per-class recall floor stopped those deploying - a guard
doing the job the objective should have been doing - and that same guard
also rejected the genuinely useful sparse-but-precise checkpoints.
Ranking is now DIRECTIONAL PRECISION: of the bars called Buy or Sell, how
many were right. That is what a trading edge is. Two anti-degenerate floors
bracket it, since precision alone is trivially maximized by calling almost
nothing: coverage must reach a fraction of the true directional base rate
(derived, not configured - it adapts to any symbol/timeframe/label rule),
and precision must at least beat that base rate.
Against the same frontier the deploy order inverts from
tau 1.00 > 0.50 > 0.35 > 0.15 (old, worst model first)
to
tau 0.35 > 0.50 > 1.00 (new; 0.00/0.15 rejected on coverage)
Balanced accuracy is kept in the log as a diagnostic and marked as such, so
a run where the two disagree - the signature of an over-caller - is visible
at a glance. MinRecall no longer decides what ships; it now only drives the
diagnostic recall line and is a candidate for removal.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 07:13:08 -04:00
# define MIN_COVERAGE_FRACTION_OF_BASE_RATE 0.25
feat(vote): edge-over-chance currency, no-skill exclusion, checkpoint burn-in
RETRAIN-FORCING and deliberately so. Two independent fixes for the same symptom -
charts that go quiet while others overtrade.
1. THE VOTE CURRENCY IS NOW EDGE OVER CHANCE, not an absolute win rate.
A tier weight is a raw win rate and a raw win rate means nothing without the
chance rate behind it: 30% is strong under a 14% base rate and catastrophic under
50%, yet both entered the mean as "30". That is why the threshold needed
re-tuning every time the label changed - 25 was permissive at ~70% win rates
under the old direction label and a near-unanimity rule at ~30% under the
pivot-event one - and why one chart's 25% was never the same statement as
another's. Subtracting the member's own chance rate makes the units percentage
points of demonstrated edge, comparable across charts, labels and regimes.
Clamped at zero: a below-chance tier is anti-informative, and contributing
negatively would act on a broken model as an inverted oracle rather than
discarding it.
2. A NO-SKILL MEMBER IS NOW ABSENT, NOT ABSTAINING.
Measured on XTIUSD: a Perceptron collapsed to B97/S6/N3, pooled win rate 11.5%
against a 14% chance rate - worse than guessing - and still voting. Three healthy
members voting Sell scored -21.06/0.77 = -27.4 and cleared; with the dead one
voting Buy it became (-21.06+1.44)/0.89 = -22.0 and was BLOCKED. It vetoed its
own ensemble on ~95% of bars, and that WAS the chart's 3.3% coverage. Neither
existing guard caught it: it IS self-ranked and its tier weights were 11-14.
The fix has to remove it from the DIVISOR, not just the sum - an abstainer
contributes weight by design, so zeroing only the contribution makes the dilution
worse. VoteCapableWeight() already means exactly "may this member's weight sit in
the denominator", so the skill test belongs there. ReconstructionWeight() and the
OOS scorer's divisor move with it or the scorer certifies a vote live does not
cast. The skill test reads the PREVIOUS era's measurement - gating this era's
vote on this era's own outcome would be circular.
3. CHECKPOINT BURN-IN (ENSEMBLE_CHECKPOINT_MIN_ERA 20).
XAUUSD deployed the checkpoint from ERA 2, XTIUSD from ERA 4, each after 69 and
65 further eras failed to beat it. Ensemble coverage measures AGREEMENT, and four
models that have barely moved off their initialisation agree almost by
construction - so coverage is inflated exactly when the models know least and
decays as they differentiate (XAUUSD 6.6% at era 8 -> 0.4% at era 75). Since
selectionScore is precision discounted by coverage, an early era outscores every
mature one and the ladder freezes on it.
INTENDED CONSEQUENCE: a chart whose MATURE coverage cannot clear the floor now
refuses to deploy rather than shipping era-2 weights. Fewer deploys, honest ones.
Burn-in eras are also kept out of g_ensCandidateEras (they could not have won, so
counting them inflates the family-wise N and raises the bar for nothing) and out
of g_ensErasSinceBest (or the run reaches "no better vote for N eras" with no
best to beat, exhausting the escalation ladder before the first era may compete).
Every pinned threshold and .stats record is in the OLD currency and is now
meaningless - this forces a fresh start on its own. Nothing needs re-tuning
because the threshold is DERIVED: the sweep re-picks the rung by itself.
Compiled clean; NOT yet run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 15:38:26 -04:00
//--- CHECKPOINT BURN-IN. Eras below this may be SCORED and reported but may not become the joint
//--- checkpoint, and therefore cannot be deployed or pin the vote threshold.
//---
//--- WHY, measured 2026-08-26: XAUUSD deployed the checkpoint from ERA 2 and XTIUSD from ERA 4, each
//--- after 69 and 65 further eras failed to beat it. That is not four models agreeing because they
//--- learned something - at era 2 they have barely moved off their initialisation and their
//--- cold-start priors, so they agree with EACH OTHER almost by construction. Ensemble coverage is a
//--- measure of agreement, so it is inflated at exactly the moment the models know least, and it
//--- decays monotonically as they differentiate:
//---
//--- XAUUSD era 8 6.6% -> era 75 0.4% XTIUSD era 8 9.7% -> era 73 1.4%
//---
//--- selectionScore is precision discounted by coverage, so an early era's trivially-high agreement
//--- outscores every mature era and the ladder freezes on it. The run then spends its whole budget
//--- failing to beat a model that had not trained yet.
//---
//--- CONSEQUENCE, AND IT IS INTENDED: a chart whose MATURE coverage cannot clear the floor will now
//--- refuse to deploy instead of shipping its era-2 weights. That is the honest outcome - the refusal
//--- names coverage, which is the real problem - but it WILL reduce the number of charts that deploy.
//---
//--- Heuristic, not derived: the coverage traces above show differentiation largely done by era
//--- ~15-20. It is deliberately not tied to a plateau stage, because the ladder's own counters are
//--- what this exists to protect.
# define ENSEMBLE_CHECKPOINT_MIN_ERA 20
2026-08-22 00:24:45 -04:00
//--- DIRECTIONAL CONFIDENCE THRESHOLD. The decision RULE carries the trading policy rather than
//--- distorting the loss (Elkan 2001). Fatal HERE because FitDirConfThreshold branches on the SIGN
//--- of (p - break-even), and a memorized curve never shows p < p0, so the get-more-selective
//--- branch could never fire.
feat: fitted directional confidence threshold - selectivity gets a mechanism
The training loss and the selection metric wanted different things and only
the second one knew it. Logit-adjusted cross-entropy has no term for "how
often should I trade", so the head calls a direction on 87-91% of bars. The
selection metric is precision x coverage credit, saturating at the coverage
floor - above the floor extra calls earn NOTHING and only precision counts.
So selection wanted few good calls, the loss produced many mediocre ones, and
all selection could do was pick the least-bad era out of what it was handed.
Nothing pushed the model toward selectivity.
This gives the decision RULE the policy instead of distorting the loss (which
is estimating class probabilities correctly, and a probability estimate should
not be bent to encode a trading policy - Elkan 2001: estimate, then choose the
operating point separately). AdjustedSignalFromSoftmax now abstains unless the
winning direction's softmax margin over its best rival clears a fitted
threshold. Margin, not the winning probability: the latter moves with overall
calibration rather than with how close the decision actually was.
Fitted on IS, applied to OOS and live. Pass 2 already forward-passes every IS
sample, so the margin histogram is harvested there for free (primary
occurrences only, so the oversampled replay queue cannot skew the operating
point); the fit runs at the end of pass 2, BEFORE pass 3, so the deploy gate
grades the thresholded model on bars the threshold never saw. Fitting on
pass 3's own predictions would be choosing the operating point on the data
being graded - the best-of-N error corrected in five other places here.
Objective: maximise IS directional precision subject to still clearing the
SAME coverage floor the deploy gate uses (base rate x 0.25, re-derived
locally so the two cannot drift apart). Swept top-down in one pass; ties go
to the LOWER threshold, since equal precision for less coverage is strictly
worse. Under DIR_CONF_MIN_FIT_CALLS (200) it runs unthresholded rather than
on a guess.
The threshold is part of the MODEL, not the run: captured with
Net.CaptureWeights(), restored with the weights at both restore sites, and
appended to the .cfg under the same length-guard convention so a deployed
model reloads at the operating point its gate actually cleared. A pre-2026-08-09
.cfg reads 0.0, which is exactly the behaviour it was trained under.
Per-era line now prints "@margin>=X.XX" next to coverage, so a coverage drop
can be attributed to the operating point rather than guessed at.
Both build variants compile 0 errors / 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:04:37 -04:00
# define DIR_CONF_THRESHOLD_BINS 50
2026-08-20 09:49:33 -04:00
//--- Below this the histogram is too sparse to pick an operating point from. The model then KEEPS THE
//--- PREVIOUS ERA'S THRESHOLD rather than falling back to 0.0 - "trade every bar" is the most dangerous
//--- setting in the range and must never be what a failed measurement decays to.
feat: fitted directional confidence threshold - selectivity gets a mechanism
The training loss and the selection metric wanted different things and only
the second one knew it. Logit-adjusted cross-entropy has no term for "how
often should I trade", so the head calls a direction on 87-91% of bars. The
selection metric is precision x coverage credit, saturating at the coverage
floor - above the floor extra calls earn NOTHING and only precision counts.
So selection wanted few good calls, the loss produced many mediocre ones, and
all selection could do was pick the least-bad era out of what it was handed.
Nothing pushed the model toward selectivity.
This gives the decision RULE the policy instead of distorting the loss (which
is estimating class probabilities correctly, and a probability estimate should
not be bent to encode a trading policy - Elkan 2001: estimate, then choose the
operating point separately). AdjustedSignalFromSoftmax now abstains unless the
winning direction's softmax margin over its best rival clears a fitted
threshold. Margin, not the winning probability: the latter moves with overall
calibration rather than with how close the decision actually was.
Fitted on IS, applied to OOS and live. Pass 2 already forward-passes every IS
sample, so the margin histogram is harvested there for free (primary
occurrences only, so the oversampled replay queue cannot skew the operating
point); the fit runs at the end of pass 2, BEFORE pass 3, so the deploy gate
grades the thresholded model on bars the threshold never saw. Fitting on
pass 3's own predictions would be choosing the operating point on the data
being graded - the best-of-N error corrected in five other places here.
Objective: maximise IS directional precision subject to still clearing the
SAME coverage floor the deploy gate uses (base rate x 0.25, re-derived
locally so the two cannot drift apart). Swept top-down in one pass; ties go
to the LOWER threshold, since equal precision for less coverage is strictly
worse. Under DIR_CONF_MIN_FIT_CALLS (200) it runs unthresholded rather than
on a guess.
The threshold is part of the MODEL, not the run: captured with
Net.CaptureWeights(), restored with the weights at both restore sites, and
appended to the .cfg under the same length-guard convention so a deployed
model reloads at the operating point its gate actually cleared. A pre-2026-08-09
.cfg reads 0.0, which is exactly the behaviour it was trained under.
Per-era line now prints "@margin>=X.XX" next to coverage, so a coverage drop
can be attributed to the operating point rather than guessed at.
Both build variants compile 0 errors / 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:04:37 -04:00
# define DIR_CONF_MIN_FIT_CALLS 200
2026-08-20 09:49:33 -04:00
//--- Share of the IS span held out to fit the operating point. 15% of ~38k bars is ~5.7k, ~28x the
//--- minimum, so the fit is never sparse. Larger buys precision at a direct cost in training data.
fix: the operating point was fitted on bars the net had memorized
FitDirConfThreshold harvested its margin histogram from pass 2's own
backprop samples. Pairing every fit against the same era's OOS result
shows what that measured:
PAI era 1 IS 25% cov @ 66.1% (-0.8pp) -> OOS 64% (-3pp) gap +2.1pp
PAI era 76 IS 90% cov @ 79.6% (+12.7pp) -> OOS 65% (-2pp) gap +14.6pp
LSTM era 9 IS 77% cov @ 81.6% (+14.6pp) -> OOS 63% (-4pp) gap +18.6pp
The gap grows monotonically while OOS stays flat, so within a handful of
eras the curve stops describing behaviour on unseen bars. That is fatal
here specifically, because the objective branches on the SIGN of
(p - break-even): the memorized curve reads +12pp at 95% coverage, so
coverage x (p - p0) correctly maximises coverage and returns ~0.02 - fire
on every bar. The "p < p0 -> get more selective" branch, which is the
actual regime and the entire point of 983a6a3, could never fire because IS
never showed p < p0.
Carve a calibration slice out of the IS span - DIR_CONF_CALIB_PCT_OF_IS,
purged from backprop by one label horizon on BOTH sides (the far-side
purge is not optional: without it the newest training bars carry labels
partly decided by price action inside the slice, putting the memorization
straight back into the curve). Score it in a new chunked pass 2.5, after
pass 2 has trained and before pass 3 grades - the only position where the
histogram is simultaneously not-trained-on, not-graded, and current with
the weights it will be applied to.
Costs 15% of the training data. Worth it beyond honesty: the deploy gate
needs dirPrecPct > chance + EDGE_MIN_SIGMAS*SE, and a threshold pinned
near zero dilutes any edge concentrated in the confident bars across every
bar the model calls, driving dirPrecPct toward chance by construction. A
threshold that can be selective is the only mechanism by which a small,
concentrated edge could ever clear that gate.
Also: a sparse histogram now KEEPS the previous threshold instead of
resetting to 0.0. A failed measurement must not decay to the most exposed
setting in the range.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 15:58:18 -04:00
# define DIR_CONF_CALIB_PCT_OF_IS 15
refactor(ai): derive the first dense layer's width instead of asking for it
InitialNeurons was an input whose only defensible value depends on two
things the user cannot see when picking from a dropdown: how wide the input
vector ended up after feature selection, and how much in-sample data the
study period actually yields. Left to a hand-picked constant it was badly
wrong - 500 units against a 420-wide input is 210,500 weights, 72% of a
292,583-weight model, against ~36,500 training bars of which only ~2,236
are directional. That is 6.6 weights per training bar, and it EXPANDS a set
of highly correlated inputs rather than compressing them.
The symptom was already in the logs and had been read as a depth problem:
the shallowest topology consistently beat the deepest (perceptron 52.7%
balanced, hybrid 41.3%). Over-parameterization predicts that ordering just
as well as covariate shift does, and only one of the two had been addressed.
ComputeFirstLayerWidth() budgets roughly one first-layer weight per
in-sample bar. Measured across the configurations in use:
M15 10y -> 256 units, 129,071 weights, 0.73 per bar
H1 10y -> 64 units, 28,727 weights, 0.65 per bar
H4 10y -> 16 units, 7,559 weights, 0.68 per bar
Two design points that matter:
- It estimates in-sample bars from the STUDY PERIOD and timeframe, not
from Bars(). What is downloaded grows over a terminal's lifetime, and a
topology that widened as history filled in would re-key its own weights
file and discard a trained model.
- The result is snapped down to a coarse power-of-two ladder, so the
estimate would have to be wrong by ~2x to change the answer.
Every field it reads is already part of the weights-filename fingerprint,
so the derived value needs no fingerprint entry of its own. The public
setter is removed - it could only have been called after construction, and
would either be ignored or silently re-key the model mid-run.
Where the data cannot support even the floor (D1 over 10 years is under
2,000 bars) it now says so and names the fixes, rather than quietly
training a model with more weights than examples.
The DB config fingerprint drops the term too, which re-keys existing
pattern databases once - correct, since a model an order of magnitude
smaller should not inherit the old one's win-rate history.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 13:01:16 -04:00
# define FIRST_LAYER_MIN_WIDTH 16
2026-08-22 00:24:45 -04:00
//--- Conv receptive field in BARS, a structural property of the front-end rather than an input: 3
//--- is the smallest window that can express a turning point (before/at/after), which is what
//--- ZigZag marks.
feat(ai): true multi-bar conv and true sequence LSTM
CONV and LSTM were each configured as a strictly lossier perceptron, which
is exactly what the panel showed: PAI 24% > CONV 18% > HYBRID 12% ~ LSTM
12%, monotone in how much reaches the dense stack (420 / 160 / 32 / 16).
CONV - receptive field 1 -> 3 bars, and the pool is gone.
Reading the reference kernels settled why 34d6aa4 killed CONV.
FeedForwardConv emits POSITION-MAJOR output (matrix_o[out + window_out*i]),
and FeedForwardProof is a flat contiguous max over `window` at stride
`step`. On that layout any window <= window_out maxes ACROSS FILTERS
within one position - it cannot pool over time at all. Our stage used
window = step = filterCount: one max over all 8 filters per position,
discarding 87.5% of the conv output and leaving only the argmax filter
with gradient. That is a property of the reference's layout, not a
porting bug, so there is no correct pool to swap in. Springenberg et al.
ICLR 2015 is the answer already cited in this file: no pooling, get the
hierarchy from strided convolution. The second conv went with it - its
window was counted in raw elements while its comment claimed positions,
so a "2-position" window actually spanned 2 filters of position 0.
Filter count now derives from the WINDOW (RF * features / 2) instead of
one bar, which at RF 3 was under-sizing the stage 3x.
Shape: 20 bars x 21 -> 18 positions x 16 filters = 288.
LSTM - sequence mode back on, forget bias 2.0 -> 1.0.
The forward path rules out the "no gradient" reading of the 2026-07-30
failure: CPU_LSTMSeqForward starts every sample at h_0 = c_0 = 0 and
unrolls that sample's own window, so nothing leaks between shuffled
samples. Flat IS error + Neutral:100% is equally the signature of an
output that does not vary with the input, and that is what bias 2.0
produces: c* = i*g/(1-sigmoid(b)) ~ 8.3*i*g, |c*| ~ 4.2, tanh pinned at
0.9995 with derivative 1e-3, so h_T is near-binary and set by the gate
biases rather than the bars. Choosing 2.0 off the reach sweep was a
method error - reach trades against saturation and the sweep never
measured saturation. 1.0 is the Gers/Jozefowicz/Keras default and leaves
tanh derivative ~0.1.
Both builds 0/0. DLL unchanged (CPU_LSTMSeqForward/Backward already
exported). Forces a retrain of CONV, LSTM and HYBRID - the .nnw pins
architecture.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:42:33 -04:00
# define CONV_RECEPTIVE_FIELD_BARS 3
2026-08-22 00:24:45 -04:00
//--- Sequence-LSTM front-end. 1 = a real recurrence over bars with BPTT, gradient-checked to
//--- 2.3e-10 (DirectML\lstm_seq_gradcheck.cpp). 0 = the pre-2026-07-30 single gate step over the
//--- flattened input.
feat(ai): true multi-bar conv and true sequence LSTM
CONV and LSTM were each configured as a strictly lossier perceptron, which
is exactly what the panel showed: PAI 24% > CONV 18% > HYBRID 12% ~ LSTM
12%, monotone in how much reaches the dense stack (420 / 160 / 32 / 16).
CONV - receptive field 1 -> 3 bars, and the pool is gone.
Reading the reference kernels settled why 34d6aa4 killed CONV.
FeedForwardConv emits POSITION-MAJOR output (matrix_o[out + window_out*i]),
and FeedForwardProof is a flat contiguous max over `window` at stride
`step`. On that layout any window <= window_out maxes ACROSS FILTERS
within one position - it cannot pool over time at all. Our stage used
window = step = filterCount: one max over all 8 filters per position,
discarding 87.5% of the conv output and leaving only the argmax filter
with gradient. That is a property of the reference's layout, not a
porting bug, so there is no correct pool to swap in. Springenberg et al.
ICLR 2015 is the answer already cited in this file: no pooling, get the
hierarchy from strided convolution. The second conv went with it - its
window was counted in raw elements while its comment claimed positions,
so a "2-position" window actually spanned 2 filters of position 0.
Filter count now derives from the WINDOW (RF * features / 2) instead of
one bar, which at RF 3 was under-sizing the stage 3x.
Shape: 20 bars x 21 -> 18 positions x 16 filters = 288.
LSTM - sequence mode back on, forget bias 2.0 -> 1.0.
The forward path rules out the "no gradient" reading of the 2026-07-30
failure: CPU_LSTMSeqForward starts every sample at h_0 = c_0 = 0 and
unrolls that sample's own window, so nothing leaks between shuffled
samples. Flat IS error + Neutral:100% is equally the signature of an
output that does not vary with the input, and that is what bias 2.0
produces: c* = i*g/(1-sigmoid(b)) ~ 8.3*i*g, |c*| ~ 4.2, tanh pinned at
0.9995 with derivative 1e-3, so h_T is near-binary and set by the gate
biases rather than the bars. Choosing 2.0 off the reach sweep was a
method error - reach trades against saturation and the sweep never
measured saturation. 1.0 is the Gers/Jozefowicz/Keras default and leaves
tanh derivative ~0.1.
Both builds 0/0. DLL unchanged (CPU_LSTMSeqForward/Backward already
exported). Forces a retrain of CONV, LSTM and HYBRID - the .nnw pins
architecture.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:42:33 -04:00
# define LSTM_SEQUENCE_MODE 1
2026-08-20 09:49:33 -04:00
//--- Where the dense taper ENDS: small enough to force a compressed representation, comfortably wider
//--- than the decision itself. The floor below covers the regression head, where 4x1 would be absurd.
refactor(ai): derive the dense taper's shape, not just its first layer
Deriving the first layer's width left NeuronsReduction and MinNeuronsCount
behind as inputs calibrated for something that no longer exists. Against a
hand-picked 500-wide first layer "keep 30%, floor at 20" produced a genuine
funnel - 500 -> 150 -> 45. Against the derived 64 it degenerates to
64 -> 20 -> 20: the reduction factor stops mattering after one step, and
"minimum neurons per layer" silently becomes the width of every layer but
the first. Two knobs whose labels no longer describe what they do.
The taper now runs geometrically from the derived first-layer width down to
a final hidden layer sized off the output count, spread evenly over however
many layers the chosen AIType implies:
MLP_3L 64 -> 28 -> 12 -> 3 29,151 dense weights
MLP_4L 64 -> 37 -> 21 -> 12 -> 3 30,450
CONV/LSTM/HYBRID_2L 64 -> 12 -> 3 27,763
and it stays a funnel at the floor, where the old rule could not:
D1 (first layer floored to 16) 16 -> 14 -> 12 -> 3
Both inputs are removed. With the width derived there is no freedom left in
the taper, so keeping either would only let the user contradict the
derivation. The layer COUNT stays selectable, because it is bundled into
AIType alongside the conv/LSTM front-end - depth is an architecture choice,
not a data-derived quantity, and pairing them means the two cannot
contradict each other.
m_minNeuronsCount / m_neuronsReduction survive as frozen members: nothing
reads them to build a topology any more, but they hold positional slots in
the .cfg sidecar and the weights fingerprint, and changing either value
would re-key every model on disk for no behavioural reason.
The DB config fingerprint drops both terms.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 14:03:42 -04:00
# define HIDDEN_TAPER_OUTPUT_MULTIPLE 4
2026-08-20 09:49:33 -04:00
//--- 8, NOT the 20 from the MQL5 "4 hidden layers" article - that floor is load-bearing on ITS 1000-wide
//--- first layer, and this codebase MEASURES the first layer instead (16 units on live SP500 H4). At 16
//--- a floor of 20 makes lastHidden >= m_initialNeuronsCount, so the WIDTH TAPER NEVER RUNS.
feat: derived taper restored; DB ranking reads a reserved slice, shrunk
TOPOLOGY - reverts the two constants and drops CausalHiddenLayerFloor.
The MQL5 article's 30%-per-layer cut and floor of 20 are load-bearing on ITS
first-layer width of 1000 (1000->300->90->27 needs a floor to stop). This
codebase MEASURES that width, and on the live SP500 H4 config it is 16 units -
already floored, with the budget printing "11360 estimated in-sample bars
cannot support a 800-wide input ... roughly 1.1 weights per training bar -
expect overfitting". At 16 units a floor of 20 makes lastHidden >=
m_initialNeuronsCount, so ComputeHiddenLayerCount returns on its first branch
and the width taper - the only part derived from this symbol's data - became
dead code on all four ensemble members, with depth (2 -> 4) set entirely by
counting feature domains. ComputeLayerWidths had already rejected this exact
pair of constants in its own comment.
The causal floor's premise does not hold either: layers are not inference
steps. The "1 layer linear / 2 nonlinear / 3 multi-connected" result is
Lippmann 1987 and is about hard-threshold units; with sigmoid/ReLU, Cybenko
1989 and Hornik 1991 give universal approximation from a single hidden layer.
Depth buys parameter efficiency for compositional functions, not reasoning
hops. ForceHiddenLayers remains for measuring depth directly.
RANKING SLICE - the backfill no longer reads the window it is judged on.
The deployed checkpoint is CHOSEN as the best-scoring era on the OOS window,
so win rates measured back over it are selection-inflated, and the backfill
was writing exactly those into the table filter weights rank on: the
selection set consumed twice, beside a deploy gate that applies a Sidak
correction for that effect. The newest RANK_SLICE_PCT_OF_OOS (20%) of the OOS
window, plus a label-horizon purge, is now reserved and graded by nothing -
not pass 3, not checkpoint selection, not the gate. The backfill reads only
that. The gate keeps ~80% of its measurement (power goes as the square root,
so ~10% of a sigma), and the slice is the newest data, which is the regime
about to be traded. RankSliceBars returns 0 when no honest slice fits and the
backfill then REFUSES and says so, rather than falling back to the scoring
window and looking like a success.
SHRINKAGE - per-tier win rates are shrunk toward the filter's own pooled rate
by MIN_TRADES_FOR_WIN_RATE pseudo-trades before becoming weights. The raw
ratio at the minimum sample count carries a ~15pp standard error, so a tier
that went 8-2 was handed weight 80 and outranked a tier measured over
hundreds of calls at 55 - the ranking was being driven by which small tier got
lucky. Opt-in per call site (priorWeight 0 keeps the raw behaviour).
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 21:49:52 -04:00
# define HIDDEN_TAPER_MIN_WIDTH 8
2026-08-22 00:24:45 -04:00
//--- ComputeHiddenLayerCount() bounds. 2.0 rather than the article's 10/3: halving still produces a
//--- taper at the widths derived here.
feat: derived taper restored; DB ranking reads a reserved slice, shrunk
TOPOLOGY - reverts the two constants and drops CausalHiddenLayerFloor.
The MQL5 article's 30%-per-layer cut and floor of 20 are load-bearing on ITS
first-layer width of 1000 (1000->300->90->27 needs a floor to stop). This
codebase MEASURES that width, and on the live SP500 H4 config it is 16 units -
already floored, with the budget printing "11360 estimated in-sample bars
cannot support a 800-wide input ... roughly 1.1 weights per training bar -
expect overfitting". At 16 units a floor of 20 makes lastHidden >=
m_initialNeuronsCount, so ComputeHiddenLayerCount returns on its first branch
and the width taper - the only part derived from this symbol's data - became
dead code on all four ensemble members, with depth (2 -> 4) set entirely by
counting feature domains. ComputeLayerWidths had already rejected this exact
pair of constants in its own comment.
The causal floor's premise does not hold either: layers are not inference
steps. The "1 layer linear / 2 nonlinear / 3 multi-connected" result is
Lippmann 1987 and is about hard-threshold units; with sigmoid/ReLU, Cybenko
1989 and Hornik 1991 give universal approximation from a single hidden layer.
Depth buys parameter efficiency for compositional functions, not reasoning
hops. ForceHiddenLayers remains for measuring depth directly.
RANKING SLICE - the backfill no longer reads the window it is judged on.
The deployed checkpoint is CHOSEN as the best-scoring era on the OOS window,
so win rates measured back over it are selection-inflated, and the backfill
was writing exactly those into the table filter weights rank on: the
selection set consumed twice, beside a deploy gate that applies a Sidak
correction for that effect. The newest RANK_SLICE_PCT_OF_OOS (20%) of the OOS
window, plus a label-horizon purge, is now reserved and graded by nothing -
not pass 3, not checkpoint selection, not the gate. The backfill reads only
that. The gate keeps ~80% of its measurement (power goes as the square root,
so ~10% of a sigma), and the slice is the newest data, which is the regime
about to be traded. RankSliceBars returns 0 when no honest slice fits and the
backfill then REFUSES and says so, rather than falling back to the scoring
window and looking like a success.
SHRINKAGE - per-tier win rates are shrunk toward the filter's own pooled rate
by MIN_TRADES_FOR_WIN_RATE pseudo-trades before becoming weights. The raw
ratio at the minimum sample count carries a ~15pp standard error, so a tier
that went 8-2 was handed weight 80 and outranked a tier measured over
hundreds of calls at 55 - the ranking was being driven by which small tier got
lucky. Opt-in per call site (priorWeight 0 keeps the raw behaviour).
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 21:49:52 -04:00
# define HIDDEN_TAPER_TARGET_RATIO 2.0
feat(nn): derive dense depth, train on all history, pin the shape in .cfg
Completes the derived-topology work. Three inputs removed.
AIType loses its depth suffix - AI_MLP/AI_CONV/AI_LSTM/AI_HYBRID, five
entries instead of eight. Depth is now derived from the two endpoints
the taper already has to connect (derived first-layer width, output-tied
final width) at a 2x per-layer compression target, clamped [2..5].
Asking a user to pick a layer count while the code derives the widths
those layers taper between was asking for half a decision: at 64 units
tapering to 12, four layers compress by 1.4x per step and five by 1.3x,
so the extra depth bought no abstraction. On the shipping H1/10y default
the derivation lands on 3 layers - the depth that actually won Run 2.
StudyPeriods removed. There is no case for training on less data than
the broker provides at a ~6% directional base rate; the honest
generalization read comes from the OOS holdout, not from withholding
history. Training now starts at the earliest available bar, floored by
MinTrainYear, which answers a different question (excluding dubious
pre-history) and stays.
That required closing the hazard the old code documented: the capacity
budget now MEASURES the symbol's real bar count, and a topology derived
from a measurement would widen as history downloads. Both ends are now
pinned. Every derived value left the weights-filename fingerprint -
keying a filename on a measured quantity means the EA looks for a file
that does not exist, starts from era 0 and orphans a trained model,
silently, because a missing cache is the normal first-run state. The
shape lives in the .cfg instead, where LoadAndCompare now ADOPTS the
four derived fields rather than diffing them; a mismatch there would
discard a fully-trained model over nothing the user did. Two fields
appended to the .cfg for the conv/LSTM stages, length-guarded on read
because FileReadInteger past EOF returns 0 with no error.
ForceHiddenLayers, a compile-time constant like DebuggingMode, pins
depth for diagnostic comparisons. It joins the fingerprint only when
non-zero, so forced depths get their own files - sequential comparisons
only, not simultaneous from one .ex5.
Derived shape, H1/10y defaults (21 features x 20 bars): first layer 64,
3 dense, 8 conv filters, 16 LSTM units. The LSTM block halves from
~58k to ~28k weights.
Both builds compile 0 errors, 0 warnings. Re-keys existing models.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 10:05:40 -04:00
# define MIN_HIDDEN_LAYERS 2
# define MAX_HIDDEN_LAYERS 5
2026-08-20 09:49:33 -04:00
//--- EstimatedInSampleBars() fallback while history is still downloading - below the trusted-bar floor
//--- the measurement says more about the sync state than about the symbol.
feat(nn): derive dense depth, train on all history, pin the shape in .cfg
Completes the derived-topology work. Three inputs removed.
AIType loses its depth suffix - AI_MLP/AI_CONV/AI_LSTM/AI_HYBRID, five
entries instead of eight. Depth is now derived from the two endpoints
the taper already has to connect (derived first-layer width, output-tied
final width) at a 2x per-layer compression target, clamped [2..5].
Asking a user to pick a layer count while the code derives the widths
those layers taper between was asking for half a decision: at 64 units
tapering to 12, four layers compress by 1.4x per step and five by 1.3x,
so the extra depth bought no abstraction. On the shipping H1/10y default
the derivation lands on 3 layers - the depth that actually won Run 2.
StudyPeriods removed. There is no case for training on less data than
the broker provides at a ~6% directional base rate; the honest
generalization read comes from the OOS holdout, not from withholding
history. Training now starts at the earliest available bar, floored by
MinTrainYear, which answers a different question (excluding dubious
pre-history) and stays.
That required closing the hazard the old code documented: the capacity
budget now MEASURES the symbol's real bar count, and a topology derived
from a measurement would widen as history downloads. Both ends are now
pinned. Every derived value left the weights-filename fingerprint -
keying a filename on a measured quantity means the EA looks for a file
that does not exist, starts from era 0 and orphans a trained model,
silently, because a missing cache is the normal first-run state. The
shape lives in the .cfg instead, where LoadAndCompare now ADOPTS the
four derived fields rather than diffing them; a mismatch there would
discard a fully-trained model over nothing the user did. Two fields
appended to the .cfg for the conv/LSTM stages, length-guarded on read
because FileReadInteger past EOF returns 0 with no error.
ForceHiddenLayers, a compile-time constant like DebuggingMode, pins
depth for diagnostic comparisons. It joins the fingerprint only when
non-zero, so forced depths get their own files - sequential comparisons
only, not simultaneous from one .ex5.
Derived shape, H1/10y defaults (21 features x 20 bars): first layer 64,
3 dense, 8 conv filters, 16 LSTM units. The LSTM block halves from
~58k to ~28k weights.
Both builds compile 0 errors, 0 warnings. Re-keys existing models.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 10:05:40 -04:00
# define TOPOLOGY_BUDGET_MIN_TRUSTED_BARS 500
# define TOPOLOGY_BUDGET_FALLBACK_YEARS 10
2026-08-20 09:49:33 -04:00
//--- How much the conv stage compresses one bar's feature vector. The layer is a per-bar projection, so
//--- filters > features EXPANDS a correlated input at the very bottom of the stack.
2026-07-30 09:22:11 -04:00
# define CONV_COMPRESSION_DIVISOR 2
# define CONV_FILTERS_MIN 4
# define CONV_FILTERS_MAX 32
# define LSTM_HIDDEN_MIN 8
# define LSTM_HIDDEN_MAX 128
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- How far above its own chance rate a best-so-far checkpoint must sit before the regression
//--- handler defends it: "still chance-level, keep exploring" versus "a real state we are sliding
2026-08-20 09:49:33 -04:00
//--- off", the case that ran unchecked for 228 eras.
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
# define WORTH_DEFENDING_MARGIN_PCT 5.0
2026-08-22 00:24:45 -04:00
//--- PLATEAU LADDER. So: count eras since the last new best and escalate. ANY new best resets
//--- counter and stage.
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
# define PLATEAU_PATIENCE_ERAS 8 / / eras with no new best selection score before escalating a stage
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric
Four of the six findings from research/training_pipeline_audit_2026-08-09.md
(F4 mini-batching and F6 feature re-encode deliberately deferred - see the
report's implementation-status section for why):
- F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%,
which is 15-bit - provably non-uniform on every full-history era over 32,768
queued samples. New 30-bit ShuffleRandomIndex().
- F2: plateau warm restarts were a no-op whenever eta already sat at its
ceiling (the normal state of a non-regressing plateau) - the ladder was just
a 24-era countdown. Restarts now overshoot to 5x the ceiling
(PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience
window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has
real range.
- F3: checkpoint restores put weights back but kept the rejected trajectory's
Adam moments, so the optimizer immediately pushed back toward the rolled-back
state (the restore->regress->restore oscillation). CNet::ResetOptimizerState()
zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta
untouched) on every mid-run restore, every boosted restart, and the
deploy-time restore that online learning continues from.
- F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk,
so the selection metric the checkpoint ranking and deploy gate read is a pure
function of the checkpoint instead of partly measuring BN drift. Defensive
unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and
the OOS continual-learning simulation stay adaptive by design.
Compiled clean (0 errors, 0 warnings) via the staged-tree recipe.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 10:54:09 -04:00
# define PLATEAU_STAGE_RESTART 1 / / first boosted warm restart ( see PLATEAU_RESTART_BOOST )
# define PLATEAU_STAGE_ANNEAL 2 / / second boosted warm restart ( the gamma anneal it named is gone )
2026-07-25 15:55:56 -04:00
# define PLATEAU_STAGE_DEPLOY 3 / / exhausted : deploy the best checkpoint and finish the run
2026-08-22 00:24:45 -04:00
//--- Restart amplitude. Escaping a basin needs a rate LARGER than the one that settled into it;
//--- SGDR restarts span 10-100x, this is tamer because MAX_WEIGHT_DELTA and the checkpoint restore
//--- already bound the blast radius.
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric
Four of the six findings from research/training_pipeline_audit_2026-08-09.md
(F4 mini-batching and F6 feature re-encode deliberately deferred - see the
report's implementation-status section for why):
- F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%,
which is 15-bit - provably non-uniform on every full-history era over 32,768
queued samples. New 30-bit ShuffleRandomIndex().
- F2: plateau warm restarts were a no-op whenever eta already sat at its
ceiling (the normal state of a non-regressing plateau) - the ladder was just
a 24-era countdown. Restarts now overshoot to 5x the ceiling
(PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience
window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has
real range.
- F3: checkpoint restores put weights back but kept the rejected trajectory's
Adam moments, so the optimizer immediately pushed back toward the rolled-back
state (the restore->regress->restore oscillation). CNet::ResetOptimizerState()
zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta
untouched) on every mid-run restore, every boosted restart, and the
deploy-time restore that online learning continues from.
- F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk,
so the selection metric the checkpoint ranking and deploy gate read is a pure
function of the checkpoint instead of partly measuring BN drift. Defensive
unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and
the OOS continual-learning simulation stay adaptive by design.
Compiled clean (0 errors, 0 warnings) via the staged-tree recipe.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 10:54:09 -04:00
# define PLATEAU_RESTART_BOOST 5.0
fix(training): a new best must beat the noise; the blank-chart census must name its cause
TWO INDEPENDENT BLOCKERS, both of which make the EA look like it is working.
1. THE LADDER NEVER ADVANCES. isBetter/isBetterEra compared selectionScore with
a bare `>`. selectionScore is a win rate over a few hundred independent
calls, so it moves several points era to era on noise alone - measured on
SP500 H4 today: 32.8 / 32.2 / 31.6 / 29.6 / 31.4 across consecutive eras, a
~3-point spread with no trend. Any upward blip was recorded as a new best,
which reset BOTH the plateau counter and the stage, which re-armed a x5
learning-rate warm restart, which injected fresh noise and produced the next
blip. The search sustained itself on its own variance and never reached
PLATEAU_STAGE_DEPLOY - the reported "thousands of eras without converging".
A new best now has to clear the incumbent by PLATEAU_NEW_BEST_SIGMAS (2.0)
times precSE, which the deploy gate already computes. 2.0 rather than 1.0
because incumbent and challenger are both noisy, so the SE of the difference
is ~sqrt(2) x SE, and a 1-SE band was already measured too narrow in a
noise-dominated search. Applied at BOTH ranking sites - the ensemble's and
the solo member's - which are documented as the same ordering. The first
scoring era still checkpoints unconditionally.
2. THE BLANK-CHART CENSUS WAS LYING. It printed "No member has a completed era
yet (snapshots fill at each member's first pass-3 completion)" while the
members were on era 23, because it inferred the cause from m_overlayVotedBars
alone - and that counter requires BOTH a non-zero divisor AND a non-zero net.
Three different states collapsed into one sentence. Split out
m_overlayHadDataBars (divisor non-zero) so the line names which it is:
hadData == 0 -> nobody published a snapshot: publication/index
hadData > 0, voted == 0 -> members looked and abstained: calibration
voted > 0, drawn == 0 -> the vote never cleared the threshold
Diagnostic only. It does not fix the missing arrows - it identifies which of
the three is happening, which the current line actively obscures.
Compile-verified in the staging copy: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 15:58:34 -04:00
//--- HOW MUCH A NEW BEST HAS TO WIN BY, in SEs of the score it is beating.
//---
//--- Zero here - a bare `score > best` - is why a run can spend thousands of eras without ever
//--- reaching PLATEAU_STAGE_DEPLOY. selectionScore is a win rate over a few hundred independent
//--- calls, so it moves several points era to era on noise alone; any upward blip is recorded as a
//--- new best, which resets BOTH the counter and the stage, which re-arms a x5 warm restart, which
//--- injects fresh noise and produces the next blip. The search sustains itself on its own variance
//--- and the ladder never advances. Observed on SP500 H4 2026-08-24: 32.8 / 32.2 / 31.6 / 29.6 /
//--- 31.4 across consecutive eras, a ~3-point spread with no trend.
//---
//--- 2.0 rather than 1.0 deliberately: the incumbent and the challenger are BOTH noisy estimates,
//--- so the SE of their difference is about sqrt(2) x SE, and a 1-SE band was already measured to
//--- be too narrow in a noise-dominated search (see the operating-point plateau note). The cost of
//--- being too wide is only that the run ratchets less often and finishes sooner, which is the
//--- behaviour that was missing.
# define PLATEAU_NEW_BEST_SIGMAS 2.0
2026-08-27 13:46:48 -04:00
//--- HOW HARD THE DEPLOY-TIME PASS OVER THE HELD-OUT SLICE PUSHES - see EnableOosFinalPass. It covers
//--- the WHOLE slice (13-15k bars measured live), so at the model's own converged rate it is a full
//--- training epoch on a model that has already been selected and certified. A quarter step keeps it
//--- a refinement. Raise toward 1.0 only with a measured reason.
# define OOS_FINAL_PASS_ETA_SCALE 0.25
2026-08-20 09:49:33 -04:00
//--- IN-SAMPLE EARLY STOP. Training error is noisy per era - mini-batch order alone moves it - and
//--- ending a run that is still learning costs far more than a few wasted eras.
feat(search): stop on the IN-SAMPLE plateau, and shrink every best-of-K effect before quoting it
Points 3 and 4 of the four-point plan.
1. IN-SAMPLE EARLY STOP - and the reason it is worth having is not compute.
The plateau ladder stops on the OOS SELECTION score. That is a peek: by the time it
fires, every one of those eras has been evaluated out of sample, so all of them sit
in the family the deploy gate corrects over (g_ensCandidateEras, Sidak). Training
longer therefore does not merely cost time - it RAISES the bar the eventual winner
has to clear.
The new stop reads the TRAINING error, which the gate never looks at. When the
optimiser has stopped improving on data it can see, more eras will not find a better
model; they will only enlarge the OOS family. Ending there shrinks the correction,
and the shrinkage is legitimate precisely BECAUSE the stopping rule never consulted
an out-of-sample number.
That distinction is the whole point and it is the one this project has got wrong four
times: stop on IS and the family really is smaller; stop on OOS and those eras were
searched and still count. Both stops now exist; only this one buys a lower bar.
Deliberately more patient than the OOS ladder (IS_ERROR_PATIENCE_MULT = 3x): training
error is noisy per era - mini-batch order alone moves it - and ending a run that is
still learning costs far more than a few wasted eras. Improvement is RELATIVE
(IS_ERROR_IMPROVE_FRAC = 1%), so it does not depend on the loss's absolute scale, and
it only acts when a checkpoint exists, since otherwise it would end a run with
nothing to deploy. Reset per RUN alongside the ladder, so a resumed run cannot
early-stop on its first era against a previous run's best.
2. WINNER'S-CURSE SHRINKAGE ON THE BARRIER-GEOMETRY WINNER.
The family-wise permutation gate already establishes that the RANKING is not noise.
It says nothing about the SIZE of the winner's effect - and a best-of-K maximum is
biased upward by construction, being the largest of K noisy draws. The adoption
message quotes that raw maximum and compares it against the incumbent, so the number
a reader plans on is the inflated one.
The penalty is now measured, not assumed: the same permutation draws that produce the
p-value also produce, per draw, the MAXIMUM excess across all candidates under pure
noise. The mean of those maxima is exactly what a best-of-K selection is expected to
report when there is nothing there. This is the empirical form of the sqrt(2 ln K) x SE
penalty the SQX EdgeFinder plugin applies to every maximum it reports (Stats.java:79-88),
and it needs no normality assumption because the draws ARE the null distribution.
Applied in James-Stein form - effect x max(0, 1 - penalty^2/effect^2) - so a large
effect is nearly untouched and a marginal one collapses toward zero.
Reported, not gated. The adoption decision still turns on the permutation p-value,
which is the right test for "is the ranking real"; the shrunk number is there so the
magnitude quoted beside it is one worth planning on. Closes the first of the two
EdgeFinder ports identified on 2026-08-12.
NOTE on the second EdgeFinder port, deliberately not done here: "let the measurement
steer the target" is already true where it matters most - ReportGeometryExpectancyScan
ADOPTS the winning barrier geometry under the family-wise gate rather than advising
it, and the MI excursion suite publishes a verdict per instrument per config. What is
still missing is steering the TRAINING TARGET itself (direction vs excursion) off
those verdicts, and that is a design change rather than a surgical one - direction is
a closed verdict while excursion SIZE keeps clearing, so the honest version of that
change is a target-selection policy, not a flag.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:29:00 -04:00
# define IS_ERROR_IMPROVE_FRAC 0.01
# define IS_ERROR_PATIENCE_MULT 3
2026-08-22 00:24:45 -04:00
//--- FILE-COMPATIBILITY SHIMS for three removed inputs. So the old default is still written and
//--- hashed, and the .cfg field is no longer COMPARED on load - a model saved under any previous
//--- value still loads.
2026-07-25 15:55:56 -04:00
# define LEGACY_CONVERGE_WR_SLOT 80
feat(nn): derive dense depth, train on all history, pin the shape in .cfg
Completes the derived-topology work. Three inputs removed.
AIType loses its depth suffix - AI_MLP/AI_CONV/AI_LSTM/AI_HYBRID, five
entries instead of eight. Depth is now derived from the two endpoints
the taper already has to connect (derived first-layer width, output-tied
final width) at a 2x per-layer compression target, clamped [2..5].
Asking a user to pick a layer count while the code derives the widths
those layers taper between was asking for half a decision: at 64 units
tapering to 12, four layers compress by 1.4x per step and five by 1.3x,
so the extra depth bought no abstraction. On the shipping H1/10y default
the derivation lands on 3 layers - the depth that actually won Run 2.
StudyPeriods removed. There is no case for training on less data than
the broker provides at a ~6% directional base rate; the honest
generalization read comes from the OOS holdout, not from withholding
history. Training now starts at the earliest available bar, floored by
MinTrainYear, which answers a different question (excluding dubious
pre-history) and stays.
That required closing the hazard the old code documented: the capacity
budget now MEASURES the symbol's real bar count, and a topology derived
from a measurement would widen as history downloads. Both ends are now
pinned. Every derived value left the weights-filename fingerprint -
keying a filename on a measured quantity means the EA looks for a file
that does not exist, starts from era 0 and orphans a trained model,
silently, because a missing cache is the normal first-run state. The
shape lives in the .cfg instead, where LoadAndCompare now ADOPTS the
four derived fields rather than diffing them; a mismatch there would
discard a fully-trained model over nothing the user did. Two fields
appended to the .cfg for the conv/LSTM stages, length-guarded on read
because FileReadInteger past EOF returns 0 with no error.
ForceHiddenLayers, a compile-time constant like DebuggingMode, pins
depth for diagnostic comparisons. It joins the fingerprint only when
non-zero, so forced depths get their own files - sequential comparisons
only, not simultaneous from one .ex5.
Derived shape, H1/10y defaults (21 features x 20 bars): first layer 64,
3 dense, 8 conv filters, 16 LSTM units. The LSTM block halves from
~58k to ~28k weights.
Both builds compile 0 errors, 0 warnings. Re-keys existing models.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 10:05:40 -04:00
# define LEGACY_STUDY_PERIOD_SLOT 0
2026-08-11 21:53:37 -04:00
# define LEGACY_HISTORY_BARS_SLOT 20
fix(topology): size the network against observations, not bars
The capacity budget is stated in weights per INDEPENDENT observation
and divides by the mean label lifespan to get there. It never once
did: EstimatedInSampleBars() deflates via m_labelOverlap, but it is
only ever called from InitNeuralNetwork, where the label cache does
not exist yet (that same function sets m_labelCachePrebuilt = false
a few lines below), so MeanLifespan() returned its "nothing measured"
default of 1.0 at every call. Every fresh model was sized as though
its labels did not overlap - over-budgeting the first dense layer by
a factor of L, which is several rungs of a power-of-two ladder. The
"expect overfitting, reduce the feature set or pool instruments"
warning is the branch that should fire on H1 and structurally could
not.
Fixed at the source rather than by reordering the boot sequence (the
prebuild is chunked across Train() calls and cannot complete inside
init): MeasureSwingGeometry() walks the ZigZag ONCE at init and
answers both questions from it - the median leg gives the window,
and the leg series gives the mean label lifespan analytically.
SwingPivotDirectionLabel resolves bar i when the SECOND pivot after
it commits, so a bar d bars before pivot P waits d + (the leg
leaving P); summed over every bar of every leg that is exactly the
mean the label walk accumulates.
That also closes the coherence gap the swing target opened: the
window was measured with a private +/-12-bar fractal while the label
aimed at ZigZag(12,5,3) pivots, so it was sized against a leg
distribution the label never used. One pivot source now, the
label's.
Also:
- ResetWeights() re-derives the shape. It rebuilt from the members a
history-starved init had pinned and re-saved them - so the "let
history download, then reset from the panel" advice in both
fallback warnings did nothing at all.
- The CAPACITY line prints the measured lifespan beside the one the
topology was sized for, and warns when they differ by more than a
ladder rung. That is the check that makes the estimator falsifiable.
- Topology reads the view's symbol, not _Symbol (latent for pooling).
- Unmeasured geometry defaults to HISTORY_BARS_FALLBACK, never 1.0:
under-sizing is recoverable, over-sizing silently is not.
Compile: 0 errors, 0 warnings (stage).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 21:21:05 -04:00
//--- Derived-window rule: median ZigZag leg, snapped DOWN to the ladder, capped. The same walk
//--- measures the mean label lifespan the capacity budget divides by - CTopology::MeasureSwingGeometry.
2026-08-11 21:53:37 -04:00
# define HISTORY_BARS_FALLBACK 20
feat(topology): cap the input window at 6 bars for capacity - 588 inputs -> 294
Three charts (SP500, XAUUSD, XTIUSD) sat on the FIRST_LAYER_MIN_WIDTH floor
even after pooling took SP500 from 4.1 to 1.8 weights per independent
observation. ComputeFirstLayerWidth needs width <= ~331 to clear it; 49 columns
x 12 bars = 588.
TWO QUESTIONS, AND THE WINDOW IS NOW THE SMALLER ANSWER. The ZigZag ladder
answers "how far back is a swing worth looking" and says 12. The capacity
budget answers "how far back can this much data support" and says 6. Taking the
min stops the first writing a cheque the second cannot cover.
WHY THE LAG AXIS AND NOT THE COLUMN AXIS - the choice was between this and a
per-column mask (designed, parked on feature/column-mask):
- On the LAG axis there is a measured null. The corrected lag profile finds no
linear structure at any lag within +/-50, on all six charts, family-wise
p=1.0000, argmax scattered across different columns and lags per chart.
- On the COLUMN axis the two measures that would justify a mask - marginal MI
retention and variance share - are explicitly blind to joint and temporal
structure, and the columns they would delete include the entire price core,
which is the one place such structure would plausibly live.
Cutting where there is a measured null beats cutting where the instrument
cannot see. Corroborating: PAI/CONV/LSTM/HYBRID score within ~1pp of each
other, so the temporal machinery is not visibly earning the deeper lags.
THE CAP IS A FLEET CONSTANT, NOT A PER-CHART DERIVATION. Pool rows are keyed on
`bars x columns`, so a capacity cap computed from a chart's own observation
count would differ across the fleet by construction and hand every chart its
own layout, its own fingerprint and its own pool of one - exactly what orphaned
SP500. Set from the most starved chart; every chart shares it.
Conv survives: CONV_RECEPTIVE_FIELD_BARS is 3, so a 6-bar window still leaves 4
sliding positions. LSTM sequence length becomes 6.
RETRAIN-FORCING and POOL-INVALIDATING: width changes, so old .nnw and old
TrainPool rows are both incompatible. Wipe both - which puts the fleet back in
the cold-start condition 6c2959d was written for, and will exercise it.
Build tag -> window6-v1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 06:16:36 -04:00
# define HISTORY_BARS_FLOOR 6
//--- THE CAPACITY CAP ON THE INPUT WINDOW, AND WHY IT IS A FLEET CONSTANT (2026-08-27).
//---
//--- The ZigZag derivation answers "how far back is a swing worth looking", and it says 12. The
//--- capacity budget answers "how far back can this much data SUPPORT", and at 49 columns it says 6:
//--- ComputeFirstLayerWidth needs width <= ~331 to clear FIRST_LAYER_MIN_WIDTH, and 49 x 12 = 588.
//--- Three charts (SP500, XAUUSD, XTIUSD) sat on that floor even after pooling took SP500 from 4.1 to
//--- 1.8 weights per independent observation. The window is the min of the two answers.
//---
//--- IT CANNOT BE DERIVED PER CHART. Pool rows are keyed on `bars x columns`, so a per-chart window
//--- gives each chart its own layout, its own fingerprint and its own pool of one - which is exactly
//--- what orphaned SP500 and cost it every peer row it could have had. A capacity cap computed from a
//--- chart's own observation count would differ across the fleet by construction. So it is a
//--- constant, set from the most starved chart, and every chart shares it.
//---
//--- WHY THE LAG AXIS RATHER THAN THE COLUMN AXIS: the corrected lag profile finds NO linear
//--- structure at any lag within +/-50, on all six charts, family-wise p=1.0000 - a measured null on
//--- the axis being cut. The column-side measures (marginal MI, variance share) are explicitly blind
//--- to joint and temporal structure, and the columns they would delete include the whole price core.
//--- Cutting where there is a measured null beats cutting where the instrument cannot see. Corroborating:
//--- PAI/CONV/LSTM/HYBRID score within ~1pp of each other, so the temporal machinery is not visibly
//--- earning the deeper lags. See project_lag_profile_verdict and project_feature_keep_screen.
# define HISTORY_BARS_CAPACITY_CAP 6
2026-08-11 21:53:37 -04:00
# define WINDOW_DERIVE_SPAN_BARS 20000
# define WINDOW_DERIVE_MIN_LEGS 30
2026-08-20 09:49:33 -04:00
//--- True OOS samples a class needs before its recall is trusted as a real pass - enough to rule out the
//--- zero-sample degenerate case that produced a false convergence at eras 44-46.
2026-07-17 09:11:42 -04:00
# define MIN_OOS_CLASS_SAMPLES_FOR_GATE 10
2026-08-22 00:24:45 -04:00
//--- Consecutive regressing eras before the checkpoint is restored and g_eta decayed. Patience is
//--- the standard ReduceLROnPlateau formulation and restores that exploration.
fix: the recall gate was unsatisfiable and the LR decay was a spiral
Both made the run structurally unable to succeed, independently of any
signal in the data. Found by reading the 13:01 log.
RECALL GATE. m_objectiveMet required Buy, Sell AND Neutral OOS recall
each >= 40%. First-touch resolution (ce52654) collapsed Neutral from
the ~94% majority it was under exact-pivot labels to a same-bar-tie
residue - 250 of 38,261 bars, 0.65% - so the floor was asking the model
to identify 40% of coin-flip ties before it could converge. Measured:
CONV, LSTM and HYBRID all logged "Neutral:0% (need >=40% each)" on
every era. No model could ever satisfy it; every run was destined for
the plateau ladder or the era cap.
Only the DIRECTIONAL floors are load-bearing for the anti-collapse job
the gate exists to do: an all-Neutral model shows Buy and Sell recall
at 0% and is blocked by them. Neutral's own floor guarded the mirror
bias (over-calling Buy/Sell at Neutral's expense), which was real at
94% prevalence and is not at 0.65% - there, almost never calling
Neutral is correct rather than biased.
Prevalence-guarded rather than hardcoded off, so it returns by itself
if a future label rule makes Neutral substantial again. Deliberately
NOT extended to Buy/Sell: exempting a thin directional class reopens
the era-44-46 hole, which directionalRecallMeasured only half-covers -
it checks those classes were MEASURED, not that they passed.
ETA DECAY. A regressing era restored the checkpoint, reset the
optimizer and cut eta - all on the FIRST regression. The next era then
started from an identical state with a smaller step, regressed again,
and got the same treatment. The loop is self-sustaining and cannot
discover anything, because rolling the weights back is exactly what
removes the exploration that would end it.
Measured on PAI: eras 2-11 every one a regression against era 1, eta
0.000594 -> 0.000024, dW/W 0.000%/0.000% from era 2 onward. Ten eras,
~45s each, reproducing era 1 exactly and unable to do anything else.
Now requires ETA_DECAY_PATIENCE_ERAS consecutive regressions - the
standard ReduceLROnPlateau formulation. A single bad era is noise, and
an improving era clears the counter so alternating runs never
accumulate into a decay.
Build tag -> gate-patience-v3. It had not moved in six commits, which
is why the running binary could not be identified from its own log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 13:28:58 -04:00
# define ETA_DECAY_PATIENCE_ERAS 3
2026-08-20 09:49:33 -04:00
//--- Bound on FindConfirmedZigZagPivot()'s backward scan. Generous rather than tight: the scan is plain
//--- array reads and its result is cached per bar, so a long scan is paid at most once per unique bar.
2026-07-19 11:04:38 -04:00
# define SWING_SCAN_CAP_BARS 750
feat(label): pivot-EVENT target replaces direction-to-next-pivot
The old target asked "which way is the next pivot", which every bar of a
~13-20 bar leg answers identically - so the net could not tell a fresh turn
from mid-trend and learned the prevailing direction instead. Its own
zero-skill reference showed it: chance sat at 56/44, i.e. the label WAS the
drift, and the gate's standing warning ("a model that only reproduces it has
found the drift, not an edge") applied to the target itself.
Buy now means a swing LOW commits within PIVOT_LABEL_TOLERANCE_BARS bars,
Sell a swing HIGH, Neutral no turn that close. Pivot type is read from
ZigZagBuffer[p] == Low[p], exact by construction in ZigZag.mq5. The existing
P1-final-once-P2-commits rule is kept and now also settles the NEGATIVE
verdict, so the Neutral majority is permanent rather than provisional.
Measured on a full fresh run, all 6 charts:
class balance 56/44/~0 -> 13.7/13.7/72.6 (imbalance 5.3:1)
label overlap ~31 bars -> 5 bars
independent obs 368-1086 -> 2331-7032
weights/obs 9.2-26.2 -> 1.1-4.2
coverage 100% of bars -> 17-48%
23 of 24 models fire all three classes at precision 18-32% vs 13-15%
chance; SP500's ensemble reaches DEPLOYABLE (32.3% vs a 24.0% bar).
Two bindings had to move with the label:
- The capacity deflator. m_swingLifespan fed EstimatedInSampleBars() as
raw/31, measured from the legs. Overlap is now a property of the LABEL -
one turn is callable by exactly the tolerance window - so it is the
window, not a leg measurement. Missing this would have kept every model
sized for a sixth of its real evidence.
- A dormant cold-start seed. Labels.mqh seeds the output bias toward the
dominant class above COLD_START_SEED_MIN_DOMINANCE (0.70); at 56/44 it
never armed, at 72.6% Neutral it does - writing a fixed +-3.0 against a
true prior spread of ~1.75, which would start every net predicting Neutral
~95% of the time. Now seeds the measured log-prior, zero-centred and
capped by the same guard rail the logit adjustment uses (Lin et al. 2017).
TGT:SWG1 -> TGT:PVT1:<tolerance>, with the window in the token because it is
part of the label: every .nnw is invalidated and the fleet retrains.
Depth is still gated, and now for a precise reason: the first dense layer
stays at FIRST_LAYER_MIN_WIDTH because budget = effN/(inputWidth+1) is 11.2
at input 624. Reaching the next rung needs inputWidth <= ~218, i.e. feature
pruning - not architecture.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 00:16:16 -04:00
//--- HOW CLOSE THE TURN HAS TO BE for a bar to be labelled as calling it. A bar is Buy when a swing LOW
//--- commits within the next PIVOT_LABEL_TOLERANCE_BARS bars, Sell for a swing HIGH, Neutral otherwise.
//---
//--- NOT 1 (the exact next candle), deliberately, and the size is SET BY THE CLASS BALANCE IT
//--- PRODUCES rather than by taste. The measured ZigZag leg here is ~20 bars (MeasureSwingGeometry's
//--- mean is 1.5*L+0.5, and it logs ~30.9), so pivot density is ~5% per bar and a T-bar window puts
//--- roughly 5*T% of bars in the two directional classes combined.
//---
//--- WHY THAT MATTERS: ApplyLogitAdjustment's tau is capped at
//--- LOGIT_ADJUST_MAX_RANGE_FRACTION * CLASS_LOGIT_SCALE / spread, so the correction it can apply
//--- against a dominant Neutral is PINNED AT 1.2 LOGITS whatever the imbalance - the cap binds at
//--- every window size. What changes with T is how much imbalance is left over:
//--- T=1 5/5/90 needs 2.89, gets 1.20 -> 5.4x residual bias toward Neutral
//--- T=2 10/10/80* needs 2.89, gets 1.20 -> 5.4x (*5/5/90, see above)
//--- T=5 12/12/75 needs 1.83, gets 1.20 -> 1.9x
//--- 5 is the smallest window whose leftover bias the head can plausibly train through. Below it the
//--- directional classes risk never firing - the failure this project already paid for once, when the
//--- same loss at a comparable ratio produced recall Buy:1% Sell:0% Neutral:100% (1b5a412).
//---
//--- Widen this if the directional classes still collapse; narrow it to sharpen entries at a real cost
//--- in positives. The prebuild census line prints the ACTUAL per-symbol balance and imbalance ratio -
//--- prefer it to the estimate above, which assumes one leg length for every instrument.
//--- CHANGING IT CHANGES THE LABEL - the TGT tag in BuildModelFingerprint() interpolates this value
//--- for that reason, so a change re-keys every model file instead of silently resuming onto a target
//--- the weights were never fitted to.
# define PIVOT_LABEL_TOLERANCE_BARS 5
2026-08-20 09:49:33 -04:00
//--- EMA shadow-weight deployment blend rate - see m_shadowNet. 0.01 matches the Tau range used for
//--- target-network soft updates in the Gizlyk reference RL algorithms: small enough that no single
//--- era's raw weights move the deployed model far, large enough to track sustained learning.
2026-07-15 21:47:37 -04:00
# define SHADOW_WEIGHT_TAU 0.01
2026-08-22 00:24:45 -04:00
//--- MINI-BATCH SIZE. 1 restores the exact per-sample SGD this engine had until 2026-08-09, and
//--- every helper below is an identity there. Fewer steps need a larger step - sqrt(B) for adaptive
//--- methods (Krizhevsky 2014; Granziol et al.
fix: the Adam second moment was never Adam - all four tiers
Root cause of the B=32 regression, and it predates F4 entirely. Every Adam
kernel stored v already square-rooted and then fed that stored value back in
as if it were the variance:
v_new = sqrt(b2 * v_old + (1 - b2) * g^2)
That recursion has a fixed point at v ~= b2 = 0.999 for ANY gradient below
unit scale, so the denominator stops tracking the gradient and Adam degrades
into plain SGD with lr = lt. Measured against the shipped WarriorCPU.dll
(batch_accum_check.cpp, TestOptimizerScaleInvariance), 4000 steps of a
constant gradient: 3285x less displacement at |g|=1e-5 than at |g|=1, where
a scale-invariant optimizer gives the same distance for both. After the fix
all six magnitudes read 1.199 and v tracks |g| exactly.
It hit conv/LSTM specifically because they sit behind a batch-norm with
running variance ~2.6e+05, so their gradients arrive divided by ~500 - deep
in the degraded regime - while the dense stack near the loss stayed in the
working one. In situ on SP500 H1: lstm1 dW/W 2.62/10.0/7.14% -> 0.024/0.022/
0.003%, conv1 decaying to 0.000% by era 30. NeuronBatchNorm.mqh already
squared v back for gamma/beta and its comment named the kernels as wrong,
which is exactly why gamma/beta kept training while the stages behind froze.
Persisted .nnw needs no migration - v keeps its std-dev meaning.
Also, the two ways F4 exposed it, both mine:
- No LR compensation for B fewer steps per era. sqrt(B) for adaptive methods
(Krizhevsky 2014; Granziol et al. 2022), applied once in
InitialEtaForOptimizer(). Linear scaling (Goyal et al. 2017) is for SGD.
- Plateau patience denominated in eras, so raising B made the ladder 32x more
impatient in its only unit. PAI converged at era 41 on ~49k updates where
the same config had been finding new bests at era 1028.
TrainPlateauPatienceEras() stretches it by the same sqrt(B).
TRAIN_BATCH_SIZE 32 -> 8 so the patience stretch stays affordable (8 -> 23
eras per stage, not 8 -> 45). Both helpers are identities at B=1.
Deploy gate: DEPLOY_MIN_SIDE_RECALL_PCT (10%) folded into tradeableOK. The
perceptron reported Sell:0% recall in all 41 eras, cleared the floor on Buy
alone at 36.6% vs 34% chance, deployed, and sprayed buy arrows. Folded into
the ranking key rather than checked at deploy time so a one-sided era cannot
become best-so-far in the first place.
Deinit: the arrow purge now runs BEFORE ExtPanel.Destroy(), an unbounded
CAppDialog teardown that sat ahead of it - the same ordering inversion the
rule there exists to prevent. CONV was force-terminated 4.8 s into OnDeinit
(vs ~1.1 s for the three that finished) having reached none of its cleanup,
so its arrows stayed on the chart. Steps are now timed in the log.
PurgeChart's verification rescan filtered on OBJ_ARROW, the same blind spot
as the bulk delete, so "persisted 10 ... cleared 0" passed silently. It now
walks every object type and reports the object counts when both are zero.
Both build variants compile 0 errors / 0 warnings; both DLLs rebuilt.
FORCES A RETRAIN (already forced by N1) and both DLLs must ship with the .ex5.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 14:02:35 -04:00
# define TRAIN_BATCH_SIZE 8
2026-08-20 09:49:33 -04:00
//--- Floor for Train()'s indicator-depth clamp. Above it a short-but-real history beats livelocking on a
//--- depth the terminal will never serve; below it a tiny BarsCalculated() is more likely an indicator
//--- mid-calculation than a hard cap, so the clamp stands down. Sized so a clamped era still holds an
//--- OOS window worth measuring.
fix(train): clamp the sweep to indicator-servable depth - the scan wall was CopyBuffer, not a cold indicator
Symptom: on a 3-chart run with contention ruled out (SP500 sitting at era 2552),
USDJPY and XAUUSD produced 0 usable windows out of 50,163 and 33,966 - forever,
re-sweeping on every discard, which is the panel oscillating 0->100%.
Bars() is the PRICE series depth. A CUSTOM indicator's is not: MT5 calculates it
in its own context bounded by "Max bars in chart" (TERMINAL_MAXBARS), and
CopyBuffer past that limit does not short-read, it FAILS - so CDoubleBuffer keeps
nothing and EVERY index answers EMPTY_VALUE. ADMovingAverage is the only custom
indicator whose feature block REJECTS on EMPTY_VALUE (ADZigZag, also CiCustom,
neutral-fills; RSI/MACD/Ichimoku/ATR are built-ins served at any depth), so the
sweep died on feature 25 of every bar while the 24 price features under it were
fine. That is exactly the "window had 24 of 832 values" the stall report named.
Perfectly depth-correlated, measured 2026-08-17:
SP500 16,234 bars -> era 2552 XAUUSD 33,982 -> 0 windows
XTIUSD 16,611 bars -> era 71 USDJPY 50,179 -> 0 windows
This RETIRES the 2026-08-17 cold-indicator reading of the same stall. f0cf659 was
right that the rejection must be transient and that the dead backoff had to arm -
the branch did change to 'cold-indicator backoff' - but waiting cannot fix a depth
the terminal will never grant. So Train() now clamps to TunableBarsCalculated()
(which existed and was only ever used for a tuner printout) and trains on the
history that IS available, naming TERMINAL_MAXBARS in the log so the cause is
readable next time. m_coldSweepTick still owns the genuinely transient case: that
reads back as -1, not a positive short count. Recomputed per era, so the clamp
lifts by itself if the setting is raised.
Also fixes a real off-by-one it was hiding: the MA block reads GetData(idx) AND
GetData(idx + 1) for its bar-over-bar change, but ResizeBuffers sized m_MA to
barIndex exactly - so the deepest bar of every sweep read one past the end and was
rejected as cold. Same shape as the +ichiKijun the Ichimoku/close pair already has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:27:14 -04:00
# define TRAIN_MIN_CLAMPED_BARS 2000
2026-08-20 09:49:33 -04:00
//--- SettledBars()'s wait. Long enough that a busy terminal makes visible progress between probes, short
//--- enough that a chart with nothing to wait for loses only seconds.
feat(depth): prime -> settle -> sweep, and name which handle is short
"Max bars in chart" is set to Unlimited, so the static-terminal-limit reading in
1dda479 was wrong. Two other candidate causes are falsified too: the price series
is fully downloaded and flat (USDJPY 50,162 -> 50,163 over 80 minutes, i.e. one new
H4 bar), and the handles have been stable since 13:07 with zero windows for the 17
minutes after, so it is not download-in-progress and not handle churn. The MA period
tops out at 200 (ADIndicatorTuner MA_PERIOD_PRESETS) against ~50k bars, so it is not
indicator cost either.
What IS verified stays verified: CopyBuffer past the calculated depth fails outright
rather than short-reading, so the buffer holds nothing and every index reads
EMPTY_VALUE; m_MA is the only CiCustom whose block REJECTS on that (m_ADZigZag
neutral-fills, RSI/MACD/Ichimoku/ATR are built-ins); the wall is therefore feature 25
of every bar, exactly as the "24 of 832" stall lines said. And it is depth-correlated:
16k-bar charts train, 34k/50k get zero windows forever.
So the WHY is still open, and this fix does not depend on it. Per the user's protocol:
the request itself is the primer, so prime at full depth, then poll TunableBarsCalculated()
every 3s and hold the sweep until it stops changing (3 steady probes), then use whatever
it settled at. Bounded at 10 min, and a give-up is logged as a give-up so an abandoned
depth is never mistaken for a settled one. This supersedes 1dda479's clamp on the two
training paths, which snapshotted a value that may still have been climbing; the clamp
remains for the paths that cannot wait (inference/online/rescan/export, see 7e63a8b).
The load-bearing part is what does NOT happen while waiting: no sweep. A 50k-bar feature
scan starves the indicator threads the request just woke, which is how the failure
sustained itself for 40 minutes at a time - discard era, re-sweep, discard, which is the
0->100% oscillation on the panel.
Also adds IndicatorDepthReport(): per-handle BarsCalculated() on the priming, cap and
stall lines. The logs proved WHICH FEATURE died but never WHICH HANDLE was short, so the
cause had to be inferred - and was guessed wrong twice. The next occurrence reads it off.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:44:27 -04:00
# define DEPTH_SETTLE_PROBE_MS 3000
# define DEPTH_SETTLE_STABLE_PROBES 3
2026-08-20 09:49:33 -04:00
//--- Hard stop. A depth that has not settled in 10 minutes is not going to, and training on the history
//--- that IS there beats waiting forever - the give-up is logged, so it is never confused with a settle.
feat(depth): prime -> settle -> sweep, and name which handle is short
"Max bars in chart" is set to Unlimited, so the static-terminal-limit reading in
1dda479 was wrong. Two other candidate causes are falsified too: the price series
is fully downloaded and flat (USDJPY 50,162 -> 50,163 over 80 minutes, i.e. one new
H4 bar), and the handles have been stable since 13:07 with zero windows for the 17
minutes after, so it is not download-in-progress and not handle churn. The MA period
tops out at 200 (ADIndicatorTuner MA_PERIOD_PRESETS) against ~50k bars, so it is not
indicator cost either.
What IS verified stays verified: CopyBuffer past the calculated depth fails outright
rather than short-reading, so the buffer holds nothing and every index reads
EMPTY_VALUE; m_MA is the only CiCustom whose block REJECTS on that (m_ADZigZag
neutral-fills, RSI/MACD/Ichimoku/ATR are built-ins); the wall is therefore feature 25
of every bar, exactly as the "24 of 832" stall lines said. And it is depth-correlated:
16k-bar charts train, 34k/50k get zero windows forever.
So the WHY is still open, and this fix does not depend on it. Per the user's protocol:
the request itself is the primer, so prime at full depth, then poll TunableBarsCalculated()
every 3s and hold the sweep until it stops changing (3 steady probes), then use whatever
it settled at. Bounded at 10 min, and a give-up is logged as a give-up so an abandoned
depth is never mistaken for a settled one. This supersedes 1dda479's clamp on the two
training paths, which snapshotted a value that may still have been climbing; the clamp
remains for the paths that cannot wait (inference/online/rescan/export, see 7e63a8b).
The load-bearing part is what does NOT happen while waiting: no sweep. A 50k-bar feature
scan starves the indicator threads the request just woke, which is how the failure
sustained itself for 40 minutes at a time - discard era, re-sweep, discard, which is the
0->100% oscillation on the panel.
Also adds IndicatorDepthReport(): per-handle BarsCalculated() on the priming, cap and
stall lines. The logs proved WHICH FEATURE died but never WHICH HANDLE was short, so the
cause had to be inferred - and was guessed wrong twice. The next occurrence reads it off.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:44:27 -04:00
# define DEPTH_SETTLE_TIMEOUT_MS 600000
2026-08-22 00:24:45 -04:00
//--- ERA-BARRIER LIVENESS. How long a member may sit on the same era before the barrier stops
//--- treating it as one the others must wait for. See NoteBarrierProgress().
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
# define ENSEMBLE_BARRIER_STUCK_MS 720000
2026-08-20 09:49:33 -04:00
//--- HARD CAP on how far a member may run ahead of the SLOWEST still-training member, counting one
2026-08-22 00:24:45 -04:00
//--- excluded from the barrier. Stopping and naming the laggard beats running and producing
//--- nothing.
fix(ensemble): the era barrier read healthy startup work as a dead member
Reported symptom: one member at era 17 while the rest sat at era 2, with
the combined vote never scoring. Two faults compound to produce exactly
that, and neither needs a broken model to trigger.
FIRST - BUSY WAS READ AS STUCK. BarrierEraHeartbeat() decides liveness
from one signal: has m_eraCount changed in the last 12 minutes. But
Train() returns early, before the era loop, for three ONE-TIME phases
that never touch m_eraCount - the label-cache prebuild, the pattern-DB
backfill and the OOS simulation walk - and those are precisely what a
slow topology spends its first many minutes doing. A member grinding
steadily through a prebuild therefore looked identical to a dead one and
was dropped from the barrier at startup, before it had trained a single
era. The constant's own comment states the flawed premise: "comfortably
past the slowest healthy ERA on the deepest chart" - true, and not the
question being asked. Those three branches now call NoteBarrierProgress()
and a chunk of phase work re-arms the watchdog exactly as an era does.
SECOND - EXCLUSION HAD NO BOUND. Once dropped, a member is skipped by
EnsembleMinTrainingEra(). Drop every OTHER member and that loop finds
nothing to take a minimum over, falls through to its `return m_eraCount`
fallback - the CALLER'S own era - and EnsembleEraBarrierHolds() evaluates
`era > era`, false, for everybody. The barrier silently becomes a no-op
and the fastest member runs away unbounded. EnsembleMinEraAnyMember()
now measures against every still-training member, excluded or not, and a
member may lead it by at most ENSEMBLE_MAX_ERA_LEAD eras.
The cap is deliberately a real stop rather than a warning. A
desynchronised ensemble is not a degraded one: the combined-vote score
and the joint checkpoint both require every member on the same era, so
weights trained past the cap can never be certified by any gate. The
hold reports which of the two it is, because the operator's next move
differs - an ordinary barrier hold resolves itself, a lead-cap hold names
a member that needs diagnosing and will not resolve on its own.
Not yet explained: "only one NN listened to the stop command". The panel
now dispatches down the filter tree and reports the count it reached
("training stopped (N model(s))"), so the next run answers that
definitively instead of leaving it to inference.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 09:35:25 -04:00
# define ENSEMBLE_MAX_ERA_LEAD 4
2026-08-20 09:49:33 -04:00
//--- How often a held member says so in the journal. The panel line is written every call; this is the
//--- durable record, without which a frozen chart leaves no trace at all.
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
# define ENSEMBLE_BARRIER_REPORT_MS 120000
2026-08-20 09:49:33 -04:00
//--- Cadence of the settled per-era diagnostics when VerboseMode is off (see TrainLogDue): each repeating
//--- print fires on eras 0-3 and then every Nth. 25 is ~one block per 15 minutes per member at ~35s/era -
//--- enough to reconstruct a run without the 22MB/9.5h firehose. CHANGE events are never throttled.
feat(logs): throttle the settled per-era diagnostics - measured 22MB/9.5h of confirmed-working systems
Measured from the journal (2026-08-19): the era deep-dive line (~2KB) plus
the excursion verdict, tier re-rank, calibration move, barrier hold and
selection-regressed note each printed EVERY era for EVERY member - ~940
eras/member/day - long after the systems they watch were confirmed
working. Yesterday's file was 1.3GB (70% of it the news-filter calendar
spam the sweep fix already removed).
VerboseMode returns as an INPUT (demoted 2026-08-01 for the marketplace;
that track is dead since the 2026-08-16 pivot) and gains a second job:
false throttles each settled per-era print to eras 0-3 plus every
TRAIN_LOG_EVERY_ERAS-th (25 ~= one deep-dive per ~15min per member);
true restores the per-era firehose, flippable live.
Never throttled: anything that marks a CHANGE - new bests, restores +
eta decays, plateau stage transitions, deploy approvals, warnings,
errors, the label-cache/adoption one-shots, and the combined-vote gate
line (the active system's primary telemetry, still every era).
Semantic fixes over blanket gating:
- barrier hold now ARMS silently and prints only when the hold outlasts
the 2-min report interval - a brief hold every era is the design, the
long hold is the watchdog case the line exists for;
- the ensemble deploy REFUSAL prints immediately when its reason
changes (that is a finding), on cadence when unchanged;
- the filtered-view census prints when its RESULT moves (drawn count,
or strongest vote by >=2pp) and at least every 10th sweep.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 09:36:15 -04:00
# define TRAIN_LOG_EVERY_ERAS 25
2026-08-20 09:49:33 -04:00
//--- Gap between attempts to rebuild a dead indicator handle. Long enough not to hammer a terminal that
//--- is genuinely refusing, short enough to recover within one stall-report interval.
fix(indicators+panel): the dead handle is MEASURED now - recreate it; and order the ensemble panel by member, not by who published first
THE ANSWER, off the instrumentation added in be39674, first run:
ConvLSTM [HYB-2484]: TUNABLE INDICATOR REPORTS NO CALCULATED BARS - 1 tunable
indicator(s) enabled and the least-ready answers BarsCalculated()=-1 ...
Per-indicator depth: price=33982 MA=-1 ZigZag=33982 ATR=33982
MA=-1 with price, ZigZag and ATR all at full depth. **The handle is INVALID, not
short.** Same line on USDJPY (price=50179 MA=-1). Depth was never the problem;
the previous session's five theories were all answering the wrong question.
And it is per-member, not per-chart: LSTM-2484 ran the 34-candidate auto-tune on
that same chart at 15:24:18 and went on to train normally (feature health, 51
features, excursion head) reading the same indicator. Only ConvLSTM's handle -
the last member constructed - was dead. WHY is still not established. All four
members request ADMovingAverage with identical params, so MT5 hands them the SAME
refcounted handle, and the tuner's inner loop is Create-then-IndicatorRelease over
exactly that shared handle; that is the obvious suspect and it is NOT yet proven,
so this commit does not act on it.
1. IndicatorDepthReport() NOW PRINTS HANDLE NUMBERS, not just depths.
"MA=-1" says the handle is dead. "MA=-1(h12)" against another member's "MA=33982
(h12)" says it is the SAME handle and someone released it; "(h-1)" says it was
never created. That is the difference between a refcount bug and a creation
failure and it is one field. This is the measurement the shared-handle suspicion
needs before anyone acts on it.
2. RECREATE A DEAD HANDLE INSTEAD OF SWEEPING AGAINST IT.
A member that cannot read its own indicator must rebuild it. RepairDeadIndicatorHandles()
re-Creates only the ENABLED tunables reporting BarsCalculated() < 0 - a merely COLD
indicator (valid handle, 0 bars) is left alone to warm up the normal way. It does
NOT release first: -1 means the terminal no longer knows the handle, so there is
nothing to give back, and MT5 recycles handle VALUES so releasing a stale one could
decrement whatever now owns that number. 30s cooldown, because every ServableBars()
consumer reaches it including live inference on every tick. The feature cache is
dropped with it, and the log names before/after depths.
Cause-agnostic on purpose. Whatever is killing the handle, sweeping 50,163 bars
against a buffer that answers EMPTY_VALUE at every index - then discarding the era
and doing it again - is not a recovery.
3. THE SWEEP NOW HOLDS ON A DEAD HANDLE.
ServableBars() keeps answering `want` (its contract; live inference and online
learning have their own refusal paths and a 0 there reads as "no history at all").
SettledBars() - the training sweep's entry, the one caller that can afford to wait -
returns 0 instead, so Train() holds and reports rather than burning a full-history
pass it is guaranteed to throw away. A recreated handle is cold, so it primes
through the existing settle path on the next call. If the repair fails the member
holds indefinitely and says so every minute, and be39674's barrier liveness escape
releases the rest of the ensemble after 12 minutes - which is the correct
degradation and is exactly what the log shows happening.
4. THE PANEL ROWS WERE ORDERED BY WHO PUBLISHED FIRST.
Reported on XAUUSD: LSTM, ConvLSTM, Perceptron, Convolutional instead of
Perceptron, Convolutional, LSTM, ConvLSTM. ClaimEnsemblePanelSlot() handed out the
next free row on each member's FIRST PublishStatus() call, so the order was a race -
the members busy sweeping published before the ones sitting idle at the era barrier,
and be39674 sharpened it by (correctly) making a held member stop writing the terse
line. Rows are now keyed to m_ensembleIndex, the registration/construction order,
which is fixed for the life of the chart. Claimed on every publish rather than once,
so it is idempotent and refreshes the tag for a member whose ID was not final when it
first published (the config-tag suffix is appended during InitIndicators, after
EnsembleMember() registers). Unclaimed rows are skipped by the render and excluded
from the model count, so a member that has not published yet leaves no gap and shifts
nobody.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:40:00 -04:00
# define HANDLE_REPAIR_COOLDOWN_MS 30000
2026-08-20 09:49:33 -04:00
//+------------------------------------------------------------------+
2026-08-22 00:30:14 -04:00
//| sqrt(B) learning-rate compensation and the matching patience |
//| stretch. Both are exactly 1.0 at B=1, so the whole mini-batch |
//| apparatus vanishes when TRAIN_BATCH_SIZE is 1. |
2026-08-20 09:49:33 -04:00
//+------------------------------------------------------------------+
fix: the Adam second moment was never Adam - all four tiers
Root cause of the B=32 regression, and it predates F4 entirely. Every Adam
kernel stored v already square-rooted and then fed that stored value back in
as if it were the variance:
v_new = sqrt(b2 * v_old + (1 - b2) * g^2)
That recursion has a fixed point at v ~= b2 = 0.999 for ANY gradient below
unit scale, so the denominator stops tracking the gradient and Adam degrades
into plain SGD with lr = lt. Measured against the shipped WarriorCPU.dll
(batch_accum_check.cpp, TestOptimizerScaleInvariance), 4000 steps of a
constant gradient: 3285x less displacement at |g|=1e-5 than at |g|=1, where
a scale-invariant optimizer gives the same distance for both. After the fix
all six magnitudes read 1.199 and v tracks |g| exactly.
It hit conv/LSTM specifically because they sit behind a batch-norm with
running variance ~2.6e+05, so their gradients arrive divided by ~500 - deep
in the degraded regime - while the dense stack near the loss stayed in the
working one. In situ on SP500 H1: lstm1 dW/W 2.62/10.0/7.14% -> 0.024/0.022/
0.003%, conv1 decaying to 0.000% by era 30. NeuronBatchNorm.mqh already
squared v back for gamma/beta and its comment named the kernels as wrong,
which is exactly why gamma/beta kept training while the stages behind froze.
Persisted .nnw needs no migration - v keeps its std-dev meaning.
Also, the two ways F4 exposed it, both mine:
- No LR compensation for B fewer steps per era. sqrt(B) for adaptive methods
(Krizhevsky 2014; Granziol et al. 2022), applied once in
InitialEtaForOptimizer(). Linear scaling (Goyal et al. 2017) is for SGD.
- Plateau patience denominated in eras, so raising B made the ladder 32x more
impatient in its only unit. PAI converged at era 41 on ~49k updates where
the same config had been finding new bests at era 1028.
TrainPlateauPatienceEras() stretches it by the same sqrt(B).
TRAIN_BATCH_SIZE 32 -> 8 so the patience stretch stays affordable (8 -> 23
eras per stage, not 8 -> 45). Both helpers are identities at B=1.
Deploy gate: DEPLOY_MIN_SIDE_RECALL_PCT (10%) folded into tradeableOK. The
perceptron reported Sell:0% recall in all 41 eras, cleared the floor on Buy
alone at 36.6% vs 34% chance, deployed, and sprayed buy arrows. Folded into
the ranking key rather than checked at deploy time so a one-sided era cannot
become best-so-far in the first place.
Deinit: the arrow purge now runs BEFORE ExtPanel.Destroy(), an unbounded
CAppDialog teardown that sat ahead of it - the same ordering inversion the
rule there exists to prevent. CONV was force-terminated 4.8 s into OnDeinit
(vs ~1.1 s for the three that finished) having reached none of its cleanup,
so its arrows stayed on the chart. Steps are now timed in the log.
PurgeChart's verification rescan filtered on OBJ_ARROW, the same blind spot
as the bulk delete, so "persisted 10 ... cleared 0" passed silently. It now
walks every object type and reports the object counts when both are zero.
Both build variants compile 0 errors / 0 warnings; both DLLs rebuilt.
FORCES A RETRAIN (already forced by N1) and both DLLs must ship with the .ex5.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 14:02:35 -04:00
double TrainBatchLrScale ( void ) { return MathSqrt ( ( double ) TRAIN_BATCH_SIZE ) ; }
int TrainPlateauPatienceEras ( void ) { return ( int ) MathRound ( PLATEAU_PATIENCE_ERAS * MathSqrt ( ( double ) TRAIN_BATCH_SIZE ) ) ; }
2026-08-22 00:24:45 -04:00
//--- ONLINE CONTINUAL LEARNING (see OnlineLearnStep()). Live-chart-only: a deployed model keeps
//--- adapting to newly-RESOLVED bars on the same supervised triple-barrier task, never on trade
//--- P&L.
2026-07-24 11:52:19 -04:00
# define ONLINE_LEARN_MAX_CATCHUP 64
# define ONLINE_ACC_SMOOTH 50.0
# define ONLINE_LEARN_WARMUP 20
# define ONLINE_LEARN_MIN_ACC 40.0
# define ONLINE_LEARN_ACC_MARGIN 10.0
# define ONLINE_LEARN_PERSIST_EVERY 32
2026-07-29 00:03:54 -04:00
# define ONLINE_LEARN_MAX_CLASS_WEIGHT 5.0
2026-08-20 09:49:33 -04:00
//--- Pinned to the shipped defaults of the removed OversampleParity / ConstrainReplay / FocalLossGamma
//--- inputs - see the class-imbalance note above.
refactor(ai): nine class-imbalance inputs down to two
The imbalance section offered nine controls for one job. Audited against the
code, five of them did not do what their names said at the shipped defaults:
AILogitPriorStrength DEAD - Inference.mqh's post-hoc prior early-returns
whenever the adjusted loss is on, which is default.
OversampleParity DEAD in training - Training.mqh gated the replay loop
on !useLogitAdjustedLoss (correctly, citing Buda et
al. 2018). Live only in the online-learning path.
EnableMinorityReplay DEAD as replay. It survived ONLY as a focal-gamma
damper - "replay minority bars through pass-2
oversampling" was a focal-loss switch.
ConstrainReplay DEAD as a cap; it only chose damper 0.125 vs 0.25.
UseStaticPrior An exact duplicate of FreezePriorCalibration - the two
were OR'd together in the single place either is read.
So they were not five mechanisms fighting; they were one mechanism plus eight
knobs that mostly described machinery that no longer ran. That is worse than
a real conflict, because the log agreed with the names: the label-cache line
printed "reps up to 28x (90% parity) (seeding era 0's class-balance
oversampling)" on every run, describing an oversampling pass that had been
switched off. It is fixed here too - it cost this session a wrong diagnosis.
The one genuine redundancy was focal loss, running at gamma*0.125 alongside
the adjusted loss: two corrections on the same axis, the exact stacking
failure this file already cited Buda et al. for in two other places, damped
by a replay flag whose replay path was itself dead. Removed rather than
re-tuned. The plateau ladder is unaffected - its escape is the learning-rate
warm restart; the gamma anneal beside it only ever stepped toward zero.
WHAT REMAINS is logit-adjusted loss (Menon et al. 2021) plus a prior freeze:
LogitAdjustTau 0 = off; replaces the separate EnableLogitAdjusted-
Loss boolean, since a strength dial where 0 already
means off does not need an on/off switch beside it.
FreezePriorCalibration unchanged.
It is the only one of the six corrections with a consistency guarantee, and
it is consistent for exactly the balanced-error metric checkpoint selection
already ranks on - so the loss and the deploy decision optimize one thing.
The online continual-learning path keeps its own alpha-balanced focal weight,
now as constants pinned to the removed inputs' shipped defaults, so its
behaviour is unchanged. It legitimately needs its own correction:
ApplyLogitAdjustment() only runs inside a training run, so a deployed model
that was reloaded carries no logit offsets and would otherwise stream 31:1
data into itself uncorrected.
The weights-filename fingerprint is BYTE-IDENTICAL. The focal slot was a
double fed to a %d conversion and had always emitted a literal 0; the |MR:
segment is written as the constant its shipped defaults produced. Dropping
either would have re-keyed every model and forced a from-scratch retrain of
the one topology currently converged and trading.
Also removed as orphans: FOCAL_GAMMA_PRESET, MAX_OVERSAMPLE_REPLICAS,
OVERSAMPLE_PARITY_FRACTION, PLATEAU_GAMMA_STEP, and the now-unreachable
"neutralized by prior correction" diagnostic.
Both builds compile 0 errors, 0 warnings. No retrain forced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 11:46:57 -04:00
# define ONLINE_LEARN_PARITY 0.9
# define ONLINE_LEARN_ALPHA_CAP 3.0
# define ONLINE_LEARN_FOCAL_GAMMA 1.0
2026-07-29 00:03:54 -04:00
# define ONLINE_LEARN_ETA_SCALE 0.25
2026-08-20 09:49:33 -04:00
//+------------------------------------------------------------------+
//| Base learning rate for the selected optimizer. |
//| A free function, not a method: the constructor's init list needs |
//| it for both m_modelEta and m_etaCeiling, which runs before |
//| member-init order could safely let one depend on another. |
//| The sqrt(B) compensation is applied HERE, at the one point that |
//| decides the base rate, so it reaches the ceiling, the plateau |
//| boost and the anneal from a single edit. |
//+------------------------------------------------------------------+
2026-07-18 14:56:41 -04:00
double InitialEtaForOptimizer ( void )
{
refactor(stdlib): adopt Math\Stat for the deploy gate's normal tail; retire the b1/b2/lr/momentum macros
The gate's NormalUpperTail was a hand-rolled Abramowitz & Stegun 26.2.17
approximation. Its own comment gave the reason - "drags a chain of headers
behind it" - and that turned out to be one file: Math\Stat\Normal.mqh
includes only Math.mqh, which includes nothing. Swapped for Cody's rational
approximation in the library (~18 significant digits vs |error| < 7.5e-8).
No past verdict changes: at the z the gate operates on, the difference is
orders of magnitude below DEPLOY_FAMILY_WISE_ALPHA.
Adopting it needed the four bare macros in AI\Network.mqh gone first.
"#define b1 AdamBeta1" collides with an identifier in Math.mqh, so the
include would have macro-expanded the library's own local and failed to
compile - the same landmine that made the original author rename the
approximation's coefficients to ntB1..ntB5 rather than use the reference's
b1..b5. lr, b2 and momentum are the same class of hazard: single-token
global macros in a 52k-line codebase. All four now resolve to the input
names they always aliased, which is a pure textual identity - verified zero
bare occurrences remain.
Also:
- SelectionSort over the buffered signals was O(n^2) with an O(n^2) count of
StructToTime calls, because the comparison rebuilt both datetimes from the
six int date fields every time. Now materialises the keys once and does an
insertion sort; ArraySort cannot permute a struct array. IsEarlier goes
with it, MakeDateTime becomes SignalTime.
- Seven FileOpen sites lacked FILE_SHARE_READ|FILE_SHARE_WRITE, including
AtomicWriteBegin, which stages every model save. All 43 sites now carry
them - an exclusive open fails outright when another process holds the
path, which here has meant a silently skipped save.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:31:36 -04:00
return ( ( TrainingOptimizer = = SGD ) ? SgdLearningRate : AdamLearningRate ) * TrainBatchLrScale ( ) ;
2026-07-18 14:56:41 -04:00
}
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric
Four of the six findings from research/training_pipeline_audit_2026-08-09.md
(F4 mini-batching and F6 feature re-encode deliberately deferred - see the
report's implementation-status section for why):
- F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%,
which is 15-bit - provably non-uniform on every full-history era over 32,768
queued samples. New 30-bit ShuffleRandomIndex().
- F2: plateau warm restarts were a no-op whenever eta already sat at its
ceiling (the normal state of a non-regressing plateau) - the ladder was just
a 24-era countdown. Restarts now overshoot to 5x the ceiling
(PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience
window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has
real range.
- F3: checkpoint restores put weights back but kept the rejected trajectory's
Adam moments, so the optimizer immediately pushed back toward the rolled-back
state (the restore->regress->restore oscillation). CNet::ResetOptimizerState()
zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta
untouched) on every mid-run restore, every boosted restart, and the
deploy-time restore that online learning continues from.
- F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk,
so the selection metric the checkpoint ranking and deploy gate read is a pure
function of the checkpoint instead of partly measuring BN drift. Defensive
unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and
the OOS continual-learning simulation stay adaptive by design.
Compiled clean (0 errors, 0 warnings) via the staged-tree recipe.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 10:54:09 -04:00
//+------------------------------------------------------------------+
2026-08-22 00:30:14 -04:00
//| Uniform random index in [0, n) for Fisher-Yates shuffles. |
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric
Four of the six findings from research/training_pipeline_audit_2026-08-09.md
(F4 mini-batching and F6 feature re-encode deliberately deferred - see the
report's implementation-status section for why):
- F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%,
which is 15-bit - provably non-uniform on every full-history era over 32,768
queued samples. New 30-bit ShuffleRandomIndex().
- F2: plateau warm restarts were a no-op whenever eta already sat at its
ceiling (the normal state of a non-regressing plateau) - the ladder was just
a 24-era countdown. Restarts now overshoot to 5x the ceiling
(PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience
window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has
real range.
- F3: checkpoint restores put weights back but kept the rejected trajectory's
Adam moments, so the optimizer immediately pushed back toward the rolled-back
state (the restore->regress->restore oscillation). CNet::ResetOptimizerState()
zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta
untouched) on every mid-run restore, every boosted restart, and the
deploy-time restore that online learning continues from.
- F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk,
so the selection metric the checkpoint ranking and deploy gate read is a pure
function of the checkpoint instead of partly measuring BN drift. Defensive
unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and
the OOS continual-learning simulation stay adaptive by design.
Compiled clean (0 errors, 0 warnings) via the staged-tree recipe.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 10:54:09 -04:00
//+------------------------------------------------------------------+
int ShuffleRandomIndex ( const int n )
{
feat(rng): ALGLIB's L'Ecuyer generator replaces MathRand, and a seed collision goes with it
MQL5's MathRand() is the 15-bit MSVC LCG - 32768 distinct values and
the lattice structure that shape of generator has. Two places here
actually lean on randomness and both were hurt by it:
WEIGHT INIT. Six He/LeCun-uniform sites drew
((MathRand()+1)/32768.0 - 0.5) * 2 * scale, so a first dense layer of
~250k weights had only 32768 possible values and thousands of
connections started byte-identical. Breaking that symmetry is the whole
job of random init.
SHUFFLING. ShuffleRandomIndex() already had to splice TWO MathRand()
draws to reach 30 bits, and its own comment documented the residual
modulo bias it still carried. HQRndUniformI() is rejection-sampled and
exactly uniform, so the splice and the bias note both go.
CHighQualityRand is L'Ecuyer's combined multiplicative congruential
generator - two differenced streams, 31-bit output, period ~2.3e18 -
and it ships with the terminal.
AND A BUG THE MIGRATION EXPOSED. The three MathSrand(GetTickCount())
calls sit immediately before "build a fresh topology", once per model.
GetTickCount() steps in ~15.6 ms on Windows and an ensemble builds every
member inside one OnInit, so members could be handed the SAME seed and
draw the SAME weights wherever their shapes coincide - and members that
start identical are not an ensemble. WarriorRandSeed() takes a salt (the
model id) plus a never-reset call counter, so a collision is impossible
rather than merely unlikely, while the tick keeps the run itself
genuinely unrepeatable the way those call sites asked for.
Seeds are masked positive rather than trusted: HQRndSeed computes
s % (M-1) + 1 and MQL5's % keeps the sign, so a negative seed leaves the
generator in a state its own assertions reject. GetTickCount() is a uint
and goes negative as an int after ~24 days of uptime - a fault that
would surface as "training is broken" on a long-running terminal and
nowhere else.
The indicator tuner's 52 draws move across too: its random search is
where sample quality earns its keep.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 00:29:12 -04:00
return WarriorRandInt ( n ) ;
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric
Four of the six findings from research/training_pipeline_audit_2026-08-09.md
(F4 mini-batching and F6 feature re-encode deliberately deferred - see the
report's implementation-status section for why):
- F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%,
which is 15-bit - provably non-uniform on every full-history era over 32,768
queued samples. New 30-bit ShuffleRandomIndex().
- F2: plateau warm restarts were a no-op whenever eta already sat at its
ceiling (the normal state of a non-regressing plateau) - the ladder was just
a 24-era countdown. Restarts now overshoot to 5x the ceiling
(PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience
window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has
real range.
- F3: checkpoint restores put weights back but kept the rejected trajectory's
Adam moments, so the optimizer immediately pushed back toward the rolled-back
state (the restore->regress->restore oscillation). CNet::ResetOptimizerState()
zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta
untouched) on every mid-run restore, every boosted restart, and the
deploy-time restore that online learning continues from.
- F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk,
so the selection metric the checkpoint ranking and deploy gate read is a pure
function of the checkpoint instead of partly measuring BN drift. Defensive
unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and
the OOS continual-learning simulation stay adaptive by design.
Compiled clean (0 errors, 0 warnings) via the staged-tree recipe.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 10:54:09 -04:00
}
refactor(training): the era log is not part of the era loop
Train() was 2,572 lines in one function. The largest single block in it
was ~200 lines of string building for the console line, reachable only
because twenty-one loose ints were declared at the top of the function
and read nine hundred lines later. Those declarations were the reason
the block could not move.
Introduce SEraTelemetry - one parameter object holding exactly those
twenty-one numbers, self-initialising to -1 ("not measured this era",
which is what era 0 and any stopped era report, and is not the same as
a measured zero). ReportEraProgress() takes it and renders it, guarding
on its own shouldLog so the call site is one unconditional line rather
than a 200-line branch.
Nothing is decided or measured in the moved code - it reads state and
prints. Verified by stripping comments and whitespace from both
revisions and diffing the remaining statements: the only differences
are the ten declarations collapsing into one object, the throttle test
moving inside the callee, and the new signature plus its call. Every
other statement is byte-identical.
Train() 2,572 -> 2,370 lines.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:25:07 -04:00
//+------------------------------------------------------------------+
refactor(training): Train()'s working state is one object, not eight locals
Train() is not a function that trains a model - it is one STEP of a
resumable state machine, called again every tick until the era ends.
Eight locals carried the state from one step to the next: bars,
totalIter, oosCutoff, i, add_loop, stop, chunkStartTick and the
forward-failure latch.
Those eight are the sole reason none of the four passes could be lifted
into a method. Each would have needed eight by-reference parameters,
and a pass that takes eight parameters is not a pass - it is the same
function under another name.
Collapse them into STrainEra. Nothing else changes: no logic, no
ordering, no early return. Verified the same way as the era log - strip
comments and whitespace from both revisions, map era.X back to X, and
diff the remaining statements. The only differences are the five
declaration lines becoming one object and two assignments losing their
type. Every other statement is byte-identical.
One incidental fix: a for(int i...) loop over g_warriorEnsemble shadowed
the era counter. It ran before the era locals were declared so it was
never a live bug, but it becomes one the moment a pass moves out. It is
now mi.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:28:11 -04:00
//| ONE Train() CALL'S WORKING STATE. |
//| |
//| Train() is not a function that trains a model - it is one STEP |
//| of a resumable state machine, called again every tick until the |
//| era finishes. Everything here is what one step hands to the |
//| next: where the era loop got to, whether the era completed, and |
//| whether the caller asked it to stop. |
//| |
//| It exists because these were eight separate locals declared at |
//| the top of a 2,300-line function and read throughout all four |
//| passes. That is the sole reason no pass could be lifted into a |
//| method of its own: each would have needed eight by-reference |
//| parameters, and a pass that takes eight is not a pass, it is the |
//| same function with a different name. |
//+------------------------------------------------------------------+
struct STrainEra
{
int bars ; // bars available to this run
int totalIter ; // training samples in the in-sample span
int oosCutoff ; // first index of the out-of-sample span
int i ; // the era loop's position, preserved across calls
bool addLoop ; // this era ran to completion (not cut short by the budget)
bool stop ; // the terminal or the panel asked training to end
uint chunkStartTick ; // when this call started, for the wall-clock budget
2026-08-23 11:40:15 -04:00
uint budgetMs ; // max wall-clock work per call before yielding
refactor(training): Train()'s working state is one object, not eight locals
Train() is not a function that trains a model - it is one STEP of a
resumable state machine, called again every tick until the era ends.
Eight locals carried the state from one step to the next: bars,
totalIter, oosCutoff, i, add_loop, stop, chunkStartTick and the
forward-failure latch.
Those eight are the sole reason none of the four passes could be lifted
into a method. Each would have needed eight by-reference parameters,
and a pass that takes eight parameters is not a pass - it is the same
function under another name.
Collapse them into STrainEra. Nothing else changes: no logic, no
ordering, no early return. Verified the same way as the era log - strip
comments and whitespace from both revisions, map era.X back to X, and
diff the remaining statements. The only differences are the five
declaration lines becoming one object and two assignments losing their
type. Every other statement is byte-identical.
One incidental fix: a for(int i...) loop over g_warriorEnsemble shadowed
the era counter. It ran before the era locals were declared so it was
never a live bug, but it becomes one the moment a pass moves out. It is
now mi.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:28:11 -04:00
//--- One-shot latch so a failing forward pass reports itself ONCE per call instead of once
//--- per sample.
bool forwardFailureReported ;
STrainEra ( void )
{
bars = totalIter = oosCutoff = i = 0 ;
addLoop = stop = forwardFailureReported = false ;
2026-08-23 11:40:15 -04:00
chunkStartTick = budgetMs = 0 ;
refactor(training): Train()'s working state is one object, not eight locals
Train() is not a function that trains a model - it is one STEP of a
resumable state machine, called again every tick until the era ends.
Eight locals carried the state from one step to the next: bars,
totalIter, oosCutoff, i, add_loop, stop, chunkStartTick and the
forward-failure latch.
Those eight are the sole reason none of the four passes could be lifted
into a method. Each would have needed eight by-reference parameters,
and a pass that takes eight parameters is not a pass - it is the same
function under another name.
Collapse them into STrainEra. Nothing else changes: no logic, no
ordering, no early return. Verified the same way as the era log - strip
comments and whitespace from both revisions, map era.X back to X, and
diff the remaining statements. The only differences are the five
declaration lines becoming one object and two assignments losing their
type. Every other statement is byte-identical.
One incidental fix: a for(int i...) loop over g_warriorEnsemble shadowed
the era counter. It ran before the era locals were declared so it was
never a live bug, but it becomes one the moment a pass moves out. It is
now mi.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:28:11 -04:00
}
2026-08-23 11:40:15 -04:00
//--- HAS THIS CALL USED ITS TIME? Every pass yields on the same question, so it is asked in one
//--- place. A pass that answers yes must leave its own resume state behind before returning.
bool BudgetSpent ( void ) const { return GetTickCount ( ) - chunkStartTick > = budgetMs ; }
refactor(training): Train()'s working state is one object, not eight locals
Train() is not a function that trains a model - it is one STEP of a
resumable state machine, called again every tick until the era ends.
Eight locals carried the state from one step to the next: bars,
totalIter, oosCutoff, i, add_loop, stop, chunkStartTick and the
forward-failure latch.
Those eight are the sole reason none of the four passes could be lifted
into a method. Each would have needed eight by-reference parameters,
and a pass that takes eight parameters is not a pass - it is the same
function under another name.
Collapse them into STrainEra. Nothing else changes: no logic, no
ordering, no early return. Verified the same way as the era log - strip
comments and whitespace from both revisions, map era.X back to X, and
diff the remaining statements. The only differences are the five
declaration lines becoming one object and two assignments losing their
type. Every other statement is byte-identical.
One incidental fix: a for(int i...) loop over g_warriorEnsemble shadowed
the era counter. It ran before the era locals were declared so it was
never a live bug, but it becomes one the moment a pass moves out. It is
now mi.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:28:11 -04:00
} ;
//+------------------------------------------------------------------+
refactor(training): the era log is not part of the era loop
Train() was 2,572 lines in one function. The largest single block in it
was ~200 lines of string building for the console line, reachable only
because twenty-one loose ints were declared at the top of the function
and read nine hundred lines later. Those declarations were the reason
the block could not move.
Introduce SEraTelemetry - one parameter object holding exactly those
twenty-one numbers, self-initialising to -1 ("not measured this era",
which is what era 0 and any stopped era report, and is not the same as
a measured zero). ReportEraProgress() takes it and renders it, guarding
on its own shouldLog so the call site is one unconditional line rather
than a 200-line branch.
Nothing is decided or measured in the moved code - it reads state and
prints. Verified by stripping comments and whitespace from both
revisions and diffing the remaining statements: the only differences
are the ten declarations collapsing into one object, the throttle test
moving inside the callee, and the new signature plus its call. Every
other statement is byte-identical.
Train() 2,572 -> 2,370 lines.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:25:07 -04:00
//| ONE ERA'S REPORTABLE NUMBERS, carried from where they are |
//| measured to where they are printed. |
//| |
//| These were twenty-one loose locals declared at the top of Train() |
//| and read ~900 lines later, which is the whole reason the era log |
//| could not be lifted out of the era loop. -1 means "not measured |
//| this era" and prints as n/a: era 0, a stopped era and a cap-hit |
//| era all score nothing, and a zero there would read as a real |
//| measurement of zero. |
//+------------------------------------------------------------------+
struct SEraTelemetry
{
int buyRecall , sellRecall , neutralRecall ;
int coverage , dirPrec , chancePrec ;
int buyPred , sellPred , buyPrec , sellPrec ;
int buyTrue , sellTrue , neutralTrue , neutralPred ;
int buyFired , sellFired , neutralFired ;
int buyFiredPrec , sellFiredPrec ;
bool shouldLog ; // throttle decision, made where the tick count is known
SEraTelemetry ( void ) { Reset ( ) ; }
void Reset ( void )
{
buyRecall = sellRecall = neutralRecall = -1 ;
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
coverage = dirPrec = chancePrec = -1 ;
refactor(training): the era log is not part of the era loop
Train() was 2,572 lines in one function. The largest single block in it
was ~200 lines of string building for the console line, reachable only
because twenty-one loose ints were declared at the top of the function
and read nine hundred lines later. Those declarations were the reason
the block could not move.
Introduce SEraTelemetry - one parameter object holding exactly those
twenty-one numbers, self-initialising to -1 ("not measured this era",
which is what era 0 and any stopped era report, and is not the same as
a measured zero). ReportEraProgress() takes it and renders it, guarding
on its own shouldLog so the call site is one unconditional line rather
than a 200-line branch.
Nothing is decided or measured in the moved code - it reads state and
prints. Verified by stripping comments and whitespace from both
revisions and diffing the remaining statements: the only differences
are the ten declarations collapsing into one object, the throttle test
moving inside the callee, and the new signature plus its call. Every
other statement is byte-identical.
Train() 2,572 -> 2,370 lines.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:25:07 -04:00
buyPred = sellPred = buyPrec = sellPrec = -1 ;
buyTrue = sellTrue = neutralTrue = neutralPred = -1 ;
buyFired = sellFired = neutralFired = -1 ;
buyFiredPrec = sellFiredPrec = -1 ;
shouldLog = false ;
}
} ;
refactor(baselines): the first real module - a class, not an #included partial
Baselines was 951 lines of CExpertSignalAIBase method bodies in a file
that only looked like a module. It is now CBaselineComparator: a class
the signal OWNS, which reads a CTrainingDataView and prints. It does
not name the signal anywhere in its code.
What the seam forced out into the open:
- Thirty-odd ArraySize() bounds tests, each carried by its caller, are
now one test per accessor next to the data. The two `hasValueN` and
one `arrowN` locals are gone with them.
- The -2.0 "never scored" sentinel on the arrow cache was tested at the
call site. It is now inside DataDirectionalCall, where it cannot be
read as a small confidence.
- DoubleToSignal needs m_outputNeuronsCount, so a raw double could not
be turned into a side by any reader. The view answers
DirectionalCall(bar, isBuy, magnitude) instead - the conversion
happens where the head width lives, and the module no longer needs
ENUM_SIGNAL at all.
- m_baselineDone was a latch on the signal for a decision only this
module makes. It is m_done, private, where it belongs.
Correction to my own earlier claim: I said Baselines had nine exclusive
members "polluting the signal class". It had none. m_x, m_f, m_ngrad,
m_AvgCE and the rest are FIELDS OF ALGLIB REPORT OBJECTS (state.m_x,
mrep.m_AvgCE) that my `\bm_\w+` scan matched after the dot. The module
needs no private state but its view pointer and that latch - which is
why it came out this cleanly.
The include sits below the g_ens* vote globals and the Alglib headers
it reads, because unlike the AIBase\*.mqh partials this is a real class
declaration compiled where it stands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 12:00:18 -04:00
//--- AFTER the g_ens* vote globals and the inputs it reads, and after ENUM_SIGNAL: this one is a
//--- real class declaration, not a body-only partial, so it is compiled where it stands.
refactor(oos): twenty-one counters with one lifetime become one object
SOosTally holds this era's OOS confusion counts and the rates they imply.
The signal keeps one member where it kept twenty-one, and the era-reset
block loses twenty of its twenty-one clearing lines.
THE SHAPE THIS ENDS is the one that produced 7452bd1: a group of tallies
read together but cleared one-per-line, so a second reset path could clear
a subset and leave stale numerators over restarted denominators. Reset()
is now the only way to clear them and it clears all of them.
The pair had already started to drift. m_oosBuyFired/m_oosBuyFiredHits sat
at line 1085 and their Sell twins at line 1140 - 55 lines and an unrelated
member apart, with the Buy comment still claiming to describe both.
DERIVED RATES MOVE WITH THE DATA. `(bars > 0) ? (int)MathRound(100.0 * x /
bars) : -1` was written out twelve times, and the "-1 means not measurable,
never 0" convention re-spelled at each - a convention the deploy gate
depends on, since every caller tests `< 0` to mean "this does not block".
One rounding rule and one sentinel now.
GROUPED BY LIFETIME, NOT BY NAME. m_oosSamples looks like it belongs here
and does not: it is RUN-level, reset only with the weights, and the status
panel prints it beside dOosError which is also a run-level EMA. That pairing
is correct and stays. But the confidence-calibration block divided per-era
numerators by it, naming the results `empiricalAccuracy` and
`avgClaimedConfidence` when neither is that - the run-level denominator
cancels in their ratio, so eraScale was right and the two named
intermediates were not. Now written as the ratio it actually is, with the
cancellation stated, so nobody logs or gates on a half that decays with era
count.
BEHAVIOUR UNCHANGED: every moved expression preserves its formula, its
denominator and its sentinel.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 14:15:22 -04:00
# include "Training\OosTally.mqh"
refactor(gate): the member gate and the ensemble gate were one rule written twice
SDeployVerdict::EvaluateRates() is now the deploy arithmetic - coverage
floor, chance + EDGE_MIN_SIGMAS x SE, tradeability, and the coverage-
discounted ranking score - and both gates call it.
The duplicate was self-documenting. The ensemble copy carried three comments
asking a reader to keep it in step with the member copy by hand: "same
intent as the member gate's coverage floor + bothSidesLive", "the two gates
have to apply the identical correction or the ensemble becomes the easier
one to clear", "same lexicographic ordering as isBetterEra". They had
already fallen out of step once - 2c443ba found the ensemble certifying a
vote the EA never casts, in the wrong currency and against the wrong
denominator.
THE TWO REAL DIFFERENCES ARE NOW ARGUMENTS, not branches:
chancePct - the ensemble filters its zero-skill reference by the
direction policy, because with shorts blocked "always short"
is not a book anyone could run.
twoSided - a member reads per-side RECALL against a floor; the vote
reads whether it actually fired both ways.
Everything else was identical and is now literally identical.
effN stays an argument so the label-overlap deflation lives where it is
measured - and so the remaining inconsistency stays visible rather than
buried: the two FAMILY-WISE selection gates still take their SE from RAW n.
Recorded in the header, deliberately not changed; tightening them is a
policy call, not a refactor.
The decision now reads no chart, holds no net, prints nothing and opens no
file, so it can be exercised against a made-up tally.
BEHAVIOUR UNCHANGED: every expression keeps its formula, its guard and its
-1 sentinel; the ensemble's chance-reference and two-sidedness rules are
passed through untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 14:19:44 -04:00
//--- AFTER OosTally (it judges one) and the policy #defines it reads.
# include "Training\DeployGate.mqh"
refactor(pool): the cross-instrument gate owns a directory, not a model
PooledGate was three CExpertSignalAIBase method bodies in an #included
partial. It is now CPooledGate, a class the signal owns.
It needed NO data view. Diagnosing that first is the point: the module
reads a directory of CSV files and knows nothing about a model. The
only things it needs from its owner - the symbol's own numbers and the
ratio they were measured at - are arguments. Handing it a
CTrainingDataView would have been machinery for a dependency that does
not exist.
The owner fills SPoolRecord (the on-disk shape, which already existed)
because only it knows its symbol, its actual TargetRR and its label
lifespan. `id` is passed per call rather than bound, so there is no
init-order question about when the identity became available - m_symbol
is set by CExpertSignal::Init and ID by SetIdentity, at different
times.
m_poolWriteWarned was a one-shot latch living on the signal for a
warning only this module emits. It is m_writeWarned, private.
targetRR is now threaded into ReadPooledEvidence rather than read from
the owner. That is not plumbing for its own sake: a peer measured at a
different ratio has a different structural break-even, and only the
caller knows which ratio it is asking about.
Caught before compiling: I declared ReadPooledEvidence from memory as
(..., double &pooledEffN, const double targetRR). The real signature
ends in `string &detail`. Read the definition, aligned both ends.
Call sites in Training.mqh are untouched - PublishPoolRecord and
PooledGatePasses remain on the signal as the thin fillers that know its
geometry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 13:32:48 -04:00
# include "Training\PooledGate.mqh"
feat(training): wire TrainingPool into pass 2 - peer rows contribute gradient only
Peer rows join m_isTrainQueue as NEGATIVE sentinels before the shuffle, so they interleave with
this chart's samples instead of training in a block at one end. A block would be a curriculum:
whatever the optimizer saw last would decide where it landed.
TrainPoolStep is a separate path on purpose. Everything in pass 2's local branch after the
forward pass reaches for something indexed by a LOCAL bar - m_labelCache, m_winLongCache, the
excursion target, the arrow cache, m_Time - and a peer row has none of those. Sharing the path
would mean inventing values for all of them, which is how another instrument's outcomes end up
inside m_cumIsCorrect and the operating point gets fitted to them. The IS-vs-OOS gap is read as
THE overfitting signal, so polluting the IS side would not crash anything; it would just quietly
stop meaning what it says.
The purge key reuses the label walk's own two bounds - the horizon and NextScheduledCloseAll -
rather than approximating with a bar offset. A second horizon model here would drift from the
real one, and this project already measured that the close-all, not the nominal horizon, is what
actually terminates labels. Cutoff is the OLDEST OOS BAR'S TIME, in wall clock, because bar
indices cannot be compared across instruments that each have their own calendar.
Contribution happens while the window is still in TempData and before the forward pass
overwrites it, and is gated to direction models: the meta head trains a different target on a
wider input, which the fingerprint gate alone would NOT catch, since a meta model's fingerprint
matches its own peers perfectly well.
Use_Training_Pool ships false and does nothing until a second chart runs a matching fingerprint.
Compile-verified against a BASELINE of the same tree without the wiring: both produce 12
errors, all error 313 invalid-resource-path from #resource directives that cannot resolve in a
headless staged build (stock Controls res\*.bmp, plus the pre-existing Network.cl). Code errors
0, warnings 0, identical to baseline. Staging copy and junctions removed; the live .ex5 was
never touched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 14:00:21 -04:00
//--- Next to PooledGate because it is the same idea one level down: that one pools the DECISION
//--- across instruments, this pools the DATA the decision is made from.
# include "Training\TrainingPool.mqh"
refactor(baselines): the first real module - a class, not an #included partial
Baselines was 951 lines of CExpertSignalAIBase method bodies in a file
that only looked like a module. It is now CBaselineComparator: a class
the signal OWNS, which reads a CTrainingDataView and prints. It does
not name the signal anywhere in its code.
What the seam forced out into the open:
- Thirty-odd ArraySize() bounds tests, each carried by its caller, are
now one test per accessor next to the data. The two `hasValueN` and
one `arrowN` locals are gone with them.
- The -2.0 "never scored" sentinel on the arrow cache was tested at the
call site. It is now inside DataDirectionalCall, where it cannot be
read as a small confidence.
- DoubleToSignal needs m_outputNeuronsCount, so a raw double could not
be turned into a side by any reader. The view answers
DirectionalCall(bar, isBuy, magnitude) instead - the conversion
happens where the head width lives, and the module no longer needs
ENUM_SIGNAL at all.
- m_baselineDone was a latch on the signal for a decision only this
module makes. It is m_done, private, where it belongs.
Correction to my own earlier claim: I said Baselines had nine exclusive
members "polluting the signal class". It had none. m_x, m_f, m_ngrad,
m_AvgCE and the rest are FIELDS OF ALGLIB REPORT OBJECTS (state.m_x,
mrep.m_AvgCE) that my `\bm_\w+` scan matched after the dot. The module
needs no private state but its view pointer and that latch - which is
why it came out this cleanly.
The include sits below the g_ens* vote globals and the Alglib headers
it reads, because unlike the AIBase\*.mqh partials this is a real class
declaration compiled where it stands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 12:00:18 -04:00
# include "Training\BaselineComparator.mqh"
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
//--- Same reason: needs MAX_PERSISTED_ARROWS/MAX_RESTORED_ARROWS/ARROW_RESTORE_BUDGET_MS, all
//--- #defined above, and only CChartView (already fully declared) otherwise.
# include "Chart\ChartUI.mqh"
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- Same reason: needs CPU_INFERENCE_MAX_DIFF (#defined above) and
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
//--- CROSSASSET_MAX_PAIRS/ALTDATA_MAX_PIN_CHARS (from System\CrossAsset.mqh/AltData.mqh, both
//--- already included near the top of this file), plus only CPersistenceView otherwise.
# include "Persistence\ModelPersistence.mqh"
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
//--- Same view+adapter shape again, online-learning side. AFTER every ONLINE_LEARN_*/LABEL_SMOOTH_*/
//--- SHADOW_WEIGHT_TAU #define (all above), which COnlineLearning reads directly (they are macros,
//--- not signal members, so no view call carries them).
# include "OnlineLearning\IOnlineLearningView.mqh"
# include "OnlineLearning\AIBaseOnlineLearningView.mqh"
# include "OnlineLearning\OnlineLearning.mqh"
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- Same view+adapter shape again, topology side - but STATELESS, like Persistence: every field
//--- BuildModelFingerprint/BuildFreshTopology and the shape-derivation helpers touch is shared
//--- elsewhere in the signal (grep-verified). AFTER OnlineLearning (ResetForFreshTopology is one of
//--- the two irreducible calls ITopologyView.mqh needs) and every LEGACY_*/TOPOLOGY_BUDGET_*/
//--- WINDOW_*/HISTORY_BARS_*/CONV_*/LSTM_HIDDEN_*/HIDDEN_TAPER_* #define (all above).
# include "Topology\ITopologyView.mqh"
# include "Topology\AIBaseTopologyView.mqh"
# include "Topology\Topology.mqh"
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- Same reasoning as Topology.mqh above (needs CADIndicatorTuner/CCrossAssetPanel, both already
//--- included, plus every LEGACY_*/DEPTH_SETTLE_*/HANDLE_REPAIR_*/CROSSASSET_FEATURES #define above).
# include "Features\IFeaturesView.mqh"
# include "Features\AIBaseFeaturesView.mqh"
# include "Features\FeatureBuilder.mqh"
refactor(labeling): CTripleBarrier - one copy of the fill/barrier arithmetic
Session B of the feature-selection/labeling refactor track. Extracts the two
pieces of triple-barrier arithmetic that were genuinely duplicated or
scattered, taking price/ATR/geometry as plain arguments - no chart, no
indicator handle - so it is testable with synthetic numbers.
CTripleBarrier::ComputeLevels() replaces the fill/barrier level arithmetic
that TripleBarrierLabel() and SimulateTradeOutcome() each spelled out by
hand; their own comments already called it "IDENTICAL... deliberately and by
copy." One caller resolves both sides at once (the both-won tie-break needs
both); the other selects the side its isLong argument names. Same for
ApplyMinStopWidening(), the broker-minimum-stop floor both walks applied.
Fuzzed 200k random (entry, spread, risk, reward, minStop, isLong) tuples
against both original hand-written forms: 0 mismatches.
CLabelOverlap replaces m_labelLifespanSum/m_labelLifespanCount - two members
reset from three separate call sites (constructor, label-cache rebuild), the
exact "N loose members cleared in more than one place" shape a candidate-
geometry incident (7452bd1) turned into a live bug. One object, one Reset(),
default-constructed like every other object member. MeanLabelLifespan() and
EffectiveSampleSize() on the signal become thin forwarders with an unchanged
signature - every one of their ~15 existing callers, direct and through the
CAIBaseTrainingData adapter, is unaffected.
SnapHorizonToLadder() forwards to CTripleBarrier::SnapToLadder(), the ladder
array's one remaining copy; EffectiveHorizonMax() (the close-all cache) stays
on the signal since that state has no clean argument form.
NOT extracted: TripleBarrierLabel()'s ~200-line walk itself. It resolves both
sides simultaneously, tracks the first-passage ladder, and feeds the label
every live order is sized from; a rewrite of it cannot be checked without a
compiler, so only the two pieces provably identical to their originals moved.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 18:57:17 -04:00
//--- BEFORE the class body: CLabelOverlap is used below as a member's TYPE, so it must already be a
//--- complete declaration by the time the class is parsed. Reads no member and no input either.
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
# include "Labeling\LabelOverlap.mqh"
//--- Same view+adapter shape again, per-config chart lock side - STATEFUL:
2026-08-24 04:39:17 -04:00
//--- m_configLockName is exclusive (grep-verified - nothing outside Lifecycle.mqh's old body ever
//--- touched it).
# include "ConfigLock\IConfigLockView.mqh"
# include "ConfigLock\AIBaseConfigLockView.mqh"
# include "ConfigLock\ConfigLock.mqh"
2026-07-14 22:36:27 -04:00
class CExpertSignalAIBase : public CExpertSignalCustom
{
protected :
string ID ;
2026-08-20 09:49:33 -04:00
//+------------------------------------------------------------------+
//| ID with the bracketed config tag stripped: "Hybrid 3L [HYB-9369]" |
//| -> "Hybrid 3L". The tag tells one CHART's model files from |
//| another's, which is a developer's problem, not an owner's. Logs |
//| and the verbose panels keep the full ID. Strips from the LAST |
//| " [" so a model name containing a bracket cannot truncate more |
//| than intended. |
//+------------------------------------------------------------------+
2026-07-30 13:39:08 -04:00
string DisplayName ( void ) const
{
int cut = StringFind ( ID , " [ " ) ;
int next = cut ;
while ( next > = 0 )
{
cut = next ;
next = StringFind ( ID , " [ " , cut + 1 ) ;
}
return ( cut > = 0 ? StringSubstr ( ID , 0 , cut ) : ID ) ;
}
2026-08-22 00:24:45 -04:00
//--- Per-MEMBER arrow namespace: "WarSig_PAI_", "WarSig_CONV_", ... Global purges still match on
//--- bare "WarSig_".
fix(ensemble): per-member arrow namespaces; ConvLSTM rename; dialog in purge list
The ensemble chart UI had a shared-namespace defect that answered the user
question "what do the arrows represent?" with "a bug": all four members drew
arrows under the same WarSig_<bartime> object names, so the chart showed
whichever member rendered LAST, one member Neutral deleted another member Buy
at the same bar, each member init sweep wiped the arrows the previous member
had just restored, and SaveChartSignals - which rebuilds the sidecar by
SCANNING the chart - persisted every other member arrows into its own history
(the exact cross-model laundering its own header warns about, now happening
BETWEEN ensemble members).
Arrows are now namespaced per member (WarSig_PAI_, WarSig_CONV_, WarSig_LSTM_,
WarSig_HYB_): draw, delete, restore, prune, member init sweep, destructor
purge and the sidecar scan are all member-scoped, and the tooltip names the
model. Global purges keep matching the bare WarSig_ prefix, which covers all
member namespaces plus old-format leftovers from earlier builds.
Labels: the ensemble panel header no longer says "HYBRID ensemble" (HYBRID is
one member; the header is the ensemble) and the CONVLSTM member displays as
ConvLSTM instead of Hybrid. Its SHORT id stays HYB deliberately - it names the
model folder and changing it would orphan every model trained under that path.
Deinit: the alt-data mapping dialog namespace (WarriorAltMap_) joins
WarriorChartPrefixes, so both the OnInit purge and the deinit final sweep now
cover it - it was in neither list, so a dialog starved of its own Destroy()
left its controls on the chart permanently.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 18:26:55 -04:00
string ArrowPrefix ( void ) const { return SIG_ARROW_PREFIX + m_id + " _ " ; }
2026-08-20 09:49:33 -04:00
//--- Every subclass of this one is a neural net. Tells the aggregate's raw-arrow layer that this
//--- filter draws its OWN arrows, from cached per-bar scans spanning the whole chart, and must not be
//--- drawn again from the once-per-bar live path.
feat(chart): filtered view - one arrow per trade the bot would actually take
Adds DrawUnfilteredSignals (default OFF) and, with it off, replaces the
per-model arrow layer with the decision the EA would really have made.
THE FILTERED ARROW IS DRAWN AT THE ORDER, NOT AT THE THRESHOLD. Clearing
Min_Vote_Open is not the same as trading: a setup can pass the vote and still
never reach the broker (invalid SL/TP, stops-level, ATR warm-up, unsynced
swing history), and every one of those lands in OpenParams' failure branch.
So DrawVoteArrow() fires only after the order parameters validate, and the
failure branch withdraws any arrow already standing on that bar. One arrow is
one entry the EA would have placed - carrying the vote, the threshold it
cleared, and the SL/TP the order would have had.
Classic signals now draw too, under their own name and weight, so a chart
running MA/RSI/MACD/Ichimoku alongside the nets reads the same way an
ensemble chart does. They can only be drawn from the aggregate's once-per-bar
pass, because unlike the AI members they have no cached per-bar scan.
Two subtleties that would each have produced a quietly wrong chart:
- The raw classic draw sits AFTER filter.Direction(), not beside the
journaling block. GetActivePattern*() are CONSUMING reads holding the
PREVIOUS evaluation - "one tick later", which at Expert_EveryTick=false is
one BAR later. Keyed off those and placed at StartIndex(), every classic
arrow would have been drawn one bar early, which on a chart is
indistinguishable from a model that genuinely leads. Peek*() accessors
(non-consuming) let pattern, weight and bar come from one evaluation.
- CExpertSignalAIBase::DrawObject() early-returns instead of gating its five
call sites, so the switch cannot be honoured in three passes and missed in
the fourth. Its delete counterparts stay ungated so flipping the input off
and rescanning clears the raw layer rather than stranding it.
SIG_ARROW_PREFIX and g_signalsVisible move from ExpertSignalAIBase.mqh down
to ExpertSignalCustom.mqh - the nearest common ancestor - because the classic
signals cannot see the AI header (it is included later in Warrior_EA.mq5).
The vote layer gets SIG_VOTE_PREFIX under the same bare prefix, so
WarriorChartPrefixes()' purge still reaches every arrow without knowing they
exist.
NOT YET BUILT: the reconstructed history behind attach. Filtered arrows
currently start where the EA starts. See the next commit.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:41:28 -04:00
virtual bool IsAIFilter ( void ) const override { return true ; }
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- m_Open/m_Close/m_High/m_Low/m_Time stay HERE (genuinely shared with Labels.mqh/AutoTune.mqh/
//--- Training.mqh, which read them directly) - CFeatureBuilder reaches them through FeatureOpenAt()/
//--- ChartBarClose()/FeatureHighAt()/FeatureLowAt()/ChartBarTime(). m_Volumes/m_MA/m_RSI/
//--- m_MACDFeature/m_Ichimoku/the AD* indicators below moved onto CFeatureBuilder (m_featureBuilder)
//--- as real members - grep-verified exclusive to Features.mqh, touched nowhere else in Expert\.
2026-07-14 22:36:27 -04:00
CiOpen m_Open ;
CiClose m_Close ;
CiHigh m_High ;
CiLow m_Low ;
CiTime m_Time ;
2026-08-20 09:49:33 -04:00
//--- "Is this AD indicator still calculating?" - cold must be a TRANSIENT rejection, never a
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- zero-fill; see the definition in Expert\Features\FeatureBuilder.mqh.
bool ADIndicatorCold ( CiCustom & ind , string block ) { return m_featureBuilder . ADIndicatorCold ( ind , block ) ; }
2026-08-20 09:49:33 -04:00
//--- Stamp of the last pass-1 sweep in which EVERY window failed on a transient cause. Non-zero arms
//--- a short era-start backoff so the retry loop stops starving the indicator threads it waits on.
2026-08-13 10:23:11 -04:00
uint m_coldSweepTick ;
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- m_indicatorDepthCapBars/m_indicatorDepthDeadWarned/m_handleRepairTick/m_depthSettleStart/
//--- m_depthProbeTick/m_depthProbeLast/m_depthProbeStable moved onto CFeatureBuilder as real
//--- members - grep-verified exclusive to Features.mqh (only Lifecycle.mqh's ctor-init-list
//--- touched them elsewhere, which is teardown bookkeeping, not real use).
2026-08-22 00:24:45 -04:00
//--- ERA-BARRIER LIVENESS STATE (see ENSEMBLE_BARRIER_STUCK_MS and BarrierEraHeartbeat()).
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
long m_barrierEraSeen ;
uint m_barrierEraTick ;
2026-08-20 09:49:33 -04:00
//--- Set by NoteBarrierProgress() when a long one-time phase advances a chunk, so the watchdog can
//--- tell "busy" from "stuck" - the distinction it could not make before.
fix(ensemble): the era barrier read healthy startup work as a dead member
Reported symptom: one member at era 17 while the rest sat at era 2, with
the combined vote never scoring. Two faults compound to produce exactly
that, and neither needs a broken model to trigger.
FIRST - BUSY WAS READ AS STUCK. BarrierEraHeartbeat() decides liveness
from one signal: has m_eraCount changed in the last 12 minutes. But
Train() returns early, before the era loop, for three ONE-TIME phases
that never touch m_eraCount - the label-cache prebuild, the pattern-DB
backfill and the OOS simulation walk - and those are precisely what a
slow topology spends its first many minutes doing. A member grinding
steadily through a prebuild therefore looked identical to a dead one and
was dropped from the barrier at startup, before it had trained a single
era. The constant's own comment states the flawed premise: "comfortably
past the slowest healthy ERA on the deepest chart" - true, and not the
question being asked. Those three branches now call NoteBarrierProgress()
and a chunk of phase work re-arms the watchdog exactly as an era does.
SECOND - EXCLUSION HAD NO BOUND. Once dropped, a member is skipped by
EnsembleMinTrainingEra(). Drop every OTHER member and that loop finds
nothing to take a minimum over, falls through to its `return m_eraCount`
fallback - the CALLER'S own era - and EnsembleEraBarrierHolds() evaluates
`era > era`, false, for everybody. The barrier silently becomes a no-op
and the fastest member runs away unbounded. EnsembleMinEraAnyMember()
now measures against every still-training member, excluded or not, and a
member may lead it by at most ENSEMBLE_MAX_ERA_LEAD eras.
The cap is deliberately a real stop rather than a warning. A
desynchronised ensemble is not a degraded one: the combined-vote score
and the joint checkpoint both require every member on the same era, so
weights trained past the cap can never be certified by any gate. The
hold reports which of the two it is, because the operator's next move
differs - an ordinary barrier hold resolves itself, a lead-cap hold names
a member that needs diagnosing and will not resolve on its own.
Not yet explained: "only one NN listened to the stop command". The panel
now dispatches down the filter tree and reports the count it reached
("training stopped (N model(s))"), so the next run answers that
definitively instead of leaving it to inference.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 09:35:25 -04:00
bool m_barrierPhaseProgress ;
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
bool m_barrierExcluded ;
uint m_barrierHoldReportTick ;
2026-08-20 09:49:33 -04:00
//--- One-shot latch for the live-inference hold in RefreshConvergedSignal(). Cleared when the depth
//--- returns, so a second outage is reported rather than swallowed.
fix(depth): route EVERY ResizeBuffers call site through one indicator-depth gate
1dda479 clamped the training sweep. It left five other paths asking the indicators
for a depth they cannot serve, and on a live account the quiet ones are worse than
the stall was - a stalled chart is visible, a chart trading on a degraded feature
window is not.
ServableBars(want, context) is now the single gate, and all six go through it:
training sweep clamp, floored at TRAIN_MIN_CLAMPED_BARS (below that a small
positive BarsCalculated is warm-up, which m_coldSweepTick owns)
label prebuild clamp - labels come from price/ADZigZag and would survive a
capped MA, but ResizeBuffers sizes EVERY buffer and a failed
CopyBuffer leaves m_MA EMPTY for the next reader, so this path
could silently re-break the block Train()'s clamp just fixed
live inference HOLD. Below `need` the swing block takes its degraded path and
inference runs on a different feature distribution than the model
was fitted on. This EA sizes real positions off that output, so
no signal beats a mismatched one
online learning HOLD, same reason and worse - this path WRITES to a live trading
model, so a mismatched (features, label) pair is not a wrong arrow,
it is a wrong weight update that compounds every bar
chart rescan clamp - SIGNAL_RESCAN_LOOKBACK_BARS is 5000 and MT5's smallest
"Max bars in chart" is also 5000, so this one is genuinely
reachable; uncapped it repaints the window all-Neutral
research export clamp before the emptiness test, so a capped symbol exports the
depth it has rather than writing a CSV with a dead feature block -
an artefact that looks complete and is silently wrong
Both HOLDs are insurance, not expected states: `need` tops out near 1,152 bars
(16 + 750 + 384 + 2) against a 5,000 floor on the terminal setting. They exist so
the failure mode is unreachable rather than merely unlikely.
Not changed: a genuinely SHORT price history still takes the old degraded path at
every site. That is pre-existing behaviour and narrowing it would mute charts that
trade today, so it stays a separate decision rather than a side effect of this fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:33:16 -04:00
bool m_inferenceDepthRefusalWarned ;
2026-08-20 09:49:33 -04:00
//--- One-shot latch for the label-prebuild block message: a prebuild that cannot prepare its buffers
//--- retries on every scheduled call forever.
fix(buffers): revert the MA +1 - it asked for a bar that does not exist and stopped every chart
REGRESSION I INTRODUCED IN 1dda479, live for ~15 minutes.
CSeries::BufferResize -> CheckLoadHistory -> CheckTerminalHistory succeeds only when
Bars() >= size. Train() calls ResizeBuffers with barIndex == Bars(), so sizing the MA
buffer to barIndex + 1 asks for one bar more than the symbol has and fails the WHOLE
ResizeBuffers call. The log named it exactly:
failed to get 50180 bars for USDJPY,PERIOD_H4 (Bars() = 50,179)
failed to get 33983 bars for XAUUSD,PERIOD_H4 (Bars() = 33,982)
StartLabelCachePrebuild() then bailed on the false return and stayed silent, so the
only symptom was Train() reporting "arming the first label-cache prebuild" forever
with labelCacheBars=0 - the panel's "getting ready".
The premise was wrong, not just the arithmetic. The MA block reads GetData(idx) AND
GetData(idx + 1), and at the OLDEST bar that second read is SUPPOSED to fail - there
is no older bar to difference against. Rejecting that one bar is correct behaviour;
buying it cost the entire history.
Two more things, since the same defect had a second instance and no alarm:
- The Ichimoku pair (closeBars and m_Ichimoku, both barIndex + ichiKijun) is the same
bug with a far larger constant, latent only because the feature is off. Both are now
clamped to Bars(). The oldest ichiKijun bars then have no cloud, which that block's
EMPTY_VALUE guard already handles per-bar - the right outcome.
- The prebuild's bare `return` on a false ResizeBuffers now says so once, naming the
depth and Bars(). MQL5's own "failed to get N bars" was in the log the entire time,
from a stack frame nothing connected to the prebuild.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 14:12:47 -04:00
bool m_prebuildBlockWarned ;
2026-08-20 09:49:33 -04:00
//--- Ground truth for the training labels - MetaTrader's own Examples\ZigZag. Always created, never
//--- gated behind an Enable* input because it is not an optional feature, it IS the label; and never
//--- touched by AutoTuneIndicators, because tuning the ground truth alongside the model scored
//--- against it would let a trial "improve" by cherry-picking an easier target.
2026-08-24 18:26:25 -04:00
CiCustom m_zigZag ;
2026-08-22 00:24:45 -04:00
//--- Live tunable values for each AD indicator plus their flatten/perturb/best-tracking logic.
refactor(ExpertSignalAIBase): extract AutoTune param state into CADIndicatorTuner
CExpertSignalAIBase (4,326 lines, one class) carried 5 struct
definitions, 10 member fields, and 3 methods (Flatten/Unflatten/
PerturbRandom) purely for the AutoTuneIndicators search-space state -
entirely self-contained (never touches Net, Train()'s resumable state
machine, or anything else in the class). Extracted into a new
Expert/ADIndicatorTuner.mqh (CADIndicatorTuner), held as a single
m_indicatorTuner member.
TuneIndicatorsAndTrain() itself - the outer loop that actually
orchestrates Train()/Net/checkpointing around this tuner - turned out
to be exactly as tightly coupled to Train()'s resumable state machine
as Train() itself, so per the same caution already applied to Train()
in this refactor pass, it stays in CExpertSignalAIBase rather than
being pulled into the collaborator; it now calls the tuner's public
Flatten()/Unflatten()/PerturbRandom()/SaveAsBest()/RestoreBest()
instead of manipulating the structs inline.
All internal field-access renames (m_adCumDeltaParams.lookback ->
m_indicatorTuner.adCumDelta.lookback, etc., ~40 sites across the 5
InitAD*() indicator-setup methods) verified against a full grep sweep
- no leftover references to the old field/method names. Compiled
clean (MetaEditor, 0 errors/0 warnings).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-18 16:22:09 -04:00
CADIndicatorTuner m_indicatorTuner ;
2026-07-14 22:36:27 -04:00
bool m_autoTuneIndicators ;
2026-08-20 09:49:33 -04:00
//--- Rebuilds only the AD* handles in place, so ReInit picks up updated param structs.
ditch(features): remove the eight dead feature groups from the input matrix
RSI, MACD, Ichimoku and the five AD/Wyckoff indicators (CumulativeDelta,
ShorteningOfThrust, WyckoffEventStream, WyckoffFailedStructure,
WyckoffSignificantBarInversion). All eight inputs shipped false and each carries a
closed verdict: the three oscillators are the same patterns that measured at chance
as entries, and the Wyckoff family returned zero out-of-sample on five independent
instruments - which is what closed the context score.
RETRAIN-NEUTRAL, and this one is worth stating precisely because the change looks
larger than it is. Every removed group contributed `flag ? N : 0` to the input
width, and every flag was false, so the width was ALREADY zero for all eight: no
.nnw's input layer changes. On the fingerprints, UseRSI and the five AD flags were
hashed unconditionally and become literal 0 legacy slots (the convention the
m_focalGamma slot above them already uses); UseMACD/UseIchimoku were appended only
when enabled, so their segments simply never appear - byte-identical to every
fingerprint ever produced, since neither ever shipped on.
CADIndicatorTuner IS DELIBERATELY NOT SHRUNK. Its flat parameter array is persisted
inside every .nnw, and Unflatten() rejects a size mismatch by falling back to
constructor defaults - so dropping the dead fields would silently revert the tuned
MA period of every model on disk while keeping its trained weights. That is the
feature/weight mismatch this project has already paid for twice, and it is not
worth 200 lines. AD_TUNE_PARAM_COUNT stays 42, the dead slots are still written and
read, and AutoTune's ParamOwner gate now matches only owner 5 (MA) so nothing
searches them. The class comment says all of this at the declaration.
Also renamed ReInitADIndicators -> ReInitTunableIndicators: it rebuilds exactly one
indicator now, and a name saying "AD" for the MA handle is the kind of stale label
that gets believed later. Its release-AFTER-recreate ordering is untouched - that
is a documented fix, not bookkeeping.
Compile-verified in the stage copy: 0 errors, 0 warnings, against the same 0/0
baseline.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 09:21:03 -04:00
bool ReInitTunableIndicators ( CIndicators * indicators ) { return m_featureBuilder . ReInitTunableIndicators ( indicators ) ; }
2026-08-20 09:49:33 -04:00
//--- Installs a param set into the tuner and rebuilds handles ONLY when the set actually differs from
//--- what the indicators already run - see the definition for the resume-time churn this avoids.
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
bool AdoptIndicatorParams ( const double & loaded [ ] , CIndicators * indicators )
{ return m_featureBuilder . AdoptIndicatorParams ( loaded , indicators ) ; }
2026-08-20 09:49:33 -04:00
//--- Builds a fresh untrained topology into Net. Split from InitNeuralNetwork() so the tuner can
//--- rebuild weights per trial without re-running indicator init, which would Add() them twice.
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- Body in Expert\Topology\Topology.mqh.
bool BuildFreshTopology ( ) { return m_topology . BuildFreshTopology ( ) ; }
ditch(features): remove the eight dead feature groups from the input matrix
RSI, MACD, Ichimoku and the five AD/Wyckoff indicators (CumulativeDelta,
ShorteningOfThrust, WyckoffEventStream, WyckoffFailedStructure,
WyckoffSignificantBarInversion). All eight inputs shipped false and each carries a
closed verdict: the three oscillators are the same patterns that measured at chance
as entries, and the Wyckoff family returned zero out-of-sample on five independent
instruments - which is what closed the context score.
RETRAIN-NEUTRAL, and this one is worth stating precisely because the change looks
larger than it is. Every removed group contributed `flag ? N : 0` to the input
width, and every flag was false, so the width was ALREADY zero for all eight: no
.nnw's input layer changes. On the fingerprints, UseRSI and the five AD flags were
hashed unconditionally and become literal 0 legacy slots (the convention the
m_focalGamma slot above them already uses); UseMACD/UseIchimoku were appended only
when enabled, so their segments simply never appear - byte-identical to every
fingerprint ever produced, since neither ever shipped on.
CADIndicatorTuner IS DELIBERATELY NOT SHRUNK. Its flat parameter array is persisted
inside every .nnw, and Unflatten() rejects a size mismatch by falling back to
constructor defaults - so dropping the dead fields would silently revert the tuned
MA period of every model on disk while keeping its trained weights. That is the
feature/weight mismatch this project has already paid for twice, and it is not
worth 200 lines. AD_TUNE_PARAM_COUNT stays 42, the dead slots are still written and
read, and AutoTune's ParamOwner gate now matches only owner 5 (MA) so nothing
searches them. The class comment says all of this at the declaration.
Also renamed ReInitADIndicators -> ReInitTunableIndicators: it rebuilds exactly one
indicator now, and a name saying "AD" for the MA handle is the kind of stale label
that gets believed later. Its release-AFTER-recreate ordering is untouched - that
is a documented fix, not bookkeeping.
Compile-verified in the stage copy: 0 errors, 0 warnings, against the same 0/0
baseline.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 09:21:03 -04:00
//--- Retained so TuneIndicatorsAndTrain() can call ReInitTunableIndicators() between trials.
2026-07-14 22:36:27 -04:00
CIndicators * m_indicatorsPtr ;
CNet * Net ;
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
//--- The shadow net, the OOS continual-learning simulation state, the pattern-database backfill
//--- state, and the online-learning watermark/guardrail/counters (see OnlineLearnStep()) now
//--- default-construct on COnlineLearning (m_onlineLearning, declared below) - see its own
//--- constructor and class comment.
2026-07-14 22:36:27 -04:00
CArrayDouble * TempData ;
double dError ;
double dUndefine ;
double dForecast ;
double dPrevSignal ;
2026-08-01 11:27:28 -04:00
//--- ALTERNATION GATE REMOVED 2026-08-01 with the triple-barrier relabel. m_lastNonNeutralSignal
2026-08-22 00:24:45 -04:00
//--- suppressed any live Buy following another Buy with no Sell between. Do not reinstate it. It
//--- also meant a one-sided (`Sell:0%`) model got ONE trade per backtest, because the awaited
//--- opposite signal that reopens the gate never came.
diag: inference-path census, to explain zero-trade backtests
A backtest of the CONVERGED CONV model produced "Final directional result:
0.00000000" on every one of 1744 bars and therefore zero trades. Nothing in
the log could separate the three candidate causes, and each needs a
different fix:
1. RefreshLatestSignal never called (new-bar gate never fires)
2. called, but bailing at one of its two early returns
3. running fine, and the model genuinely answers Neutral every bar
Counts all three plus the Buy/Sell/Neutral split, printed once at shutdown
via StopTraining (which the tester reaches through OnDeinit). Three
increments per bar against a full feedForward - not worth gating.
Ruled out while writing this, so the next session does not re-derive it:
- the alternation gate (m_lastNonNeutralSignal) is NOT the cause. It starts
at Neutral, so a first Buy would still fire and show up as one non-zero
direction. We saw zero. It IS still a live hazard for a one-sided model -
CONV currently calls Buy:17% Sell:0%, and after the first Buy every later
Buy is suppressed until a Sell that never comes - but it cannot explain
an all-zero run.
- shallow buffers do not hard-fail the feature builder: the swing-context
Donchian loop breaks gracefully when it runs off loaded history. It does
mean converged-path inference computes Donchian/return/SMA features over
a TRUNCATED window versus training, which is a real train/inference skew
worth its own fix, but it degrades features rather than zeroing them.
Both builds 0/0. Diagnostic only.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 18:24:32 -04:00
long m_refreshOk ;
long m_refreshFailFeatures ;
long m_refreshFailShort ;
long m_refreshBuy ;
long m_refreshSell ;
long m_refreshNeutral ;
2026-08-22 00:24:45 -04:00
//--- VOTE-GATE census. Without these two the census reads as "the model answers Neutral" -
//--- false, and it points at a completely different fix. They separate the model's ANSWER from
//--- whether that answer was allowed to become a vote.
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets
The 31:1 class imbalance was self-inflicted by the TARGET, not a property
of the market. Labelling only the exact bar where a ZigZag pivot confirms
gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism
this codebase accumulated sits downstream of that one choice: the
logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias
seed, balanced-accuracy-then-precision selection with its coverage floor,
the recall floor and its catch-22, the alternation gate, NMS, and the four
oversampling designs that collapsed before them.
The reference this engine is built on (references/neuronetworksbook.pdf
ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT
EXTREMUM on every bar - ~50/50 by construction, with no imbalance to
correct at all. It never had this problem because it never asked "is this
the pivot bar".
Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's
OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its
target before its stop, within a horizon. Buy = long resolves, Sell =
short resolves, Neutral = neither. Consequences:
- dir-precision in the era line stops being a proxy and becomes the win
rate of the strategy under its own exit rules.
- Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e.
~2:1 instead of 31:1. Measured and logged at the end of the prebuild.
- Spread is charged on both legs, so it is a NET win rate.
- Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches
inside one bar and the optimistic reading is how a backtested edge
becomes a live loss.
ZigZag stays as input features (EnableSwingContext) and now also supplies
the vertical barrier: the horizon is the median confirmed leg length,
snapped to a coarse ladder. Derived, not configured, and deliberately kept
out of the filename fingerprint - a filename keyed on a measured quantity
orphans a trained model the moment the measurement moves.
Removed, because the premise died with the old target:
- the alternation gate. Correct for pivot labels (a ZigZag cannot emit two
same-type pivots in a row, so a repeat was provably a false fire), and
wrong for barrier labels, which answer each bar independently. It also
took its worst consequence with it: a one-sided model previously got ONE
trade per backtest, a hard blocker on marketplace validation.
- SignalClusterWindow now defaults off - it de-duplicated repeats that are
now real trades. Kept as an opt-in display control.
- LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel.
- the era-0 output-bias seed now needs a genuinely dominant class (0.70)
rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a
correction.
Also fixed, both found while wiring the above:
1. RefreshConvergedSignal sized its buffers from a date delta
(Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training
watermark; in the tester it is loaded from a live-chart save AHEAD of
the simulated date, so the interval inverted, Bars() returned ~0, and
the buffer came out at exactly m_historyBars - deep enough for the OHLC
window and far too shallow for the Donchian-50 / 20-bar-return / SMA
extension behind it. Inference silently computed DIFFERENT features
from the ones training learned on, live as well as in the tester. Now
sized from what the feature builder actually needs.
2. The barrier horizon is resolved on the deployed path too. A deployed
model never enters Train(), so it never reached the prebuild, and
OnlineLearnStep reads the horizon as its confirmation delay - left at
the fallback it would have backpropped bars whose barriers had not
resolved. Silent lookahead in the one place that writes to a live model.
SL_Mode/TP_Mode join the weights fingerprint: they define the labels now,
so a model trained at 1:3 must never be silently reused at 1:1. This
re-keys every pre-existing model by design - none were trained on this task.
Inference census extended with the vote gate. LongCondition/ShortCondition
open with a readiness check the refresh counters never see; in the tester it
reduces to "the seeded _optcache.nnw must have LOADED", and if it did not,
every vote is hard-zeroed while the model still answers Buy. The old three
counters would have read that as "the model says Neutral" - false, and a
completely different fix. This is the leading candidate for the
zero-direction backtest and the census can now name it in one run.
Both builds compile 0 errors / 0 warnings. Forces a full retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 20:39:49 -04:00
long m_voteGateBlocked ; // directional decisions the readiness gate discarded
long m_voteGatePassed ; // directional decisions that became a real vote
//--- Flag pair as of the first vote attempt, latched so the tally can name WHICH half of the gate
//--- failed rather than just reporting that it did. -1 = no vote was ever attempted.
int m_voteGateCompleteAtFirst ;
int m_voteGateLoadedAtFirst ;
2026-08-22 00:24:45 -04:00
//--- Non-max suppression window for directional signals: keeps the FIRST bar of a same-direction
//--- run and drops same-direction neighbours within it, 0 disables. Per-direction, so a missed
//--- opposite signal never blocks a later reversal, and causal, so live and drawn history
//--- declutter alike.
2026-07-21 00:03:45 -04:00
int m_signalClusterWindow ;
feat(signal): make the signal cooldown tunable, and add a hard any-direction gate
The declustering the charts needed already existed - NmsLiveAccept, per-direction
run-collapse plus cross-direction resolution plus strict alternation - and it was
already set to 10 bars. It could not be TUNED: SignalClusterWindow was a compile-
time const, so finding the right value needed a rebuild. That is the actual gap.
Now three inputs, as enum dropdowns:
Signal_CooldownScope per-direction, or a hard any-direction gate on top
Signal_CooldownBars SCB_OFF..SCB_50, default 10
Signal_CooldownMinutes SCM_OFF..SCM_1440, overrides bars when set
Minutes resolve against the CHART period and round UP, so a cooldown asked for in
wall-clock is never silently shorter than requested and survives a timeframe
change.
SCB_/SCM_ prefixes are deliberately unique. M15/M30/M60 are ALREADY members of
NF_LOOKBACK_PRESETS, and MQL5 binds a duplicated enum member to the first-declared
enum silently - the obvious names would have compiled straight into the news
filter's values.
THE ANY-DIRECTION GATE IS ADDITIVE, NOT A REPLACEMENT, and the first cut of this
had it backwards. Measured on the live log: the current rules draw 222 arrows over
4999 bars, while a BARE 10-bar cooldown permits up to 454 - because ALTERNATION is
what declutters today, not the window. Swapping the rules out would have roughly
doubled the clutter it was asked to remove. Layered, it can only ever suppress
more. Suppressed bars still advance the per-direction last-SEEN cursors, so a run
straddling the boundary does not restart as if it were fresh.
Applied at all THREE sites that must agree - live inference, OOS pass-3 scoring
and the chart renderer. Their own comments say why: an arrow set that does not
obey the same rule as the traded set shows calls the EA would never take.
Also corrects a stale comment that called this window "display only". It is not:
when it suppresses, the live path zeroes the signal outright - no arrow, no vote,
no position. Training never sees it, so these cost no retrain and are correctly
absent from the fingerprint.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 09:48:28 -04:00
//--- WHAT A KEPT SIGNAL BLOCKS - see SIGNAL_COOLDOWN_SCOPE. Under ANY_SIGNAL the three rules below
//--- collapse to ONE: a kept signal of either direction silences everything for the window. Rules
//--- 2 and 3 are then not merely redundant but WRONG - alternation would keep blocking a repeat
//--- direction forever, long after the cooldown a user asked for had expired.
SIGNAL_COOLDOWN_SCOPE m_signalCooldownScope ;
2026-08-22 00:24:45 -04:00
//--- Per-bar predicted signed signal for THIS era (index = now-relative bar index; -2 = not
//--- scored this era). Recording (not drawing) also decouples NMS from pass 2's SHUFFLED order,
//--- which no inline cursor could dedup.
2026-07-21 00:03:45 -04:00
double m_arrowSignalCache [ ] ;
2026-07-21 12:30:29 -04:00
//--- Live-side NMS state: bar TIME of the last SEEN signal per direction (advances on every same-
//--- direction bar, kept or suppressed, so a contiguous live run collapses to one) plus the cached
//--- accept/suppress decision for that exact bar (keeps repeated same-bar RefreshLatestSignal calls
//--- idempotent - re-evaluating the same bar returns its first decision, not a flipped one). 0 = none.
2026-07-21 00:03:45 -04:00
datetime m_nmsLiveBuyTime ;
datetime m_nmsLiveSellTime ;
2026-07-21 12:30:29 -04:00
bool m_nmsLiveBuyAccept ;
bool m_nmsLiveSellAccept ;
//--- Last KEPT live signal of either direction, for cross-direction resolution: a Buy and a Sell
2026-08-22 00:24:45 -04:00
//--- within m_signalClusterWindow bars are flicker at one turn zone (real opposite pivots are a
//--- whole leg apart), so only the higher-confidence side is kept.
2026-07-21 12:30:29 -04:00
datetime m_nmsLiveKeptTime ;
ENUM_SIGNAL m_nmsLiveKeptDir ;
double m_nmsLiveKeptConf ;
2026-07-14 22:36:27 -04:00
datetime dtStudied ;
perf(deinit): I/O-free chart cleanup, dead-panel purge, skip clean weight saves
The 18:23 terminal close (20260825.log) killed two of six charts inside
OnDeinit: they printed "shutting down" then nothing for 5.9 s until
"Abnormal termination", stranding ~700 objects each - including the one
family no prefix sweep can reach, the control panel (CAppDialog names
its 15 objects <numeric instance id><control>, and a re-attach mints a
new id, so a killed panel is a permanent ghost; XTIUSD carried one
across sessions). The stall sat in the two file writes that preceded
all visible cleanup while the four sibling charts flooded the same
2013-era disk - the ~4x18MB-per-chart shutdown weight saves.
Three changes:
1. OnDeinit touches no file until the chart is clean. CVoteArrowStore
splits Save() into Snapshot() (the chart scan, in memory) and
WriteSnapshot() (the disk half, consuming). New order: status label,
vote-arrow snapshot, prefix sweep, panel destroy - all object ops -
then member sidecars, final sweep, timings, and only then the
visibility file, the vote-arrow write and the weight saves.
2. PurgeOrphanedPanelObjects() at OnInit: deletes numeric-prefix
CAppDialog ghosts by name (6 chrome + 9 buttons), qualifying a
prefix only when >=4 of OUR button names carry it, so a foreign
dialog sharing stock chrome names is never touched.
3. m_netDirty: set by every net mutation (both backProp sites, both
RestoreWeights sites, online learning conservatively, panel reset),
cleared only on a successful Net.Save. Shutdown AND the per-bar
autosave now skip the ~18MB write when the net is provably unchanged
- for converged ensembles that is every save - which removes the
very flood that starved the sibling charts. .stats still writes
every time (small; carries the vote record and calibration). A
skipped save leaves the .nnw header dtStudied stale, which is the
already-handled attach-after-offline-gap case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 18:46:51 -04:00
//--- TRUE whenever the live net's weights differ from the last successful .nnw save. Set by every
//--- mutation path (both Net.backProp sites, both Net.RestoreWeights sites, online learning);
//--- cleared only on a successful Net.Save. PersistWeightsOnShutdown skips the ~18MB write when
//--- clean - on a terminal close every chart's models used to write at once, and that flood
//--- starved two sibling charts' OnDeinit past MetaTrader's budget mid-cleanup (2026-08-25 18:23).
//--- Defaults TRUE (unknown state must save); the header's dtStudied watermark going stale on a
//--- skip is the same situation as attaching after an offline gap, which the new-bar gate already
//--- resolves on the first inference.
bool m_netDirty ;
2026-07-14 22:36:27 -04:00
long m_eraCount ; // cumulative era counter, persisted in the .nnw so restarts don't look like they reset progress
bool m_trainingComplete ; // persisted: true only once Train() converged (objective+stability), not just interrupted
bool bEventStudy ;
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
//--- This instance's study-event id (STUDY_EVENT_ID_BASE + construction order) and the tick-count
//--- when bEventStudy was last armed - see the STUDY_EVENT_ID_BASE comment for why these exist.
//--- All arming goes through ArmStudyEvent() so the id and the watchdog stamp can never drift apart.
ushort m_studyEventId ;
uint m_studyArmedTick ;
2026-07-14 22:36:27 -04:00
//--- out-of-sample holdout: share (%) of the study period never trained on, used only to
//--- measure genuine forward accuracy so overfitting shows up in the stats, not just live/OOS trading
int m_oosSplitPct ;
double dOosError ; // smoothed OOS mismatch rate (0..100), lower is better
double dOosForecast ; // smoothed OOS accuracy (0..100)
int m_oosSamples ; // count of OOS predictions evaluated this Train() call
2026-07-19 14:50:52 -04:00
//--- per-era raw (pre-softmax) output-neuron stats over pass 3's OOS scan, reset at pass 3
2026-08-22 00:24:45 -04:00
//--- start; surfaced in the era-end log line.
2026-07-19 14:50:52 -04:00
double m_oosOutMin [ 3 ] ;
double m_oosOutMax [ 3 ] ;
double m_oosOutSpreadSum ;
int m_oosOutCount ;
2026-08-20 09:54:23 -04:00
//--- WHY "Neutral" WON, per OOS bar. ApplyClassificationSoftmax() requires a STRICT majority and
2026-08-22 00:24:45 -04:00
//--- sends every tie to Neutral, so one label covers two events needing OPPOSITE fixes: strict -
//--- the net really ranks Neutral highest.
feat(diagnostics): split a reported "Neutral" into CHOSE vs TIED - they need opposite fixes
ApplyClassificationSoftmax() requires a STRICT majority over both rivals and
sends every tie, 2-way or 3-way, to Neutral. So "OOS recall Neutral:100%" is
two completely different events sharing one label:
CHOSE - the net genuinely ranks Neutral highest. A class-prior/label problem.
TIED - the top two are EXACTLY equal, so the net expressed no preference and
the tie-break reported Neutral. A SATURATION problem: the head is
SIGMOID, and a saturated sigmoid returns exactly 0.0f or 1.0f in the
DLL's float32, so two classes pinned to the same rail compare equal
and the bar is silently discarded.
Nothing in the logs could tell them apart, and the fixes point opposite ways.
Eras 1-25 of the 2026-08-17 solo PAI run read "Neutral 100%" at spread avg 0.99
- fully saturated - and broke out at era 27 as the spread fell to 0.75. That is
consistent with EITHER story. The user reports the Neutral phase on most runs,
so it is worth four longs to stop guessing.
Four per-era counters on the pass 3 OOS walk, reported as:
| Neutral CHOSE 12.4% / TIED 38.1% (of which B=S 1204) | rail 61.2%
m_oosNeutralStrict - Neutral strictly highest
m_oosNeutralTie - no strict winner; the tie-break produced Neutral
m_oosTieBuySell - the costly subset: Buy and Sell tied AT the top, i.e. a
DIRECTIONAL reading thrown away by float equality
m_oosRailBars - any raw output sitting on a sigmoid asymptote, the
saturation that makes exact ties possible at all
Read on the RAW logits, before ApplyClassificationSoftmax() overwrites TempData
in place. Legitimate because softmax is strictly monotone: it cannot change the
ordering and cannot break a tie either, so the raw reading and the decision
always agree. Placed alongside the existing min/max/spread capture so all the
output diagnostics describe the same values.
Measurement only - no decision path reads these.
NOT COMPILED - user compiles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 00:06:54 -04:00
long m_oosNeutralStrict ;
long m_oosNeutralTie ;
long m_oosTieBuySell ;
long m_oosRailBars ; // any raw output pinned to a sigmoid rail (<=0+eps or >=1-eps)
2026-07-14 22:36:27 -04:00
//--- per-era counts of the network's own classification of each bar it fed forward (IS+OOS),
2026-07-17 21:28:59 -04:00
//--- reset at the start of every era; surfaced in the status label text so class imbalance
2026-07-14 22:36:27 -04:00
//--- (e.g. the network collapsing to all-Neutral) is visible while training runs
int m_countBuySignals ;
int m_countSellSignals ;
int m_countNeutralSignals ;
//--- per-era counts of the *true* label of every bar fed forward (IS+OOS), reset alongside the
2026-08-22 00:24:45 -04:00
//--- predicted counts above.
2026-07-14 22:36:27 -04:00
int m_trueBuyCount ;
int m_trueSellCount ;
int m_trueNeutralCount ;
//--- snapshot of the class totals above, taken at the end of the PREVIOUS era (see Train()'s
2026-08-22 00:24:45 -04:00
//--- era-reset block) and held fixed for the whole of the current era.
2026-07-14 22:36:27 -04:00
int m_prevEraTrueBuyCount ;
int m_prevEraTrueSellCount ;
int m_prevEraTrueNeutralCount ;
refactor(oos): twenty-one counters with one lifetime become one object
SOosTally holds this era's OOS confusion counts and the rates they imply.
The signal keeps one member where it kept twenty-one, and the era-reset
block loses twenty of its twenty-one clearing lines.
THE SHAPE THIS ENDS is the one that produced 7452bd1: a group of tallies
read together but cleared one-per-line, so a second reset path could clear
a subset and leave stale numerators over restarted denominators. Reset()
is now the only way to clear them and it clears all of them.
The pair had already started to drift. m_oosBuyFired/m_oosBuyFiredHits sat
at line 1085 and their Sell twins at line 1140 - 55 lines and an unrelated
member apart, with the Buy comment still claiming to describe both.
DERIVED RATES MOVE WITH THE DATA. `(bars > 0) ? (int)MathRound(100.0 * x /
bars) : -1` was written out twelve times, and the "-1 means not measurable,
never 0" convention re-spelled at each - a convention the deploy gate
depends on, since every caller tests `< 0` to mean "this does not block".
One rounding rule and one sentinel now.
GROUPED BY LIFETIME, NOT BY NAME. m_oosSamples looks like it belongs here
and does not: it is RUN-level, reset only with the weights, and the status
panel prints it beside dOosError which is also a run-level EMA. That pairing
is correct and stays. But the confidence-calibration block divided per-era
numerators by it, naming the results `empiricalAccuracy` and
`avgClaimedConfidence` when neither is that - the run-level denominator
cancels in their ratio, so eraScale was right and the two named
intermediates were not. Now written as the ratio it actually is, with the
cancellation stated, so nobody logs or gates on a half that decays with era
count.
BEHAVIOUR UNCHANGED: every moved expression preserves its formula, its
denominator and its sentinel.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 14:15:22 -04:00
//--- THIS ERA'S OOS CONFUSION COUNTS - all twenty-one of them, cleared and read together. The
//--- per-class recalls they imply are the convergence gate that a model "winning" by calling
//--- everything Neutral must not pass, and the fired-population rates are what the deploy gate
//--- judges. Grouped by LIFETIME: m_oosSamples and dOosError are deliberately NOT in here, being
//--- run-level. See Training\OosTally.mqh.
SOosTally m_oos ;
2026-08-22 00:24:45 -04:00
//--- Confidence calibration. EMA-blended across eras so one noisy era cannot swing it, and
//--- clamped to [0.3, 1.5] so a degenerate OOS window cannot drive it somewhere absurd.
2026-07-17 23:21:12 -04:00
double m_confidenceCalScale ;
fix(telemetry): one ensemble member had never printed a single era line, in any run on record
The era-progress rate limit was a function-scope `static`:
static uint lastProgressLogTick = 0;
shouldLogProgress = (nowTick - lastProgressLogTick >= 5000);
In MQL5 that is ONE variable for the whole build, not one per object. A 5s
limit meant to keep a single model's console readable was therefore a limit
across the WHOLE ENSEMBLE, and it did not distribute fairly - it starved
whichever member finishes last, every era, deterministically.
Measured on today's run: the era barrier releases the members together and
LSTM landed 2.06s, 2.43s and 2.28s behind ConvLSTM on eras 1-3, against a 5s
window it could never reach. LSTM printed zero era lines. PAI, CONV and HYB
printed all of theirs - 3 each this run, 17 each in the 10:00 run, LSTM 0 in
both, and 0 again in the 08:32 run.
So one model in four has been training with NO per-era telemetry: no
per-class recall, no dW/W ratios, no zero-skill comparison, no deploy-bar
line. It was still doing the work - tier re-ranks, threshold fits and
exit-policy replays all appear on cadence - which is what made the hole look
like a grep that kept missing the line rather than a line that was never
written. It cost me the LSTM half of a gradient check earlier today.
Now a member, so each model rate-limits its own console output.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 14:48:56 -04:00
//--- Last era-progress Print for THIS member (see the era loop's 5s rate limit).
uint m_lastProgressLogTick ;
2026-08-22 00:24:45 -04:00
//--- ONE-SHOT DETECTABILITY REPORT: how many calls this configuration must fire before the
//--- deploy gate could certify an edge of a given size AT ALL, and what share of the OOS window
//--- that is. Purely a report; it gates nothing.
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
void ReportDetectability ( int oosBars ) { m_featureBuilder . ReportDetectability ( oosBars ) ; }
//--- m_detectabilityReported moved onto CFeatureBuilder as a real member (exclusive, ctor-init-list
//--- only elsewhere).
2026-07-26 18:33:12 -04:00
//--- There is deliberately NO minimum-confidence input here any more, and no member holding one.
2026-08-22 00:24:45 -04:00
//--- That is what makes ONE input genuinely govern both engines. See ConfidenceTier().
2026-07-28 17:42:12 -04:00
bool m_freezePriorCalibration ;
fix(ai): cap logit-adjustment strength to the head's usable logit range
tau=1.0 inverted the collapse instead of curing it. The head is SIGMOID, so
each output is bounded to [0,1] and the widest logit gap the net can express
between two classes is CLASS_LOGIT_SCALE * (1-0) = 6. The offsets are
tau*log(prior_c), whose spread on this 30:1 imbalance is 3.42 - so tau=1.0
spent 57% of the ENTIRE expressible range on the prior correction.
The network did the only thing available to it: saturate Buy/Sell outputs to
1.0 to overcome a -3.42 training handicap. The offsets are absent at
inference, so that surplus made every bar directional. Measured across all
five still-training charts: Neutral recall 0%, directional calls on ~100% of
bars, win rate 5-7% against a ~6% base rate - no information whatsoever -
while balanced accuracy read a flattering 58-64% because two of its three
terms sat near 95%. OOS accuracy 6%.
Menon et al. assume an unbounded logit head where a 3.42 shift is negligible
against the reachable range. It is not negligible here, so the strength is
now expressed RELATIVE to the range actually available:
tau_eff = min(tau_cfg, LOGIT_ADJUST_MAX_RANGE_FRACTION * SCALE / spread)
At 20% that gives tau 0.35 on this data. Deliberately a fraction rather than
a tau ceiling: it stays correct if CLASS_LOGIT_SCALE changes, if the head
becomes unbounded, or on any symbol whose imbalance differs. The input
remains effective below the cap, so dialling it down needs no rebuild.
Simulated at a signal strength where the task is genuinely learnable, the
precision/recall frontier is monotone: tau 1.0 -> 49.6% call rate at 6.4%
precision (base rate 6.1%, i.e. worthless); tau 0.35 -> 2.0% at 15.5%;
tau 0.15 -> 0.2% at 33.3%. The capped value lands in the same regime the
pre-logit-adjustment run occupied (1-6% of bars at 20-35% win rate).
Also logs the measured priors, the spread, and whether the cap bound.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 23:20:07 -04:00
bool m_logitAdjustLogged ;
fix: the imbalance correction never ran during the auto-tune search
Neutral collapse on all four topologies by era 5 with a 2:6 barrier
(recall Buy 0% / Sell 0% / Neutral 100%), and the panel stuck on
"measuring...". One root cause, and it was not the barrier.
The labels were fine: Buy 25.4% / Sell 22.0% / Neutral 52.5%, which is
exactly gambler's ruin for m=2,k=6 (2/8 = 25% per side), with only 0.1%
of Neutral coming from the vertical barrier - so the new m*k horizon
scaling is right, arguably generous.
What was broken: Train()'s era-start block wrapped UpdateClassPriors() in
`if(!m_evalMode)`. The auto-tune GA scores every candidate in eval mode,
and AutoTuneIndicators ships ON, so on a default configuration EVERY era
of the search ran with unmeasured priors. ApplyLogitAdjustment() requires
measured priors; without them it calls ClearLogitAdjustment() and returns.
So the entire search trained under PLAIN cross-entropy. With a 52.5%
majority class the optimum of plain CE is "always predict Neutral", and
that is precisely what all four models found. The panel followed: its
counters only advance on bars the model CALLED Buy or Sell, so a
collapsed model leaves them at zero and the line reads "measuring..."
forever.
This was latent, not new. It has been true for every auto-tuned run, but
it was invisible while the labels were near-balanced - last night's
accidental 1:1 barrier gave 43/40/17, where plain CE has no majority to
collapse into. Widening the stop to 2*ATR (correctly - 1*ATR is too tight
to survive noise) moved Neutral to the majority and exposed it.
The guard's stated fear cannot happen. These priors are measured from the
LABEL distribution, and the tuner only perturbs indicator periods
(MA/RSI/MACD/Ichimoku/AD). The barrier label depends on ATR, SL_Mode and
TP_Mode - none of which the search touches - so every candidate sees
byte-identical labels and identical priors. There is nothing to
contaminate. What the guard actually protected was the .stats write, and
that is gated separately: eval candidates never checkpoint and never
persist.
Also, because this is the THIRD quiet no-op to cost a run in this
codebase (after the fictional oversampling log line and the shadow-blend
skip):
- ApplyLogitAdjustment() now WARNS when it declines to install, instead
of silently clearing. A mechanism that cannot announce it is not
running is indistinguishable from one that is.
- The panel distinguishes "measuring..." (before era 1, nothing scored
yet - an honest warm-up) from "no directional calls yet" (eras trained,
zero calls - a finding, not a wait).
Both builds compile 0 errors / 0 warnings. No retrain forced by this
commit itself, but the collapsed models must be discarded.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 00:46:24 -04:00
//--- Latch for the counterpart warning: the correction DECLINING to install. See ApplyLogitAdjustment().
bool m_logitAdjustSkipWarned ;
feat(ai): logit-adjusted loss, replacing oversampling and the post-hoc prior
Menon et al. 2021 (ICLR), "Long-tail learning via logit adjustment": add
tau*log(prior_c) to each class logit inside the training gradient. Softmax
CE on adjusted logits is consistent for BALANCED error - the metric
checkpoint selection already ranks on - so the loss and the deploy decision
finally optimize the same thing.
The engine already computed a true softmax + categorical-CE gradient and
wrote it over the per-neuron sigmoid delta, so this is an offset added to
three logits in the two places that gradient is built (backProp scalar path
and backPropOCL). No backend, kernel or DLL change; the forward pass and
every inference path are untouched, which is the point - the network learns
to absorb the offset, so its raw argmax becomes the balanced-optimal
decision with nothing applied at inference.
Replaces rather than stacks. Minority replay is disabled while this is on,
and the post-hoc inference prior is forced off. Stacking is not a
theoretical worry: simulated on the measured 1118/1119/34298 distribution
in the weak-signal regime, plain CE collapses to Neutral (33.4% balanced,
Buy 0%), replay reaches 48.1%, logit adjustment 50.9% with better balance -
and BOTH together score 45.4% with Neutral recall at 0%, worse than either
alone. Buda et al. 2018 predicts exactly that.
Motivation from the six-chart run: every topology took one direction to
~50% recall and abandoned the other, the direction chosen arbitrarily (the
batch-norm control went Buy 1% / Sell 42%, the inverse of the other five).
One era in 1,301 cleared the per-class recall floor.
Fingerprinted conditionally, so the converged 60.7% models on disk keep
their filenames and stay loadable as the fallback.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 19:05:14 -04:00
2026-07-23 19:36:34 -04:00
//--- True class base rates (natural, un-oversampled), measured from the label distribution each era
//--- (UpdateClassPriors, EMA-blended for stability) and PERSISTED alongside the weights (.stats
//--- sidecar) so live inference - including after a restart, when no training re-runs - calibrates
//--- exactly as training did. 0 = not yet measured => AdjustedSignalFromSoftmax falls back to raw.
double m_priorBuy , m_priorSell , m_priorNeutral ;
refactor(oos): twenty-one counters with one lifetime become one object
SOosTally holds this era's OOS confusion counts and the rates they imply.
The signal keeps one member where it kept twenty-one, and the era-reset
block loses twenty of its twenty-one clearing lines.
THE SHAPE THIS ENDS is the one that produced 7452bd1: a group of tallies
read together but cleared one-per-line, so a second reset path could clear
a subset and leave stale numerators over restarted denominators. Reset()
is now the only way to clear them and it clears all of them.
The pair had already started to drift. m_oosBuyFired/m_oosBuyFiredHits sat
at line 1085 and their Sell twins at line 1140 - 55 lines and an unrelated
member apart, with the Buy comment still claiming to describe both.
DERIVED RATES MOVE WITH THE DATA. `(bars > 0) ? (int)MathRound(100.0 * x /
bars) : -1` was written out twelve times, and the "-1 means not measurable,
never 0" convention re-spelled at each - a convention the deploy gate
depends on, since every caller tests `< 0` to mean "this does not block".
One rounding rule and one sentinel now.
GROUPED BY LIFETIME, NOT BY NAME. m_oosSamples looks like it belongs here
and does not: it is RUN-level, reset only with the weights, and the status
panel prints it beside dOosError which is also a run-level EMA. That pairing
is correct and stays. But the confidence-calibration block divided per-era
numerators by it, naming the results `empiricalAccuracy` and
`avgClaimedConfidence` when neither is that - the run-level denominator
cancels in their ratio, so eraScale was right and the two named
intermediates were not. Now written as the ratio it actually is, with the
cancellation stated, so nobody logs or gates on a half that decays with era
count.
BEHAVIOUR UNCHANGED: every moved expression preserves its formula, its
denominator and its sentinel.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 14:15:22 -04:00
//--- The live-fired population (see SOosTally::buyFired), BUCKETED BY CONFIDENCE TIER (ConfidenceTier(),
2026-08-22 00:24:45 -04:00
//--- 4 buckets quartiled from the head's structural floor).
feat(ai): measure precision per confidence tier; fix stale metric labels
Two things the 2026-07-30 run exposed.
1. Every user-facing message still called the selection metric "balanced
accuracy". It has ranked on directional precision since a142749, so
"CONVERGED ... balanced accuracy 32.5%" was reporting a 32.5%
PRECISION as if it were macro-recall, while the same era logged an
actual balanced accuracy of 49%. Two different numbers under one
name, in the line that announces a deploy. Relabelled at every site,
including the stage-3 refusal, which still described the per-class
recall floor that stopped being the gate.
2. Precision is now bucketed by confidence tier and logged per era,
both per-tier and cumulatively from each tier upward:
| tier prec T0:19%(410)[>=28%/1204] T1:31%(520)[>=34%/794] ...
The per-tier number says whether confidence is calibrated to
correctness at all; if it does not rise T0->T3, raising the floor
buys nothing and that is the finding. The ">=" number is what a floor
would actually deliver, with its fire count, so the coverage cost is
visible in the same line. Tier weights are 25/50/75/100, so for an
AI-only config Min_Vote_Open maps straight across: 50 = ">=T1",
75 = ">=T2", 100 = ">=T3".
Bucketing happens at the existing live-fired accounting site, so it
measures exactly the population that trades - not the raw argmax.
Both builds compile 0 errors, 0 warnings. No retrain needed for either.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 11:47:15 -04:00
int m_oosTierFired [ 4 ] , m_oosTierHits [ 4 ] ;
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
//--- Set by RankTiersFromOos() the first time this model measures its own tiers on held-out
//--- bars. Gates the signal DB out of this filter's pattern weights from then on.
bool m_tiersSelfRanked ;
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era
Full-pipeline analysis after "threshold 30, attained often, nothing drawn,
still glued to buy". The log falsified the premise before any code did:
21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0%
21:42:07 swept 4999, 0 voters, drew 0
21:51:30 swept 4999, 0 voters, drew 0
21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0%
The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root
cause, three symptoms: every display path read m_arrowSignalCache, which is
wiped to sentinel at each era start and only complete again when pass 3
finishes. With eras at ~30s and a sweep at ~17s:
* ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its
else-branch deleted the arrow on every voteless bar - erasing the previous
sweep's entire output. The chart cycled populated -> blank -> populated;
the user kept catching the blank phase.
* READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every
era and fell through to dPrevSignal - the frozen purge-band edge bar that
reads Buy. 659638e fixed which bar was frozen, not the freezing.
* VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a
different fraction of half-rebuilt caches.
THE FIX, structural rather than another patch:
1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one
moment the cache is complete - and now copies it (raw signals, newest
LOOKBACK+16 bars) into member-owned snapshot state, unconditionally,
BEFORE its early return: an all-Neutral era is a snapshot worth showing,
not an absence of one. Raw signals rather than votes, so a tier re-rank
between eras reprices them at read time via LiveVoteContribution for free.
2. The sweep (SnapshotVoteAt) and the prospective readout both read
snapshots; the readout's fallback chain is live-cache -> snapshot ->
dPrevSignal, and the snapshot leg is the one that fires most of the time.
3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual
sub-threshold vote takes an arrow down. This alone ends the wipe half of
the flicker even where snapshots are missing (before the first era).
4. Arming moved from an era-counter diff (which fires at era BOUNDARIES,
i.e. precisely when caches are about to be wiped) to
g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's
snapshot just got fresher", the only event a redraw can act on. 60s rate
limit collapses the four members' burst into one sweep. Classic-only
charts arm once at start.
5. Census now reports the direction split - "922 had a voter (610 buy / 312
sell)" - so "the vote leans buy" is checkable from the log instead of
inferred from arrow colours.
Also visible in the log and worth knowing: the threshold flip-flopped
30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51
ran at 40), so part of the observed blankness was configuration, not code.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 22:03:20 -04:00
//--- ERA-END SNAPSHOTS of the arrow cache, taken in RankTiersFromOos() at pass-3 completion -
2026-08-22 00:24:45 -04:00
//--- the one moment the cache is complete for the era.
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era
Full-pipeline analysis after "threshold 30, attained often, nothing drawn,
still glued to buy". The log falsified the premise before any code did:
21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0%
21:42:07 swept 4999, 0 voters, drew 0
21:51:30 swept 4999, 0 voters, drew 0
21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0%
The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root
cause, three symptoms: every display path read m_arrowSignalCache, which is
wiped to sentinel at each era start and only complete again when pass 3
finishes. With eras at ~30s and a sweep at ~17s:
* ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its
else-branch deleted the arrow on every voteless bar - erasing the previous
sweep's entire output. The chart cycled populated -> blank -> populated;
the user kept catching the blank phase.
* READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every
era and fell through to dPrevSignal - the frozen purge-band edge bar that
reads Buy. 659638e fixed which bar was frozen, not the freezing.
* VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a
different fraction of half-rebuilt caches.
THE FIX, structural rather than another patch:
1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one
moment the cache is complete - and now copies it (raw signals, newest
LOOKBACK+16 bars) into member-owned snapshot state, unconditionally,
BEFORE its early return: an all-Neutral era is a snapshot worth showing,
not an absence of one. Raw signals rather than votes, so a tier re-rank
between eras reprices them at read time via LiveVoteContribution for free.
2. The sweep (SnapshotVoteAt) and the prospective readout both read
snapshots; the readout's fallback chain is live-cache -> snapshot ->
dPrevSignal, and the snapshot leg is the one that fires most of the time.
3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual
sub-threshold vote takes an arrow down. This alone ends the wipe half of
the flicker even where snapshots are missing (before the first era).
4. Arming moved from an era-counter diff (which fires at era BOUNDARIES,
i.e. precisely when caches are about to be wiped) to
g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's
snapshot just got fresher", the only event a redraw can act on. 60s rate
limit collapses the four members' burst into one sweep. Classic-only
charts arm once at start.
5. Census now reports the direction split - "922 had a voter (610 buy / 312
sell)" - so "the vote leans buy" is checkable from the log instead of
inferred from arrow colours.
Also visible in the log and worth knowing: the threshold flip-flopped
30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51
ran at 40), so part of the observed blankness was configuration, not code.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 22:03:20 -04:00
double m_overlaySigSnap [ ] ;
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
//--- Deployed-model rebuild state machine (AdvanceDeployedRebuild): 0 = not started / not needed,
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
//--- 1 = rescan in flight, 2 = done. Labels are resolved inline during scoring, NOT prebuilt -
//--- see AdvanceDeployedRebuild's header for the windowing bug a prebuild stage caused.
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
int m_deployedRebuildStage ;
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era
Full-pipeline analysis after "threshold 30, attained often, nothing drawn,
still glued to buy". The log falsified the premise before any code did:
21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0%
21:42:07 swept 4999, 0 voters, drew 0
21:51:30 swept 4999, 0 voters, drew 0
21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0%
The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root
cause, three symptoms: every display path read m_arrowSignalCache, which is
wiped to sentinel at each era start and only complete again when pass 3
finishes. With eras at ~30s and a sweep at ~17s:
* ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its
else-branch deleted the arrow on every voteless bar - erasing the previous
sweep's entire output. The chart cycled populated -> blank -> populated;
the user kept catching the blank phase.
* READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every
era and fell through to dPrevSignal - the frozen purge-band edge bar that
reads Buy. 659638e fixed which bar was frozen, not the freezing.
* VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a
different fraction of half-rebuilt caches.
THE FIX, structural rather than another patch:
1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one
moment the cache is complete - and now copies it (raw signals, newest
LOOKBACK+16 bars) into member-owned snapshot state, unconditionally,
BEFORE its early return: an all-Neutral era is a snapshot worth showing,
not an absence of one. Raw signals rather than votes, so a tier re-rank
between eras reprices them at read time via LiveVoteContribution for free.
2. The sweep (SnapshotVoteAt) and the prospective readout both read
snapshots; the readout's fallback chain is live-cache -> snapshot ->
dPrevSignal, and the snapshot leg is the one that fires most of the time.
3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual
sub-threshold vote takes an arrow down. This alone ends the wipe half of
the flicker even where snapshots are missing (before the first era).
4. Arming moved from an era-counter diff (which fires at era BOUNDARIES,
i.e. precisely when caches are about to be wiped) to
g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's
snapshot just got fresher", the only event a redraw can act on. 60s rate
limit collapses the four members' burst into one sweep. Classic-only
charts arm once at start.
5. Census now reports the direction split - "922 had a voter (610 buy / 312
sell)" - so "the vote leans buy" is checkable from the log instead of
inferred from arrow colours.
Also visible in the log and worth knowing: the threshold flip-flopped
30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51
ran at 40), so part of the observed blankness was configuration, not code.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 22:03:20 -04:00
int m_overlaySnapBars ;
double m_prospectiveSigSnap ;
feat(hud): per-member neuron lines + a vote label that moves as the nets learn
Both 2026-08-19 reports were the same staleness: every source behind the
label was an ERA artifact (live cache refills at pass-3 completion, the
snapshot copies once per era, dPrevSignal is the frozen purge-band edge
bar) - so the readout stepped at era cadence at best, stayed glued to
one direction, and lagged the era counter.
DisplayInference(): throttled (4s, 1s across an era boundary),
SIDE-EFFECT-FREE forward of the current decision bar (window ending on
bar 1, same question the live path asks) through the LEARNER net.
Batch-norm running stats are bracketed frozen/RESTORED via the new
CNet::GetBatchNormFrozen() + CNeuronBatchNormOCL::StatsFrozen() - restore,
not unfreeze, because a display tick can land between pass-3 chunks whose
whole scan holds them frozen. Writes nothing a trading or training path
reads (dPrevSignal, NMS state, tallies, watermarks all untouched;
RefreshLatestSignal is not reusable here precisely because it writes all
of them). LSTM safe by construction: h/c zeroed per forward.
ProspectiveVote() reads the fresh forward as its FIRST source; the
era-artifact chain becomes the fallback (meta head, warm-up, window
holes).
DisplayHudLine(): the reference library's training label, per ensemble
member - name, output activations (softmax probs or raw scalar), the
decision, its weighted vote (the exact consensus numerator term), era,
recent average error, "(trn)" while not vote-capable. Rendered under the
vote line in RefreshVoteReadout BEFORE the live-vote defer (member lines
are telemetry, not tradable readings), coloured by the member's own
direction in muted tones - the vote line's strict
green-only-when-it-would-trade rule is untouched.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 08:48:34 -04:00
//--- HUD display state: the last throttled DISPLAY forward's outputs (softmax probabilities for
//--- 3-class heads, the raw scalar in [0] for regression heads), the adjusted signal they
2026-08-22 00:24:45 -04:00
//--- resolve to, and the throttle bookkeeping.
feat(hud): per-member neuron lines + a vote label that moves as the nets learn
Both 2026-08-19 reports were the same staleness: every source behind the
label was an ERA artifact (live cache refills at pass-3 completion, the
snapshot copies once per era, dPrevSignal is the frozen purge-band edge
bar) - so the readout stepped at era cadence at best, stayed glued to
one direction, and lagged the era counter.
DisplayInference(): throttled (4s, 1s across an era boundary),
SIDE-EFFECT-FREE forward of the current decision bar (window ending on
bar 1, same question the live path asks) through the LEARNER net.
Batch-norm running stats are bracketed frozen/RESTORED via the new
CNet::GetBatchNormFrozen() + CNeuronBatchNormOCL::StatsFrozen() - restore,
not unfreeze, because a display tick can land between pass-3 chunks whose
whole scan holds them frozen. Writes nothing a trading or training path
reads (dPrevSignal, NMS state, tallies, watermarks all untouched;
RefreshLatestSignal is not reusable here precisely because it writes all
of them). LSTM safe by construction: h/c zeroed per forward.
ProspectiveVote() reads the fresh forward as its FIRST source; the
era-artifact chain becomes the fallback (meta head, warm-up, window
holes).
DisplayHudLine(): the reference library's training label, per ensemble
member - name, output activations (softmax probs or raw scalar), the
decision, its weighted vote (the exact consensus numerator term), era,
recent average error, "(trn)" while not vote-capable. Rendered under the
vote line in RefreshVoteReadout BEFORE the live-vote defer (member lines
are telemetry, not tradable readings), coloured by the member's own
direction in muted tones - the vote line's strict
green-only-when-it-would-trade rule is untouched.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 08:48:34 -04:00
double m_dispProbs [ 3 ] ;
double m_dispSignal ;
bool m_dispValid ;
uint m_dispStamp ;
long m_dispEra ;
feat(logs): throttle the settled per-era diagnostics - measured 22MB/9.5h of confirmed-working systems
Measured from the journal (2026-08-19): the era deep-dive line (~2KB) plus
the excursion verdict, tier re-rank, calibration move, barrier hold and
selection-regressed note each printed EVERY era for EVERY member - ~940
eras/member/day - long after the systems they watch were confirmed
working. Yesterday's file was 1.3GB (70% of it the news-filter calendar
spam the sweep fix already removed).
VerboseMode returns as an INPUT (demoted 2026-08-01 for the marketplace;
that track is dead since the 2026-08-16 pivot) and gains a second job:
false throttles each settled per-era print to eras 0-3 plus every
TRAIN_LOG_EVERY_ERAS-th (25 ~= one deep-dive per ~15min per member);
true restores the per-era firehose, flippable live.
Never throttled: anything that marks a CHANGE - new bests, restores +
eta decays, plateau stage transitions, deploy approvals, warnings,
errors, the label-cache/adoption one-shots, and the combined-vote gate
line (the active system's primary telemetry, still every era).
Semantic fixes over blanket gating:
- barrier hold now ARMS silently and prints only when the hold outlasts
the 2-min report interval - a brief hold every era is the design, the
long hold is the watchdog case the line exists for;
- the ensemble deploy REFUSAL prints immediately when its reason
changes (that is a finding), on cadence when unchanged;
- the filtered-view census prints when its RESULT moves (drawn count,
or strongest vote by >=2pp) and at least every 10th sweep.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 09:36:15 -04:00
//--- Which refusal reason the ensemble deploy verdict printed last (1 = never tradeable,
//--- 2 = joint checkpoint incomplete, 3 = gate not cleared): a CHANGED reason prints
//--- immediately, the same reason repeats only on the TRAIN_LOG_EVERY_ERAS cadence.
int m_lastEnsRefusalKey ;
2026-08-22 00:24:45 -04:00
//--- Cumulative (compounded, persistent) DIRECTIONAL accuracy = the win-rate of the model's
//--- Buy/Sell calls: of the bars it actually called Buy or Sell, how many matched the true
//--- label.
2026-07-25 00:02:34 -04:00
long m_cumIsCorrect , m_cumIsTotal ;
long m_cumOosCorrect , m_cumOosTotal ;
2026-07-23 19:36:34 -04:00
//--- Latest live-fired precision (%) and fire count per direction (-1 = n/a), cached at era end for
//--- the status panel/log the same way m_lastBuyRecallPct is (see its comment).
int m_lastBuyFiredPrecPct , m_lastSellFiredPrecPct ;
int m_lastBuyFired , m_lastSellFired ;
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- ZigZag repainting embargo for the swing-context FEATURES, in bars. The stock ZigZag revises
//--- its most recent legs as new bars arrive, so a feature read is trusted only once this many
//--- MORE bars have closed after it. The LABEL's lookahead control is the pivot-pair finality
//--- rule, not this.
2026-07-14 22:36:27 -04:00
int m_swingConfirmationBars ;
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- LABEL RESOLUTION LAG in bars: how long after entry this bar's label became KNOWABLE - the
//--- earliest bar its pivot pair could have committed on. NOT a diagnostic. See
//--- EffectiveSampleSize().
fix(labels): overlapping-label sample correction + horizon cap on the scale ladder
Three defects, all surfaced by the 2026-08-17 SP500 H4 run that shipped
stop 4.86 / target 9.71 (width 14.57*ATR, horizon 384).
1. EVERY STANDARD ERROR ASSUMED INDEPENDENT SAMPLES. Triple-barrier labels
started one per bar overlap by the label's lifespan, so n calls are worth
~n/L independent observations (Lopez de Prado, AFML ch. 4 - sample
uniqueness). All three sqrt(p(1-p)/n) sites divided by the RAW count.
The tell: the operating point's null-of-the-maximum gate is family-wise and
should fire on ~5% of eras under the null. Measured fire rates - PAI 47/73
(64%), ConvLSTM 9/24, LSTM 8/21 (38%), CONV 4/62 (6%). CONV, the only model
whose margin distribution admits few bins, sat on the null; the rest cleared
a bar that was too low by ~sqrt(L). PAI's deployed threshold consequently
alternated between the ENDS of its own range era to era (0.10 -> 0.88 ->
0.86 -> 0.66; coverage 16% <-> 73%).
TripleBarrierLabel now records when each label became KNOWABLE - the first
winning touch, or both stops, or the timeout - and the prebuild accumulates
the mean. EffectiveSampleSize() feeds the operating point, the member deploy
gate and the ensemble vote gate. Conservative by construction (n/L is an
upper bound on the damage); gates get harder, never easier.
2. THE SCALE LADDER RAN AWAY, again. Horizon scales as swingMedian*sl*tp, and
since 4d8cb08 reachability is measured OVER that horizon - so a wider rung
buys itself the time that makes it look reachable. Same target -> horizon ->
reach -> target loop the excursion window is kept short to avoid; fixing the
window confusion reopened it through the other door. It walked 128 -> 256 ->
384 bars and stopped at q90, the widest rung there is, with every rung
reading 39-48% against a 20% floor. A floor nothing fails selects nothing.
Rungs whose required horizon exceeds BARRIER_HORIZON_MAX are now rejected -
the same rule ReportGeometryExpectancyScan already applied. It was printing
the shipped pair as CLAMPED and disqualified ('h384!') two lines under the
deriver that chose it: two subsystems, one geometry, opposite verdicts.
3. THE RUNG SNAP DESTROYED THE RATIO IT WAS COMPARING. Both legs snapped
independently to the coarse first-passage grid, re-rating each candidate:
q90 4.86/9.71 -> 5.00/10.00 (2.00), q85 4.07/8.14 -> 5.00/10.00 (IDENTICAL
measurement), q75 3.07/6.13 -> 4.00/6.50 (1.63 - a nearer target). So the
ladder compared win shares taken at ratios from 1.63 to 2.17 and read the
differences as scale. It is why the reach column came out non-monotone in
width (q75 48.5% above q90 42.9%). The stop now snaps to its nearest rung in
log space and the target follows the ratio off it; the pair actually measured
is returned and logged, so a collision reads as a collision.
Also: LadderWinShare guarded against the conditional (fractal) geometry path,
which fills n from m_fracLegCount while leaving idxList empty - a latent
out-of-bounds on a currently-dead path.
New log lines: mean label lifespan and effective n on the label-cache line, the
required-vs-available horizon per rung, and the grid pair the reconciliation
actually measured (its tolerance now scales with the grid skew instead of a flat
5pp).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 01:12:05 -04:00
int m_lastLabelLifespan ;
diag(label): expose bars-to-pivot - the confound the payoff test was missing
CORRECTION to what the payoff instrument was measuring. The 5-bar horizon looked
like the powered test and it is confounded.
SwingPivotDirectionLabel returns Buy when a swing LOW lands up to
PIVOT_LABEL_TOLERANCE_BARS bars AHEAD, and says the quiet part itself: gating on
where the pivot sits relative to entry "would drop exactly the bars where the
turn has not finished coming to us", and how much adverse move remains before
the turn "is a trade-management question".
So on a CORRECT Buy call price is often still falling for d more bars. A window
shorter than d measures the APPROACH, not the leg, and its negative contribution
is expected on the calls that are RIGHT. The tight null at 5 bars
(-0.012 +/- 0.074) is therefore not evidence of no payoff. Neither horizon is
both clean and powered: 5 bars is powered and confounded, 18-19 is clean and has
an SE of 0.277.
(idx - P1) was computed in the label and thrown away. Now cached beside
m_labelResolveAge under the same validity flag, and bucketed in the era verdict.
DELIBERATELY NOT USED AS A PER-CALL HORIZON, which is the trap sitting right
next to this: d exists only on bars the label found a pivot for, so a horizon
that varied with d would hand correct and incorrect calls different windows and
bias the comparison outright. The horizon stays fixed; d only buckets.
The bucket for "the label called no pivot here" is reported by name rather than
folded in, because it is the control the others are read against. Buckets 1..N
condition on the label, so they describe the MECHANISM, not what a book earns.
Reads: rising with d means the edge is in EARLY calls and the tolerance window
is spending it - fixable by reweighting the loss, not by a new label. Flat means
that hypothesis dies.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 08:16:47 -04:00
//--- Companion to m_lastLabelLifespan, written by the same call - see m_labelBarsToPivot.
int m_lastLabelBarsToPivot ;
refactor(labeling): CTripleBarrier - one copy of the fill/barrier arithmetic
Session B of the feature-selection/labeling refactor track. Extracts the two
pieces of triple-barrier arithmetic that were genuinely duplicated or
scattered, taking price/ATR/geometry as plain arguments - no chart, no
indicator handle - so it is testable with synthetic numbers.
CTripleBarrier::ComputeLevels() replaces the fill/barrier level arithmetic
that TripleBarrierLabel() and SimulateTradeOutcome() each spelled out by
hand; their own comments already called it "IDENTICAL... deliberately and by
copy." One caller resolves both sides at once (the both-won tie-break needs
both); the other selects the side its isLong argument names. Same for
ApplyMinStopWidening(), the broker-minimum-stop floor both walks applied.
Fuzzed 200k random (entry, spread, risk, reward, minStop, isLong) tuples
against both original hand-written forms: 0 mismatches.
CLabelOverlap replaces m_labelLifespanSum/m_labelLifespanCount - two members
reset from three separate call sites (constructor, label-cache rebuild), the
exact "N loose members cleared in more than one place" shape a candidate-
geometry incident (7452bd1) turned into a live bug. One object, one Reset(),
default-constructed like every other object member. MeanLabelLifespan() and
EffectiveSampleSize() on the signal become thin forwarders with an unchanged
signature - every one of their ~15 existing callers, direct and through the
CAIBaseTrainingData adapter, is unaffected.
SnapHorizonToLadder() forwards to CTripleBarrier::SnapToLadder(), the ladder
array's one remaining copy; EffectiveHorizonMax() (the close-all cache) stays
on the signal since that state has no clean argument form.
NOT extracted: TripleBarrierLabel()'s ~200-line walk itself. It resolves both
sides simultaneously, tracks the first-passage ladder, and feeds the label
every live order is sized from; a rewrite of it cannot be checked without a
compiler, so only the two pieces provably identical to their originals moved.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 18:57:17 -04:00
//--- Running mean of the above over every bar the label cache has resolved this process - see
//--- MeanLabelLifespan()/EffectiveSampleSize(). Was two members (a sum and a count) reset from
//--- three separate sites; CLabelOverlap owns both and resets in one call - see its declaration.
CLabelOverlap m_labelOverlap ;
2026-08-22 00:24:45 -04:00
//--- DIRECTIONAL CONFIDENCE THRESHOLD - see DIR_CONF_THRESHOLD_BINS for the rationale. Refitted
//--- at the end of every pass 2 from that era's own IS margins, because the margin distribution
//--- moves with the weights.
feat: fitted directional confidence threshold - selectivity gets a mechanism
The training loss and the selection metric wanted different things and only
the second one knew it. Logit-adjusted cross-entropy has no term for "how
often should I trade", so the head calls a direction on 87-91% of bars. The
selection metric is precision x coverage credit, saturating at the coverage
floor - above the floor extra calls earn NOTHING and only precision counts.
So selection wanted few good calls, the loss produced many mediocre ones, and
all selection could do was pick the least-bad era out of what it was handed.
Nothing pushed the model toward selectivity.
This gives the decision RULE the policy instead of distorting the loss (which
is estimating class probabilities correctly, and a probability estimate should
not be bent to encode a trading policy - Elkan 2001: estimate, then choose the
operating point separately). AdjustedSignalFromSoftmax now abstains unless the
winning direction's softmax margin over its best rival clears a fitted
threshold. Margin, not the winning probability: the latter moves with overall
calibration rather than with how close the decision actually was.
Fitted on IS, applied to OOS and live. Pass 2 already forward-passes every IS
sample, so the margin histogram is harvested there for free (primary
occurrences only, so the oversampled replay queue cannot skew the operating
point); the fit runs at the end of pass 2, BEFORE pass 3, so the deploy gate
grades the thresholded model on bars the threshold never saw. Fitting on
pass 3's own predictions would be choosing the operating point on the data
being graded - the best-of-N error corrected in five other places here.
Objective: maximise IS directional precision subject to still clearing the
SAME coverage floor the deploy gate uses (base rate x 0.25, re-derived
locally so the two cannot drift apart). Swept top-down in one pass; ties go
to the LOWER threshold, since equal precision for less coverage is strictly
worse. Under DIR_CONF_MIN_FIT_CALLS (200) it runs unthresholded rather than
on a guess.
The threshold is part of the MODEL, not the run: captured with
Net.CaptureWeights(), restored with the weights at both restore sites, and
appended to the .cfg under the same length-guard convention so a deployed
model reloads at the operating point its gate actually cleared. A pre-2026-08-09
.cfg reads 0.0, which is exactly the behaviour it was trained under.
Per-era line now prints "@margin>=X.XX" next to coverage, so a coverage drop
can be attributed to the operating point rather than guessed at.
Both build variants compile 0 errors / 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:04:37 -04:00
double m_dirConfThreshold ;
2026-08-22 00:24:45 -04:00
//--- The value that belongs to the CHECKPOINTED weights.
feat: fitted directional confidence threshold - selectivity gets a mechanism
The training loss and the selection metric wanted different things and only
the second one knew it. Logit-adjusted cross-entropy has no term for "how
often should I trade", so the head calls a direction on 87-91% of bars. The
selection metric is precision x coverage credit, saturating at the coverage
floor - above the floor extra calls earn NOTHING and only precision counts.
So selection wanted few good calls, the loss produced many mediocre ones, and
all selection could do was pick the least-bad era out of what it was handed.
Nothing pushed the model toward selectivity.
This gives the decision RULE the policy instead of distorting the loss (which
is estimating class probabilities correctly, and a probability estimate should
not be bent to encode a trading policy - Elkan 2001: estimate, then choose the
operating point separately). AdjustedSignalFromSoftmax now abstains unless the
winning direction's softmax margin over its best rival clears a fitted
threshold. Margin, not the winning probability: the latter moves with overall
calibration rather than with how close the decision actually was.
Fitted on IS, applied to OOS and live. Pass 2 already forward-passes every IS
sample, so the margin histogram is harvested there for free (primary
occurrences only, so the oversampled replay queue cannot skew the operating
point); the fit runs at the end of pass 2, BEFORE pass 3, so the deploy gate
grades the thresholded model on bars the threshold never saw. Fitting on
pass 3's own predictions would be choosing the operating point on the data
being graded - the best-of-N error corrected in five other places here.
Objective: maximise IS directional precision subject to still clearing the
SAME coverage floor the deploy gate uses (base rate x 0.25, re-derived
locally so the two cannot drift apart). Swept top-down in one pass; ties go
to the LOWER threshold, since equal precision for less coverage is strictly
worse. Under DIR_CONF_MIN_FIT_CALLS (200) it runs unthresholded rather than
on a guess.
The threshold is part of the MODEL, not the run: captured with
Net.CaptureWeights(), restored with the weights at both restore sites, and
appended to the .cfg under the same length-guard convention so a deployed
model reloads at the operating point its gate actually cleared. A pre-2026-08-09
.cfg reads 0.0, which is exactly the behaviour it was trained under.
Per-era line now prints "@margin>=X.XX" next to coverage, so a coverage drop
can be attributed to the operating point rather than guessed at.
Both build variants compile 0 errors / 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:04:37 -04:00
double m_bestDirConfThreshold ;
fix: the operating point was fitted on bars the net had memorized
FitDirConfThreshold harvested its margin histogram from pass 2's own
backprop samples. Pairing every fit against the same era's OOS result
shows what that measured:
PAI era 1 IS 25% cov @ 66.1% (-0.8pp) -> OOS 64% (-3pp) gap +2.1pp
PAI era 76 IS 90% cov @ 79.6% (+12.7pp) -> OOS 65% (-2pp) gap +14.6pp
LSTM era 9 IS 77% cov @ 81.6% (+14.6pp) -> OOS 63% (-4pp) gap +18.6pp
The gap grows monotonically while OOS stays flat, so within a handful of
eras the curve stops describing behaviour on unseen bars. That is fatal
here specifically, because the objective branches on the SIGN of
(p - break-even): the memorized curve reads +12pp at 95% coverage, so
coverage x (p - p0) correctly maximises coverage and returns ~0.02 - fire
on every bar. The "p < p0 -> get more selective" branch, which is the
actual regime and the entire point of 983a6a3, could never fire because IS
never showed p < p0.
Carve a calibration slice out of the IS span - DIR_CONF_CALIB_PCT_OF_IS,
purged from backprop by one label horizon on BOTH sides (the far-side
purge is not optional: without it the newest training bars carry labels
partly decided by price action inside the slice, putting the memorization
straight back into the curve). Score it in a new chunked pass 2.5, after
pass 2 has trained and before pass 3 grades - the only position where the
histogram is simultaneously not-trained-on, not-graded, and current with
the weights it will be applied to.
Costs 15% of the training data. Worth it beyond honesty: the deploy gate
needs dirPrecPct > chance + EDGE_MIN_SIGMAS*SE, and a threshold pinned
near zero dilutes any edge concentrated in the confident bars across every
bar the model calls, driving dirPrecPct toward chance by construction. A
threshold that can be selective is the only mechanism by which a small,
concentrated edge could ever clear that gate.
Also: a sparse histogram now KEEPS the previous threshold instead of
resetting to 0.0. A failed measurement must not decay to the most exposed
setting in the range.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 15:58:18 -04:00
//--- Margin histogram for the fit, rebuilt each era from the CALIBRATION slice (see
2026-08-22 00:24:45 -04:00
//--- DIR_CONF_CALIB_PCT_OF_IS).
feat: fitted directional confidence threshold - selectivity gets a mechanism
The training loss and the selection metric wanted different things and only
the second one knew it. Logit-adjusted cross-entropy has no term for "how
often should I trade", so the head calls a direction on 87-91% of bars. The
selection metric is precision x coverage credit, saturating at the coverage
floor - above the floor extra calls earn NOTHING and only precision counts.
So selection wanted few good calls, the loss produced many mediocre ones, and
all selection could do was pick the least-bad era out of what it was handed.
Nothing pushed the model toward selectivity.
This gives the decision RULE the policy instead of distorting the loss (which
is estimating class probabilities correctly, and a probability estimate should
not be bent to encode a trading policy - Elkan 2001: estimate, then choose the
operating point separately). AdjustedSignalFromSoftmax now abstains unless the
winning direction's softmax margin over its best rival clears a fitted
threshold. Margin, not the winning probability: the latter moves with overall
calibration rather than with how close the decision actually was.
Fitted on IS, applied to OOS and live. Pass 2 already forward-passes every IS
sample, so the margin histogram is harvested there for free (primary
occurrences only, so the oversampled replay queue cannot skew the operating
point); the fit runs at the end of pass 2, BEFORE pass 3, so the deploy gate
grades the thresholded model on bars the threshold never saw. Fitting on
pass 3's own predictions would be choosing the operating point on the data
being graded - the best-of-N error corrected in five other places here.
Objective: maximise IS directional precision subject to still clearing the
SAME coverage floor the deploy gate uses (base rate x 0.25, re-derived
locally so the two cannot drift apart). Swept top-down in one pass; ties go
to the LOWER threshold, since equal precision for less coverage is strictly
worse. Under DIR_CONF_MIN_FIT_CALLS (200) it runs unthresholded rather than
on a guess.
The threshold is part of the MODEL, not the run: captured with
Net.CaptureWeights(), restored with the weights at both restore sites, and
appended to the .cfg under the same length-guard convention so a deployed
model reloads at the operating point its gate actually cleared. A pre-2026-08-09
.cfg reads 0.0, which is exactly the behaviour it was trained under.
Per-era line now prints "@margin>=X.XX" next to coverage, so a coverage drop
can be attributed to the operating point rather than guessed at.
Both build variants compile 0 errors / 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:04:37 -04:00
long m_dirConfBinCalls [ DIR_CONF_THRESHOLD_BINS ] ;
long m_dirConfBinHits [ DIR_CONF_THRESHOLD_BINS ] ;
fix: the operating point was fitted on bars the net had memorized
FitDirConfThreshold harvested its margin histogram from pass 2's own
backprop samples. Pairing every fit against the same era's OOS result
shows what that measured:
PAI era 1 IS 25% cov @ 66.1% (-0.8pp) -> OOS 64% (-3pp) gap +2.1pp
PAI era 76 IS 90% cov @ 79.6% (+12.7pp) -> OOS 65% (-2pp) gap +14.6pp
LSTM era 9 IS 77% cov @ 81.6% (+14.6pp) -> OOS 63% (-4pp) gap +18.6pp
The gap grows monotonically while OOS stays flat, so within a handful of
eras the curve stops describing behaviour on unseen bars. That is fatal
here specifically, because the objective branches on the SIGN of
(p - break-even): the memorized curve reads +12pp at 95% coverage, so
coverage x (p - p0) correctly maximises coverage and returns ~0.02 - fire
on every bar. The "p < p0 -> get more selective" branch, which is the
actual regime and the entire point of 983a6a3, could never fire because IS
never showed p < p0.
Carve a calibration slice out of the IS span - DIR_CONF_CALIB_PCT_OF_IS,
purged from backprop by one label horizon on BOTH sides (the far-side
purge is not optional: without it the newest training bars carry labels
partly decided by price action inside the slice, putting the memorization
straight back into the curve). Score it in a new chunked pass 2.5, after
pass 2 has trained and before pass 3 grades - the only position where the
histogram is simultaneously not-trained-on, not-graded, and current with
the weights it will be applied to.
Costs 15% of the training data. Worth it beyond honesty: the deploy gate
needs dirPrecPct > chance + EDGE_MIN_SIGMAS*SE, and a threshold pinned
near zero dilutes any edge concentrated in the confident bars across every
bar the model calls, driving dirPrecPct toward chance by construction. A
threshold that can be selective is the only mechanism by which a small,
concentrated edge could ever clear that gate.
Also: a sparse histogram now KEEPS the previous threshold instead of
resetting to 0.0. A failed measurement must not decay to the most exposed
setting in the range.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 15:58:18 -04:00
long m_dirConfPrimaryBars ; // denominator for coverage: every calibration bar scored
feat: fitted directional confidence threshold - selectivity gets a mechanism
The training loss and the selection metric wanted different things and only
the second one knew it. Logit-adjusted cross-entropy has no term for "how
often should I trade", so the head calls a direction on 87-91% of bars. The
selection metric is precision x coverage credit, saturating at the coverage
floor - above the floor extra calls earn NOTHING and only precision counts.
So selection wanted few good calls, the loss produced many mediocre ones, and
all selection could do was pick the least-bad era out of what it was handed.
Nothing pushed the model toward selectivity.
This gives the decision RULE the policy instead of distorting the loss (which
is estimating class probabilities correctly, and a probability estimate should
not be bent to encode a trading policy - Elkan 2001: estimate, then choose the
operating point separately). AdjustedSignalFromSoftmax now abstains unless the
winning direction's softmax margin over its best rival clears a fitted
threshold. Margin, not the winning probability: the latter moves with overall
calibration rather than with how close the decision actually was.
Fitted on IS, applied to OOS and live. Pass 2 already forward-passes every IS
sample, so the margin histogram is harvested there for free (primary
occurrences only, so the oversampled replay queue cannot skew the operating
point); the fit runs at the end of pass 2, BEFORE pass 3, so the deploy gate
grades the thresholded model on bars the threshold never saw. Fitting on
pass 3's own predictions would be choosing the operating point on the data
being graded - the best-of-N error corrected in five other places here.
Objective: maximise IS directional precision subject to still clearing the
SAME coverage floor the deploy gate uses (base rate x 0.25, re-derived
locally so the two cannot drift apart). Swept top-down in one pass; ties go
to the LOWER threshold, since equal precision for less coverage is strictly
worse. Under DIR_CONF_MIN_FIT_CALLS (200) it runs unthresholded rather than
on a guess.
The threshold is part of the MODEL, not the run: captured with
Net.CaptureWeights(), restored with the weights at both restore sites, and
appended to the .cfg under the same length-guard convention so a deployed
model reloads at the operating point its gate actually cleared. A pre-2026-08-09
.cfg reads 0.0, which is exactly the behaviour it was trained under.
Per-era line now prints "@margin>=X.XX" next to coverage, so a coverage drop
can be attributed to the operating point rather than guessed at.
Both build variants compile 0 errors / 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:04:37 -04:00
//--- one-shot so the "histogram too sparse" explanation is stated once per run, not once per era
bool m_dirConfSparseWarned ;
2026-07-14 22:36:27 -04:00
//--- safety valve: Train()'s do-while loop has no other bound on how many eras it will run
//--- before giving up, so a config that can't reach the convergence objective (e.g. too few
//--- swing-confirmed examples for the min recall bar to be reachable) would otherwise loop
2026-07-17 21:28:59 -04:00
//--- forever, permanently keeping the era-progress status label up instead of the normal per-tick
2026-07-22 13:33:56 -04:00
//--- info line and burning CPU nonstop. When the cap is hit, the operator is prompted (see
//--- PromptContinuePastEraCap): CONTINUE resets the era counter and keeps training; STOP deploys
//--- the best checkpoint found so far (FinalizeTrainRun) and terminates training. Headless
//--- (tester/optimizer) runs can't prompt, so they take the STOP branch automatically.
2026-07-14 22:36:27 -04:00
int m_maxErasPerRun ;
//--- Train() runs its per-bar loop synchronously, and MQL5 is single-threaded per chart - a
//--- multi-minute era would otherwise starve the terminal's chart-event queue for that whole
//--- stretch, including the control panel's own click/drag hit-testing (Panel\ControlPanel.mqh),
2026-08-22 00:24:45 -04:00
//--- which depends entirely on CHARTEVENT_MOUSE_MOVE being delivered promptly.
2026-07-14 22:36:27 -04:00
bool m_trainRunActive ; // true: a run (schedule -> convergence/stop) is in progress, possibly spanning many Train() calls
bool m_eraResumePending ; // true: yielded mid-bar-loop last call - resume the SAME era, don't start a new one
2026-08-22 00:36:36 -04:00
//--- One writer for all five, so a yield point cannot save a partial context.
void StashEraResume ( const int bars , const int totalIter , const int oosCutoff ,
const bool add_loop , const int barIndex ) ;
2026-07-14 22:36:27 -04:00
int m_resumeBars ;
int m_resumeTotalIter ;
int m_resumeOosCutoff ;
int m_resumeBarIndex ;
bool m_resumeAddLoop ;
2026-08-22 00:24:45 -04:00
//--- Bars pass 1 queued as IS-eligible, trained on in pass 2 in a freshly shuffled order rather
//--- than pass 1's chronological one.
2026-07-18 11:33:21 -04:00
int m_isTrainQueue [ ] ;
int m_isTrainQueueCount ;
2026-08-22 00:24:45 -04:00
//--- NO PARALLEL WEIGHT/PRIMARY ARRAYS. Removed 2026-08-20; the queue is one bar per slot.
2026-07-28 17:42:12 -04:00
int m_isTrainCursor ;
//--- true: pass 1 (sequential) has finished for this era and pass 2 (shuffled backProp) is either
//--- running or has yielded mid-queue - Train() skips straight past pass 1's loop on resume when
//--- this is set. Reset to false only at a fresh era's start (never mid-run).
bool m_isPass2Active ;
//--- true: pass 2 has already run to natural completion for this era (m_isPass2Active's own
//--- false state is ambiguous between "not started yet" and "already finished" - both look
2026-08-22 00:24:45 -04:00
//--- identical to a plain `if(!m_isPass2Active)` check).
2026-07-28 17:42:12 -04:00
bool m_isPass2Done ;
//--- Pass 3: chronological, OOS-region-only re-walk that happens AFTER pass 2 has actually trained
//--- on this era's IS data - see m_isTrainQueue's declaration comment for why OOS scoring can no
//--- longer just happen inline during pass 1 (that would score every era's OOS window against
//--- weights from BEFORE this era's training, one full era stale - and for era 0 specifically,
//--- against the still-untrained cold-start network, which is why era 0's OOS recall used to show
2026-07-18 12:10:37 -04:00
//--- a meaningless 100% Neutral / 0% Buy / 0% Sell every time). Cursor walks i downward from
//--- m_oosScoreStartIndex to 0, mirroring pass 1's own iteration bounds/order for whichever bars
//--- satisfy isOOS - order matters here (unlike pass 2) since dOosForecast/dOosError are recursive
//--- EMAs over the visitation sequence, not order-independent.
bool m_isPass3Active ;
int m_oosScoreIndex ;
int m_oosScoreStartIndex ;
2026-08-22 00:24:45 -04:00
//--- Pass 2.5: the CALIBRATION walk.
fix: the operating point was fitted on bars the net had memorized
FitDirConfThreshold harvested its margin histogram from pass 2's own
backprop samples. Pairing every fit against the same era's OOS result
shows what that measured:
PAI era 1 IS 25% cov @ 66.1% (-0.8pp) -> OOS 64% (-3pp) gap +2.1pp
PAI era 76 IS 90% cov @ 79.6% (+12.7pp) -> OOS 65% (-2pp) gap +14.6pp
LSTM era 9 IS 77% cov @ 81.6% (+14.6pp) -> OOS 63% (-4pp) gap +18.6pp
The gap grows monotonically while OOS stays flat, so within a handful of
eras the curve stops describing behaviour on unseen bars. That is fatal
here specifically, because the objective branches on the SIGN of
(p - break-even): the memorized curve reads +12pp at 95% coverage, so
coverage x (p - p0) correctly maximises coverage and returns ~0.02 - fire
on every bar. The "p < p0 -> get more selective" branch, which is the
actual regime and the entire point of 983a6a3, could never fire because IS
never showed p < p0.
Carve a calibration slice out of the IS span - DIR_CONF_CALIB_PCT_OF_IS,
purged from backprop by one label horizon on BOTH sides (the far-side
purge is not optional: without it the newest training bars carry labels
partly decided by price action inside the slice, putting the memorization
straight back into the curve). Score it in a new chunked pass 2.5, after
pass 2 has trained and before pass 3 grades - the only position where the
histogram is simultaneously not-trained-on, not-graded, and current with
the weights it will be applied to.
Costs 15% of the training data. Worth it beyond honesty: the deploy gate
needs dirPrecPct > chance + EDGE_MIN_SIGMAS*SE, and a threshold pinned
near zero dilutes any edge concentrated in the confident bars across every
bar the model calls, driving dirPrecPct toward chance by construction. A
threshold that can be selective is the only mechanism by which a small,
concentrated edge could ever clear that gate.
Also: a sparse histogram now KEEPS the previous threshold instead of
resetting to 0.0. A failed measurement must not decay to the most exposed
setting in the range.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 15:58:18 -04:00
bool m_isCalibActive ;
bool m_isCalibDone ;
int m_calibIndex ;
int m_calibStartIndex ;
2026-07-14 22:36:27 -04:00
datetime m_lastBarTime ;
2026-08-22 00:24:45 -04:00
//--- This model's own learning-rate trajectory. g_eta is one file-scope global shared by every
//--- CNet in the process, so one member's era-end decay silently changed the rate the OTHER
//--- members' next backProp() used - an unintended coupling between independent trajectories.
2026-07-17 23:21:12 -04:00
double m_modelEta ;
2026-08-22 00:24:45 -04:00
//--- Ceiling the era-end recovery bump (Train()'s isBetterEra block) restores `g_eta` toward -
//--- used to be the raw AdamLearningRate unconditionally, which is only correct for ADAM.
2026-07-18 14:56:41 -04:00
double m_etaCeiling ;
2026-07-14 22:36:27 -04:00
int m_erasSinceCooldown ; // eras completed since the last cooldown reset - replaces the old per-call-only "erasThisCall"
CArrayDouble m_oosWindow ; // run-scoped OOS stability window (used to be a Train()-local CArrayDouble)
double m_bestOosForecast ;
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- Selection score (coverage-weighted directional precision) of the era the current checkpoint
//--- was taken from.
double m_bestSelectionScore ;
//--- whether the era m_bestOosForecast/the checkpoint was taken from also cleared the gate's
//--- deployability floor - part of the "best" ranking itself, not just a side note, so blended
//--- accuracy alone can never outrank a deployable era.
2026-07-15 21:47:37 -04:00
bool m_bestPassedRecall ;
fix: a one-sided era can no longer become the best checkpoint
Measured on HYBRID, era 29 of the first win-scored run: the model collapsed
to always-Buy and was crowned "new best selection score 67.1%". Under
win-based scoring that is not a coincidence - the always-call-the-drift-side
model IS the chance reference, so it scores exactly chance (P(winLong) ~ 67%
on SP500), while every honest two-sided era scores 63-66% because shorts win
less often against the drift. Raw score ranking therefore actively prefers
the degenerate model, every regression restores back to it, and live NMS
collapses its near-constant signal to ~25 trades per era - observed as
"hybrid barely trades".
bothSidesLive already blocked one-sided eras from DEPLOYING (tradeableOK,
371f8aa), but among not-yet-deployable eras the score alone ranked - the same
early phase the coverage credit was added for, failing the same way through a
different door.
The ranking key is now three lexicographic tiers: deployable > two-sided >
score. A one-sided era cannot displace a two-sided best regardless of score -
by construction its score is a property of the data's drift, not the model -
and a two-sided era displaces a one-sided best no matter how much lower it
scores. m_bestBothSidesLive is snapshotted with the checkpoint and reset with
the rest of the best-tracking state.
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 21:19:44 -04:00
//--- Was the checkpointed era calling BOTH directions? Middle tier of the ranking key - see
2026-08-22 00:24:45 -04:00
//--- isBetterEra.
fix: a one-sided era can no longer become the best checkpoint
Measured on HYBRID, era 29 of the first win-scored run: the model collapsed
to always-Buy and was crowned "new best selection score 67.1%". Under
win-based scoring that is not a coincidence - the always-call-the-drift-side
model IS the chance reference, so it scores exactly chance (P(winLong) ~ 67%
on SP500), while every honest two-sided era scores 63-66% because shorts win
less often against the drift. Raw score ranking therefore actively prefers
the degenerate model, every regression restores back to it, and live NMS
collapses its near-constant signal to ~25 trades per era - observed as
"hybrid barely trades".
bothSidesLive already blocked one-sided eras from DEPLOYING (tradeableOK,
371f8aa), but among not-yet-deployable eras the score alone ranked - the same
early phase the coverage credit was added for, failing the same way through a
different door.
The ranking key is now three lexicographic tiers: deployable > two-sided >
score. A one-sided era cannot displace a two-sided best regardless of score -
by construction its score is a property of the data's drift, not the model -
and a two-sided era displaces a one-sided best no matter how much lower it
scores. m_bestBothSidesLive is snapshotted with the checkpoint and reset with
the rest of the best-tracking state.
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 21:19:44 -04:00
bool m_bestBothSidesLive ;
2026-08-22 00:24:45 -04:00
//--- SLOW-ERA HEARTBEAT (2026-08-10).
diag: slow eras must explain themselves - heartbeat + era time split + pass-1 paint
The 23:42 restart left all four charts grinding ~25x slower than the 18:01
baseline (era lines in 86 seconds there; 20+ minutes of nothing here), and
NOTHING could say why from outside: pass 1 logs nothing, its status paint sat
inside the !wouldQueue branch so the IS sweep - 80% of the pass, processed
FIRST - painted nothing either, the VPS has no debugger for a thread stack,
and the hourly new-bar cache invalidation cancels and restarts an unfinished
era, so a slow era can stay invisible FOREVER. Externals gave: four chart
threads at ~95% pure user-mode compute, DLL pool idle, no file writes. That
narrows it to "MQL5-side per-item work in the era passes" and no further.
So training now explains itself:
- TrainHeartbeat: one line per 4096 processed items, only after an era has
already run 60s, at most 6 lines per era - a healthy era stays exactly as
quiet as before. Reports position and the cumulative split: feature-window
builds vs net forward/backprop vs everything else. Hooked into all three
passes.
- The era summary line gains "| ERA TOOK Ns (feature windows X, net fwd/back
Y, other Z)" whenever an era exceeded 120s.
- Pass 1 paints its progress for QUEUED bars too, not just the OOS slice, so
the panel shows "learning (era N)" instead of sitting on the idle writer's
"Getting ready..." for the entire IS sweep. The label is throttled
internally; painting per bar costs nothing.
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 00:10:45 -04:00
uint m_eraStartTick ;
ulong m_passFeatUs ; // cumulative BuildFeatureWindow time this era, microseconds
ulong m_passNetUs ; // cumulative feedForward/backProp time this era, microseconds
int m_passHeartbeatPrints ;
2026-08-10 07:44:03 -04:00
uint m_lastHeartbeatTick ;
2026-08-22 00:24:45 -04:00
//--- How many of pass 1's bars produced a usable feature window, and how many did not.
diag: an era that discards itself now says so instead of scanning forever
add_loop is exactly "at least one bar produced a usable feature
window". When it stays false, pass 2, pass 3, the era counter, the
checkpoint and every log line in the era-end block are ALL skipped:
Train() returns having done nothing, m_eraResumePending is still false,
and the next call restarts the SAME era from bar 0. That is an
infinite 0->100% "scan" loop that prints absolutely nothing - the only
remaining silent restart path in Train(), and it matches the reported
symptom exactly.
Pass 1 now counts usable vs unusable windows and reports at the pass
boundary, which demonstrably executes:
- total failure routes through ReportTrainStall (already capped at
one line a minute, and carries the run-state flags) naming the
counts, the required window width and the bar count
- success prints how long the scan took and how many samples it
handed to pass 2, but only once the era has passed 10s - a fast
era stays as quiet as before, a slow one distinguishes "advancing"
from "sweeping the same bars forever"
A PARTIAL failure is normal and deliberately does not shout: pass 1
walks oldest-to-newest and the deepest bars predate the indicators'
warm-up, so those windows fail and are cached as misses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:29:33 -04:00
int m_passWindowOk ;
int m_passWindowFail ;
fix: the recall gate was unsatisfiable and the LR decay was a spiral
Both made the run structurally unable to succeed, independently of any
signal in the data. Found by reading the 13:01 log.
RECALL GATE. m_objectiveMet required Buy, Sell AND Neutral OOS recall
each >= 40%. First-touch resolution (ce52654) collapsed Neutral from
the ~94% majority it was under exact-pivot labels to a same-bar-tie
residue - 250 of 38,261 bars, 0.65% - so the floor was asking the model
to identify 40% of coin-flip ties before it could converge. Measured:
CONV, LSTM and HYBRID all logged "Neutral:0% (need >=40% each)" on
every era. No model could ever satisfy it; every run was destined for
the plateau ladder or the era cap.
Only the DIRECTIONAL floors are load-bearing for the anti-collapse job
the gate exists to do: an all-Neutral model shows Buy and Sell recall
at 0% and is blocked by them. Neutral's own floor guarded the mirror
bias (over-calling Buy/Sell at Neutral's expense), which was real at
94% prevalence and is not at 0.65% - there, almost never calling
Neutral is correct rather than biased.
Prevalence-guarded rather than hardcoded off, so it returns by itself
if a future label rule makes Neutral substantial again. Deliberately
NOT extended to Buy/Sell: exempting a thin directional class reopens
the era-44-46 hole, which directionalRecallMeasured only half-covers -
it checks those classes were MEASURED, not that they passed.
ETA DECAY. A regressing era restored the checkpoint, reset the
optimizer and cut eta - all on the FIRST regression. The next era then
started from an identical state with a smaller step, regressed again,
and got the same treatment. The loop is self-sustaining and cannot
discover anything, because rolling the weights back is exactly what
removes the exploration that would end it.
Measured on PAI: eras 2-11 every one a regression against era 1, eta
0.000594 -> 0.000024, dW/W 0.000%/0.000% from era 2 onward. Ten eras,
~45s each, reproducing era 1 exactly and unable to do anything else.
Now requires ETA_DECAY_PATIENCE_ERAS consecutive regressions - the
standard ReduceLROnPlateau formulation. A single bad era is noise, and
an improving era clears the counter so alternating runs never
accumulate into a decay.
Build tag -> gate-patience-v3. It had not moved in six commits, which
is why the running binary could not be identified from its own log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 13:28:58 -04:00
//--- Consecutive regressing eras since the last new best - the patience counter for the checkpoint
2026-08-20 07:00:09 -04:00
//--- restore / g_eta decay (see ETA_DECAY_PATIENCE_ERAS). Reset by any era that improves.
fix: the recall gate was unsatisfiable and the LR decay was a spiral
Both made the run structurally unable to succeed, independently of any
signal in the data. Found by reading the 13:01 log.
RECALL GATE. m_objectiveMet required Buy, Sell AND Neutral OOS recall
each >= 40%. First-touch resolution (ce52654) collapsed Neutral from
the ~94% majority it was under exact-pivot labels to a same-bar-tie
residue - 250 of 38,261 bars, 0.65% - so the floor was asking the model
to identify 40% of coin-flip ties before it could converge. Measured:
CONV, LSTM and HYBRID all logged "Neutral:0% (need >=40% each)" on
every era. No model could ever satisfy it; every run was destined for
the plateau ladder or the era cap.
Only the DIRECTIONAL floors are load-bearing for the anti-collapse job
the gate exists to do: an all-Neutral model shows Buy and Sell recall
at 0% and is blocked by them. Neutral's own floor guarded the mirror
bias (over-calling Buy/Sell at Neutral's expense), which was real at
94% prevalence and is not at 0.65% - there, almost never calling
Neutral is correct rather than biased.
Prevalence-guarded rather than hardcoded off, so it returns by itself
if a future label rule makes Neutral substantial again. Deliberately
NOT extended to Buy/Sell: exempting a thin directional class reopens
the era-44-46 hole, which directionalRecallMeasured only half-covers -
it checks those classes were MEASURED, not that they passed.
ETA DECAY. A regressing era restored the checkpoint, reset the
optimizer and cut eta - all on the FIRST regression. The next era then
started from an identical state with a smaller step, regressed again,
and got the same treatment. The loop is self-sustaining and cannot
discover anything, because rolling the weights back is exactly what
removes the exploration that would end it.
Measured on PAI: eras 2-11 every one a regression against era 1, eta
0.000594 -> 0.000024, dW/W 0.000%/0.000% from era 2 onward. Ten eras,
~45s each, reproducing era 1 exactly and unable to do anything else.
Now requires ETA_DECAY_PATIENCE_ERAS consecutive regressions - the
standard ReduceLROnPlateau formulation. A single bad era is noise, and
an improving era clears the counter so alternating runs never
accumulate into a decay.
Build tag -> gate-patience-v3. It had not moved in six commits, which
is why the running binary could not be identified from its own log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 13:28:58 -04:00
int m_consecutiveRegressions ;
2026-08-10 07:22:29 -04:00
void TrainHeartbeat ( const string tag , int done , int total , const string shortLabel ) ;
2026-08-22 00:24:45 -04:00
//--- Progress of the pass currently running, and its name, for the simple panel.
2026-08-10 07:22:29 -04:00
int m_passProgressPct ;
string m_passLabel ;
fix: prebuild and era sized different windows; diag: Train() names its branch
TWO things, one incident.
1) THE BUG I SHIPPED IN 0c85c54. m_tuneStartTrainBar is declared, initialised
to 0, and NEVER ASSIGNED - the assignment existed before the God-class split
and the split dropped it, leaving a dead member. Harmless while nothing read
it; a real defect the moment 0c85c54 made StartLabelCachePrebuild() reset
dtStudied from it. Train() then computed the window as
max(StartTrainBar, floor) while the prebuild computed max(0, floor), where
StartTrainBar is the non-zero datetime OnChartEventHandler passes through from
the "New Bar" event. The two therefore disagreed about `bars`, so
EnsureBarCachesCapacity() saw a changed size at era start, wiped the caches,
and re-armed a full 38k-bar prebuild - instead of training. Restored the
assignment so both sides evaluate the identical expression.
2) THE REASON IT TOOK ALL NIGHT TO FIND. Train() is a state machine with six
early-return branches above the era loop and every one of them is silent. Four
charts burned a core each for 15 minutes with an empty journal: the pass
heartbeats (694b756) proved the era loop was never reached, no prebuild
completion line appeared either, and nothing external can see inside a single
MQL5 thread - per-thread CPU says "busy", file writes say nothing, and the VPS
has no debugger. That is an undiagnosable state, and it is the thing to fix,
not just the bug of the day.
ReportTrainStall() now names the branch Train() is taking whenever no era has
completed for 3 minutes, at most once a minute per signal, with the state that
decides the branch: run/prebuild/simOos/resume flags, era, dtStudied, and -
for the cache-invalidation branch specifically - BOTH bar counts, since two
sizings disagreeing is exactly what re-arms the prebuild forever. Silent on a
healthy run: an era completing resets the clock.
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 07:32:08 -04:00
//--- STALL REPORTER. Train() is a state machine with several early-return branches ABOVE the era
//--- loop (OOS simulation walk, label prebuild, history sync, warm-up, cache invalidation), and
2026-08-22 00:24:45 -04:00
//--- every one of them is silent.
fix: prebuild and era sized different windows; diag: Train() names its branch
TWO things, one incident.
1) THE BUG I SHIPPED IN 0c85c54. m_tuneStartTrainBar is declared, initialised
to 0, and NEVER ASSIGNED - the assignment existed before the God-class split
and the split dropped it, leaving a dead member. Harmless while nothing read
it; a real defect the moment 0c85c54 made StartLabelCachePrebuild() reset
dtStudied from it. Train() then computed the window as
max(StartTrainBar, floor) while the prebuild computed max(0, floor), where
StartTrainBar is the non-zero datetime OnChartEventHandler passes through from
the "New Bar" event. The two therefore disagreed about `bars`, so
EnsureBarCachesCapacity() saw a changed size at era start, wiped the caches,
and re-armed a full 38k-bar prebuild - instead of training. Restored the
assignment so both sides evaluate the identical expression.
2) THE REASON IT TOOK ALL NIGHT TO FIND. Train() is a state machine with six
early-return branches above the era loop and every one of them is silent. Four
charts burned a core each for 15 minutes with an empty journal: the pass
heartbeats (694b756) proved the era loop was never reached, no prebuild
completion line appeared either, and nothing external can see inside a single
MQL5 thread - per-thread CPU says "busy", file writes say nothing, and the VPS
has no debugger. That is an undiagnosable state, and it is the thing to fix,
not just the bug of the day.
ReportTrainStall() now names the branch Train() is taking whenever no era has
completed for 3 minutes, at most once a minute per signal, with the state that
decides the branch: run/prebuild/simOos/resume flags, era, dtStudied, and -
for the cache-invalidation branch specifically - BOTH bar counts, since two
sizings disagreeing is exactly what re-arms the prebuild forever. Silent on a
healthy run: an era completing resets the clock.
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 07:32:08 -04:00
uint m_lastEraCompleteTick ;
uint m_lastStallReportTick ;
void ReportTrainStall ( const string branch ) ;
diag(training): report the window an era ACTUALLY trains on
With VerboseMode on, pass 1 reported eras of 422 / 949 / 1358 / 2562 bars on
SP500 H4 - four models, same chart, same second - against a series holding
~16,264 bars, and the number moved every era (CONV: 2562, 3671, 3405, 3532,
2830). Nothing in the journal said so. ReportDetectability and the CAPACITY
line both quote EstimatedInSampleBars, which is derived from the configuration
and not from the era, so they kept reporting "11385 in-sample rows / OOS window
4874 bars" for a window that was a tenth of that.
era.bars is MathMin(Bars(symbol, PERIOD_CURRENT, dtStudied, now) + historyBars,
Bars(symbol, PERIOD_CURRENT)). A short era is therefore either a dtStudied that
is too recent or a short price series, and those need opposite fixes - so the
new line carries all three quantities plus the resolved dtStudied and
SERIES_FIRSTDATE, not just the result.
Reported on change only: an era over a warm feature cache runs in a fraction of
a second here, and a per-era line would bury the journal.
Diagnostic only - no training behaviour is changed by this commit.
Compile-verified in the staging copy: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 15:12:05 -04:00
//--- THE SIZE OF THE WINDOW AN ERA ACTUALLY TRAINS ON, which nothing reported until 2026-08-24.
//--- ReportDetectability and the CAPACITY line both quote EstimatedInSampleBars - derived from the
//--- configuration, not from the era - so a window that collapses to a few hundred bars is invisible
//--- while every surrounding diagnostic keeps quoting the full history. Reported on CHANGE, because
//--- an era over a warm feature cache can finish in a fraction of a second.
int m_lastEraWindowBars ;
void ReportEraWindow ( const int barsNow ) ;
refactor(training): the era log is not part of the era loop
Train() was 2,572 lines in one function. The largest single block in it
was ~200 lines of string building for the console line, reachable only
because twenty-one loose ints were declared at the top of the function
and read nine hundred lines later. Those declarations were the reason
the block could not move.
Introduce SEraTelemetry - one parameter object holding exactly those
twenty-one numbers, self-initialising to -1 ("not measured this era",
which is what era 0 and any stopped era report, and is not the same as
a measured zero). ReportEraProgress() takes it and renders it, guarding
on its own shouldLog so the call site is one unconditional line rather
than a 200-line branch.
Nothing is decided or measured in the moved code - it reads state and
prints. Verified by stripping comments and whitespace from both
revisions and diffing the remaining statements: the only differences
are the ten declarations collapsing into one object, the throttle test
moving inside the callee, and the new signature plus its call. Every
other statement is byte-identical.
Train() 2,572 -> 2,370 lines.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:25:07 -04:00
//--- The era's console line and panel refresh. Self-guarding: it decides nothing, measures
//--- nothing and returns immediately when this era had nothing to report, so the era loop no
//--- longer has to carry ~200 lines of string building through its own control flow.
void ReportEraProgress ( const SEraTelemetry & tel ) ;
refactor(training): each of an era's four passes is its own method
Train() was still 2,572 lines because the four passes it runs - the
scan/queue sweep, the shuffled replay, the purged calibration walk and
the OOS scoring walk - were written inline as four consecutive blocks
sharing one scope. STrainEra removed the only obstacle to moving them.
Each pass now keeps its own guard inside its own body, so it is
self-contained: RunPass2 still tests !era.stop && era.addLoop &&
!m_isPass2Done itself rather than being called conditionally. Train()
reads as the sequence it always was.
RunOosPass MEASURES and nothing more - the recall gate, plateau ladder,
deploy gate and era checkpoint read its numbers afterwards and stay in
Train(). The first version of its header comment claimed it ran those
too; the body is 504 lines and does not, so the comment was corrected
rather than shipped.
Verified by stripping comments and whitespace from both revisions:
ZERO statements removed, sixteen added - four signatures, their eight
braces and four call sites. Every other statement byte-identical.
Train() 2,572 -> 1,282 lines.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:30:31 -04:00
//--- THE FOUR PASSES OF ONE ERA, in the order Train() runs them. Each one guards its own
//--- precondition and yields on the shared wall-clock budget, so Train() is the sequence and
//--- these are the steps - which is the whole point of STrainEra existing.
void RunPass1 ( STrainEra & era ) ;
void RunPass2 ( STrainEra & era ) ;
void RunCalibrationPass ( STrainEra & era ) ;
void RunOosPass ( STrainEra & era ) ;
refactor(train): Train() is the era lifecycle again, not the whole of it
Train() was 1,273 lines. It is now 79, of which about 35 are statements, and
they read as what the function is: preempt, begin run, begin era, four
passes, advance, complete, report, finalize.
Seven methods carry what left it:
TrainCallPreempted 107 six ways this call is not a training call at all
BeginTrainRun 130 once per run - history sync, window, one-shot walks
BeginEra 232 once per era, or resume a chunk that yielded
ReportPass1Outcome 105 what pass 1 found, said out loud
AdvanceEra 68 count the era, decide whether the RUN ends
CompleteEra 590 calibrate, gate, rank, checkpoint, ladders, persist
ReportBarrierHold 62 why this member is idle at the era barrier
ClaimCallForWalk 15 the preamble the three exclusive walks shared
TWO DRY FIXES fell out rather than being looked for. The three exclusive
walks each had to tell TWO watchdogs the same thing - the stall reporter
which branch is running, the era-barrier watchdog that this member is BUSY
rather than stuck - written out three times, so a fourth walk was three
chances to be added with only one of them. And the barrier-hold reporting
was 44 lines inline in a branch whose only other statement was resetting a
tick.
CompleteEra is lifted WHOLE and stays that way for now. Its parts share
thirty-odd locals - the recalls, the gate verdict, the better/worse flags -
and threading those through three signatures would recreate exactly the
eight-locals-across-four-passes problem STrainEra was built to end.
Splitting it needs an era-outcome object first, not more parameters.
VERIFIED AS A PURE MOVE: statement multisets, old file vs new, differ only
by the 14 `return;` that became 17 `return true;` plus 3 new returns at the
call sites, the 3 collapsed walk preambles, the 8 new signatures and their
braces. Nothing else moved, and every function closes at depth 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 14:50:59 -04:00
//--- BEFORE ANY OF THAT: does this call belong to training at all? Paused, stopping, deploying,
//--- held at the ensemble era barrier, or occupied by one of the three exclusive walks. True =
//--- Train() is done for this call. None of it is training, which is exactly why it is not in
//--- Train() any more.
bool TrainCallPreempted ( STrainEra & era ) ;
//--- Everything an era does after its last pass scores: calibrate, gate, rank, checkpoint,
//--- advance the ladders, persist. Lifted whole because its parts share thirty-odd locals -
//--- splitting it further needs an era-outcome object first, not more parameters.
void CompleteEra ( STrainEra & era , SEraTelemetry & tel ) ;
//--- THE TWO SETUPS, on the same contract as TrainCallPreempted: true = this call is spent.
//--- Both defer rather than block - a branch that cannot proceed declines the call and lets the
//--- next scheduled one try, so the chart's single thread is never slept.
fix(train): BeginTrainRun read Train()'s parameter from a scope it no longer had
The run-start block calls TrainWindowStart(StartTrainBar), and StartTrainBar
is Train()'s parameter. Moving the block into its own method left the read
behind. Now passed explicitly.
THIRD TIME THIS FAMILY HAS BILLED THIS SESSION, and the third distinct
sub-shape:
1d7ebbd a DELETED loop's variable still read by its body
d7469c6 a RENAMED field still read by its call site
here a MOVED block still reading its old enclosing scope
Same root cause each time: I verify the side I edited. What I had been
checking - statement multisets, brace balance, field-name resolution - all
passed, because none of them models SCOPE. The move was faithful; the
scope was not.
So scope is now checked too. For every CExpertSignalAIBase::Method, collect
the identifiers its body reads and subtract what can actually resolve:
names declared in the body (any type, and every name in a multi-declarator),
the method's own parameters, class members, file-scope globals and #defines.
Parameter names from OTHER declarations must NOT count as resolvable - that
is the bug in the first version of this check, which let StartTrainBar
through because Train() declares it in the header.
Validated against the broken commit before being trusted: it reports
StartTrainBar there and not here. The only residual output is MQL5 enum
members and EA inputs declared outside the scanned headers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 14:58:35 -04:00
bool BeginTrainRun ( STrainEra & era , const datetime startTrainBar ) ; // once per run
refactor(train): Train() is the era lifecycle again, not the whole of it
Train() was 1,273 lines. It is now 79, of which about 35 are statements, and
they read as what the function is: preempt, begin run, begin era, four
passes, advance, complete, report, finalize.
Seven methods carry what left it:
TrainCallPreempted 107 six ways this call is not a training call at all
BeginTrainRun 130 once per run - history sync, window, one-shot walks
BeginEra 232 once per era, or resume a chunk that yielded
ReportPass1Outcome 105 what pass 1 found, said out loud
AdvanceEra 68 count the era, decide whether the RUN ends
CompleteEra 590 calibrate, gate, rank, checkpoint, ladders, persist
ReportBarrierHold 62 why this member is idle at the era barrier
ClaimCallForWalk 15 the preamble the three exclusive walks shared
TWO DRY FIXES fell out rather than being looked for. The three exclusive
walks each had to tell TWO watchdogs the same thing - the stall reporter
which branch is running, the era-barrier watchdog that this member is BUSY
rather than stuck - written out three times, so a fourth walk was three
chances to be added with only one of them. And the barrier-hold reporting
was 44 lines inline in a branch whose only other statement was resetting a
tick.
CompleteEra is lifted WHOLE and stays that way for now. Its parts share
thirty-odd locals - the recalls, the gate verdict, the better/worse flags -
and threading those through three signatures would recreate exactly the
eight-locals-across-four-passes problem STrainEra was built to end.
Splitting it needs an era-outcome object first, not more parameters.
VERIFIED AS A PURE MOVE: statement multisets, old file vs new, differ only
by the 14 `return;` that became 17 `return true;` plus 3 new returns at the
call sites, the 3 collapsed walk preambles, the 8 new signatures and their
braces. Nothing else moved, and every function closes at depth 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 14:50:59 -04:00
bool BeginEra ( STrainEra & era ) ; // once per era, or resume a yielded chunk
//--- What pass 1 found, said out loud. Reporting only; self-guarding on era.stop.
void ReportPass1Outcome ( STrainEra & era ) ;
//--- The era completed: count it, age the shadow net, and decide whether the RUN ends here
//--- (plateau ladder / ensemble gate, or the operator's answer at the era cap).
void AdvanceEra ( STrainEra & era , SEraTelemetry & tel ) ;
//--- Says WHY this member is idle at the barrier, on the report cadence rather than on entry -
//--- a brief hold every era is the design, and printing on entry logged ~950 lines/member/day.
void ReportBarrierHold ( void ) ;
//--- The three exclusive walks each take a whole call. Shared preamble: tell the stall watchdog
//--- which branch is running and the era-barrier watchdog that this member is BUSY, not stuck.
void ClaimCallForWalk ( const string branch ) ;
2026-07-14 22:36:27 -04:00
bool m_haveOosCheckpoint ;
bool m_oosStable ;
bool m_objectiveMet ;
feat: gate deployment on the null of the MAXIMUM, not the per-era null
EDGE_MIN_SIGMAS is a PER-ERA test and the deployed model is the MAXIMUM
over every era a run ranks. A 2-sigma one-sided test passes on noise with
probability 0.0228 per era, so over N eras the chance at least one clears
it is 1-(1-0.0228)^N: 34% by era 18, 80% by era 70, 93% by era 112. The
gate was near-certain to open on a long run whatever the data held.
It did. HYBRID deployed 2026-08-08 at dir-precision 35.5% vs 34% chance -
+1.5pp, best of 112 eras whose per-era values wandered 30%..35.5%. At the
call counts these runs produce that is p_family 0.92..0.9999.
Every OTHER best-of-N decision here already carries this correction, and
every one REJECTS on this data: the barrier-geometry winner (null of the
maximum over 6, p=0.3902), the indicator tuner (Sidak, p=1.0000), the MI
lag profile (null of the maximum over 21 lags). The one decision that
ships a model to a live account had none.
BestCheckpointSurvivesSelection() re-tests the checkpoint that is about to
deploy:
z = (precision - chance)/SE, SE = sqrt(p0(1-p0)/n)
p_single = P(Z >= z)
p_family = 1 - (1-p_single)^N
against DEPLOY_FAMILY_WISE_ALPHA. It uses the checkpoint's OWN
snapshotted precision/chance/call-count, not the latest era's, because
the model that ships is the one that has to clear the bar.
N counts CANDIDATE eras (coverage measurable, at least one directional
call) - an era that called nothing directional could never have become
the best, so counting it would make the gate stricter than the search
that actually happened.
Conservative on purpose: consecutive eras share OOS bars and differ by
one gradient step, so they are nowhere near N independent draws and the
true family-wise error is below this bound. This gate decides what trades
real money and the house posture is reject-unless-demonstrated.
Effect at 2900 directional calls / N=112: required edge goes 1.76pp ->
2.92pp. A real edge clears it; +1.5pp does not.
Applied to BOTH automatic paths - the plateau ladder's stage-3 deploy and
the m_trainingComplete assignment - which must stay identical or the flag
persisted into the .nnw disagrees with the decision to stop, and a reload
runs inference on a model the ladder refused.
NOT applied to the two operator paths (era-cap deploy, panel Deploy
button). Those stay the operator's call; ReportSelectionGateVerdict()
logs the verdict beside them so an authorised deploy can never later be
misread as a validated one.
NormalUpperTail() is A&S 26.2.17 (|err| < 7.5e-8), self-contained rather
than pulling in Math\Stat. Verified against reference values to 6dp:
Q(1.645)=0.049985, Q(1.96)=0.024998, Q(3.0)=0.001350. Its locals are
ntB1..ntB5 because AI\Network.mqh line 79 does "#define b1 AdamBeta1".
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 18:01:04 -04:00
//--- RAW inputs to the family-wise deployment gate, snapshotted at the same instant as the checkpoint
//--- so the test re-runs on the era that will actually ship rather than on whatever the latest era
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- happened to score. m_bestSelectionScore alone cannot serve: it is precision already multiplied by
feat: gate deployment on the null of the MAXIMUM, not the per-era null
EDGE_MIN_SIGMAS is a PER-ERA test and the deployed model is the MAXIMUM
over every era a run ranks. A 2-sigma one-sided test passes on noise with
probability 0.0228 per era, so over N eras the chance at least one clears
it is 1-(1-0.0228)^N: 34% by era 18, 80% by era 70, 93% by era 112. The
gate was near-certain to open on a long run whatever the data held.
It did. HYBRID deployed 2026-08-08 at dir-precision 35.5% vs 34% chance -
+1.5pp, best of 112 eras whose per-era values wandered 30%..35.5%. At the
call counts these runs produce that is p_family 0.92..0.9999.
Every OTHER best-of-N decision here already carries this correction, and
every one REJECTS on this data: the barrier-geometry winner (null of the
maximum over 6, p=0.3902), the indicator tuner (Sidak, p=1.0000), the MI
lag profile (null of the maximum over 21 lags). The one decision that
ships a model to a live account had none.
BestCheckpointSurvivesSelection() re-tests the checkpoint that is about to
deploy:
z = (precision - chance)/SE, SE = sqrt(p0(1-p0)/n)
p_single = P(Z >= z)
p_family = 1 - (1-p_single)^N
against DEPLOY_FAMILY_WISE_ALPHA. It uses the checkpoint's OWN
snapshotted precision/chance/call-count, not the latest era's, because
the model that ships is the one that has to clear the bar.
N counts CANDIDATE eras (coverage measurable, at least one directional
call) - an era that called nothing directional could never have become
the best, so counting it would make the gate stricter than the search
that actually happened.
Conservative on purpose: consecutive eras share OOS bars and differ by
one gradient step, so they are nowhere near N independent draws and the
true family-wise error is below this bound. This gate decides what trades
real money and the house posture is reject-unless-demonstrated.
Effect at 2900 directional calls / N=112: required edge goes 1.76pp ->
2.92pp. A real edge clears it; +1.5pp does not.
Applied to BOTH automatic paths - the plateau ladder's stage-3 deploy and
the m_trainingComplete assignment - which must stay identical or the flag
persisted into the .nnw disagrees with the decision to stop, and a reload
runs inference on a model the ladder refused.
NOT applied to the two operator paths (era-cap deploy, panel Deploy
button). Those stay the operator's call; ReportSelectionGateVerdict()
logs the verdict beside them so an authorised deploy can never later be
misread as a validated one.
NormalUpperTail() is A&S 26.2.17 (|err| < 7.5e-8), self-contained rather
than pulling in Math\Stat. Verified against reference values to 6dp:
Q(1.645)=0.049985, Q(1.96)=0.024998, Q(3.0)=0.001350. Its locals are
ntB1..ntB5 because AI\Network.mqh line 79 does "#define b1 AdamBeta1".
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 18:01:04 -04:00
//--- the coverage credit, and the significance test needs the unweighted precision, the chance rate it
//--- is measured against, and the call count that sets its standard error. -1 until the first ranked era.
double m_bestDirPrecPct ;
double m_bestChancePrecPct ;
int m_bestDirCalls ;
2026-08-22 00:24:45 -04:00
//--- DECLUSTERED OOS tally: the calls that survive NMS, i.e. the ones that actually become
//--- positions now that live NMS gates the trade (see RefreshLatestSignal). Era-scoped, reset
//--- with the rest of the OOS counters.
fix: NMS gates the TRADE, not just the arrow - one arrow is now one trade
NmsLiveAccept() appeared in exactly one place: wrapped around DrawObject().
It never touched dPrevSignal, and dPrevSignal is what LongCondition() /
ShortCondition() / SignedAIConfidence() read. So a declustered bar lost its
arrow and still opened a position.
Measured on SP500 H1 2026-08-09: CONV called a direction on 64% of bars,
so the ~500 bars visible on screen held ~320 decisions - and ~40 arrows
were drawn. Roughly one arrow per eight positions the EA would take.
And the survivors are not a random eighth. Rule 2 of the declustering
keeps the HIGHER-CONFIDENCE side of a cluster, so the visible set is
systematically the best member of each run. A chart showing the best of
every eight decisions and hiding the rest reads far better than the model
is - the same best-of-N selection error already corrected in the geometry
scan, the indicator tuner, the lag profile and the deploy gate, this time
on the display layer, where it is most likely to mislead the person
deciding whether to trade.
Fixed by neutralising dPrevSignal when NMS rejects, rather than adding a
"may trade" flag consulted at each read site: that leaves exactly ONE
definition of what the model decided this bar, so the arrow, the panel's
"Current signal", the confidence feeding sizing/SL/TP/trailing, the
refresh tally and the order itself cannot drift apart again.
Also reports the consequence instead of hiding it. Every OOS counter on
the era line still scores every directional call - a population ~8x larger
than what now trades - so the line carries a second figure:
| TRADED (declustered) NN% on N calls (edge +Npp)
replaying the identical rule over pass 3 (which walks OOS bars oldest to
newest, the same order the live sweep sees). Its cursors are separate
members from the live ones so a training pass can never disturb the live
chart's declustering.
Deliberately NOT switched into selectionScore yet. Declustering cuts
coverage from ~64% of bars to ~8%, well under
MIN_COVERAGE_FRACTION_OF_BASE_RATE, which would make every checkpoint
undeployable overnight - the minRR collision and the recall-floor catch-22
twice over. The floor gets re-derived from these measurements first.
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-09 10:22:31 -04:00
int m_oosNmsFired ;
int m_oosNmsHits ;
int m_oosNmsLastBuyIdx ;
int m_oosNmsLastSellIdx ;
int m_oosNmsKeptIdx ;
double m_oosNmsKeptConf ;
ENUM_SIGNAL m_oosNmsKeptDir ;
2026-08-22 00:24:45 -04:00
//--- How many eras the maximum was taken over - the N in the Sidak correction. Run-scoped: reset
//--- with the rest of the best-checkpoint tracking at the top of a fresh run.
feat: gate deployment on the null of the MAXIMUM, not the per-era null
EDGE_MIN_SIGMAS is a PER-ERA test and the deployed model is the MAXIMUM
over every era a run ranks. A 2-sigma one-sided test passes on noise with
probability 0.0228 per era, so over N eras the chance at least one clears
it is 1-(1-0.0228)^N: 34% by era 18, 80% by era 70, 93% by era 112. The
gate was near-certain to open on a long run whatever the data held.
It did. HYBRID deployed 2026-08-08 at dir-precision 35.5% vs 34% chance -
+1.5pp, best of 112 eras whose per-era values wandered 30%..35.5%. At the
call counts these runs produce that is p_family 0.92..0.9999.
Every OTHER best-of-N decision here already carries this correction, and
every one REJECTS on this data: the barrier-geometry winner (null of the
maximum over 6, p=0.3902), the indicator tuner (Sidak, p=1.0000), the MI
lag profile (null of the maximum over 21 lags). The one decision that
ships a model to a live account had none.
BestCheckpointSurvivesSelection() re-tests the checkpoint that is about to
deploy:
z = (precision - chance)/SE, SE = sqrt(p0(1-p0)/n)
p_single = P(Z >= z)
p_family = 1 - (1-p_single)^N
against DEPLOY_FAMILY_WISE_ALPHA. It uses the checkpoint's OWN
snapshotted precision/chance/call-count, not the latest era's, because
the model that ships is the one that has to clear the bar.
N counts CANDIDATE eras (coverage measurable, at least one directional
call) - an era that called nothing directional could never have become
the best, so counting it would make the gate stricter than the search
that actually happened.
Conservative on purpose: consecutive eras share OOS bars and differ by
one gradient step, so they are nowhere near N independent draws and the
true family-wise error is below this bound. This gate decides what trades
real money and the house posture is reject-unless-demonstrated.
Effect at 2900 directional calls / N=112: required edge goes 1.76pp ->
2.92pp. A real edge clears it; +1.5pp does not.
Applied to BOTH automatic paths - the plateau ladder's stage-3 deploy and
the m_trainingComplete assignment - which must stay identical or the flag
persisted into the .nnw disagrees with the decision to stop, and a reload
runs inference on a model the ladder refused.
NOT applied to the two operator paths (era-cap deploy, panel Deploy
button). Those stay the operator's call; ReportSelectionGateVerdict()
logs the verdict beside them so an authorised deploy can never later be
misread as a validated one.
NormalUpperTail() is A&S 26.2.17 (|err| < 7.5e-8), self-contained rather
than pulling in Math\Stat. Verified against reference values to 6dp:
Q(1.645)=0.049985, Q(1.96)=0.024998, Q(3.0)=0.001350. Its locals are
ntB1..ntB5 because AI\Network.mqh line 79 does "#define b1 AdamBeta1".
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 18:01:04 -04:00
int m_deployCandidateEras ;
//--- THE GATE. Re-tests the checkpoint that is about to deploy against the null of the MAXIMUM over
//--- m_deployCandidateEras eras, and reports the pieces so the log can show its working. See
//--- DEPLOY_FAMILY_WISE_ALPHA. Returns false (refuse) whenever the inputs are missing.
bool BestCheckpointSurvivesSelection ( double & zObs , double & pFamily , int & nTried ) ;
//--- Logs that verdict WITHOUT enforcing it, for the two deploy paths that are explicit operator
//--- decisions (the era cap and the panel's Deploy button). Those stay the operator's call; this just
//--- makes sure the log never lets an authorised deploy read as a validated one.
void ReportSelectionGateVerdict ( string context ) ;
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- Plateau ladder state (see the PLATEAU_* constants). m_erasSinceBest counts eras
//--- since the last NEW BEST selection score; m_plateauStage is how far up the escalation it
2026-08-22 00:24:45 -04:00
//--- has climbed.
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
int m_erasSinceBest ;
2026-07-25 15:55:56 -04:00
int m_plateauStage ;
feat(search): stop on the IN-SAMPLE plateau, and shrink every best-of-K effect before quoting it
Points 3 and 4 of the four-point plan.
1. IN-SAMPLE EARLY STOP - and the reason it is worth having is not compute.
The plateau ladder stops on the OOS SELECTION score. That is a peek: by the time it
fires, every one of those eras has been evaluated out of sample, so all of them sit
in the family the deploy gate corrects over (g_ensCandidateEras, Sidak). Training
longer therefore does not merely cost time - it RAISES the bar the eventual winner
has to clear.
The new stop reads the TRAINING error, which the gate never looks at. When the
optimiser has stopped improving on data it can see, more eras will not find a better
model; they will only enlarge the OOS family. Ending there shrinks the correction,
and the shrinkage is legitimate precisely BECAUSE the stopping rule never consulted
an out-of-sample number.
That distinction is the whole point and it is the one this project has got wrong four
times: stop on IS and the family really is smaller; stop on OOS and those eras were
searched and still count. Both stops now exist; only this one buys a lower bar.
Deliberately more patient than the OOS ladder (IS_ERROR_PATIENCE_MULT = 3x): training
error is noisy per era - mini-batch order alone moves it - and ending a run that is
still learning costs far more than a few wasted eras. Improvement is RELATIVE
(IS_ERROR_IMPROVE_FRAC = 1%), so it does not depend on the loss's absolute scale, and
it only acts when a checkpoint exists, since otherwise it would end a run with
nothing to deploy. Reset per RUN alongside the ladder, so a resumed run cannot
early-stop on its first era against a previous run's best.
2. WINNER'S-CURSE SHRINKAGE ON THE BARRIER-GEOMETRY WINNER.
The family-wise permutation gate already establishes that the RANKING is not noise.
It says nothing about the SIZE of the winner's effect - and a best-of-K maximum is
biased upward by construction, being the largest of K noisy draws. The adoption
message quotes that raw maximum and compares it against the incumbent, so the number
a reader plans on is the inflated one.
The penalty is now measured, not assumed: the same permutation draws that produce the
p-value also produce, per draw, the MAXIMUM excess across all candidates under pure
noise. The mean of those maxima is exactly what a best-of-K selection is expected to
report when there is nothing there. This is the empirical form of the sqrt(2 ln K) x SE
penalty the SQX EdgeFinder plugin applies to every maximum it reports (Stats.java:79-88),
and it needs no normality assumption because the draws ARE the null distribution.
Applied in James-Stein form - effect x max(0, 1 - penalty^2/effect^2) - so a large
effect is nearly untouched and a marginal one collapses toward zero.
Reported, not gated. The adoption decision still turns on the permutation p-value,
which is the right test for "is the ranking real"; the shrunk number is there so the
magnitude quoted beside it is one worth planning on. Closes the first of the two
EdgeFinder ports identified on 2026-08-12.
NOTE on the second EdgeFinder port, deliberately not done here: "let the measurement
steer the target" is already true where it matters most - ReportGeometryExpectancyScan
ADOPTS the winning barrier geometry under the family-wise gate rather than advising
it, and the MI excursion suite publishes a verdict per instrument per config. What is
still missing is steering the TRAINING TARGET itself (direction vs excursion) off
those verdicts, and that is a design change rather than a surgical one - direction is
a closed verdict while excursion SIZE keeps clearing, so the honest version of that
change is a target-selection policy, not a flag.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:29:00 -04:00
//--- IN-SAMPLE early-stop state (see IS_ERROR_IMPROVE_FRAC). Best training error seen this run and
//--- eras since it last improved. -1 = nothing measured yet.
double m_bestIsError ;
int m_erasSinceBestIsError ;
2026-08-22 00:24:45 -04:00
//--- LATCHES when the IN-SAMPLE error stops improving, and it is a separate flag from
//--- m_plateauStage for one measured reason: EnsembleEraVerdict mirrors the shared ladder onto
//--- every member with `mm.m_plateauStage = g_ensPlateauStage` on EVERY era, purely so each
//--- member's status line reads the collective stage.
fix(plateau): the IS-error early stop was inert for every ensemble member
15 hours of training, and the stop that exists to END a run announced itself
1,299 consecutive times without ending anything:
SP500 ConvLSTM IN-SAMPLE ERROR PLATEAU - not improved in 1297 / 1298 / 1299
eras (best 0.2689, now 0.3269) ... era 1396, 1397, 1398
SP500 LSTM 536 eras SP500 CONV 442 eras SP500 PAI 150 eras
XAUUSD HYB 478 eras XAUUSD LSTM 296 eras XAUUSD CONV 366 eras
CAUSE: it wrote its decision into m_plateauStage, and EnsembleEraVerdict mirrors
the shared ladder onto every member - `mm.m_plateauStage = g_ensPlateauStage` -
on EVERY era, purely so each member's status line shows the collective stage. A
display mirror was silently overwriting a decision, so the stop re-armed and
re-fired the next era, forever.
This is the worst possible direction for this particular bug. Every one of those
1,299 eras was scored out of sample and joined the family the deploy gate
corrects over (Sidak, g_ensCandidateEras). The stop's entire purpose is to make
that family SMALLER; instead the run spent fifteen hours raising its own bar.
- m_isErrorPlateaued: a one-way per-member latch, cleared only by a fresh run.
Nothing in the ladder may reset it. The stop condition and the two solo deploy
conditions read the latch, not the mirrored stage.
- The orchestrator combines: EnsembleEraVerdict requires UNANIMITY across
participating members (same participation test the era barrier uses, so an
excluded or finished member cannot veto). One member still learning can still
move the combined vote, and the vote is what the gate certifies.
- Fed in as `dueStage = PLATEAU_STAGE_DEPLOY`, NOT written to g_ensPlateauStage.
The block that actually ends the run sits under `dueStage > g_ensPlateauStage`,
so assigning the stage directly makes that test false and the deploy never
happens - the same inert-write shape as the bug being fixed. Caught before
committing; raising dueStage carries it through the ladder's own path (warm
restarts skipped, family-wise vote test, measurement screen, joint checkpoint)
unchanged.
- g_ensIsPlateauAnnounced: announce once per run, not once per era.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 13:09:26 -04:00
bool m_isErrorPlateaued ;
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric
Four of the six findings from research/training_pipeline_audit_2026-08-09.md
(F4 mini-batching and F6 feature re-encode deliberately deferred - see the
report's implementation-status section for why):
- F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%,
which is 15-bit - provably non-uniform on every full-history era over 32,768
queued samples. New 30-bit ShuffleRandomIndex().
- F2: plateau warm restarts were a no-op whenever eta already sat at its
ceiling (the normal state of a non-regressing plateau) - the ladder was just
a 24-era countdown. Restarts now overshoot to 5x the ceiling
(PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience
window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has
real range.
- F3: checkpoint restores put weights back but kept the rejected trajectory's
Adam moments, so the optimizer immediately pushed back toward the rolled-back
state (the restore->regress->restore oscillation). CNet::ResetOptimizerState()
zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta
untouched) on every mid-run restore, every boosted restart, and the
deploy-time restore that online learning continues from.
- F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk,
so the selection metric the checkpoint ranking and deploy gate read is a pure
function of the checkpoint instead of partly measuring BN drift. Defensive
unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and
the OOS continual-learning simulation stay adaptive by design.
Compiled clean (0 errors, 0 warnings) via the staged-tree recipe.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 10:54:09 -04:00
//--- Eras remaining in the current warm-restart boost window (see PLATEAU_RESTART_BOOST): set to
2026-08-22 00:24:45 -04:00
//--- PLATEAU_PATIENCE_ERAS by each boosted restart, decremented by the era-end anneal that walks
//--- g_eta back to the ceiling, cleared by any new best.
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric
Four of the six findings from research/training_pipeline_audit_2026-08-09.md
(F4 mini-batching and F6 feature re-encode deliberately deferred - see the
report's implementation-status section for why):
- F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%,
which is 15-bit - provably non-uniform on every full-history era over 32,768
queued samples. New 30-bit ShuffleRandomIndex().
- F2: plateau warm restarts were a no-op whenever eta already sat at its
ceiling (the normal state of a non-regressing plateau) - the ladder was just
a 24-era countdown. Restarts now overshoot to 5x the ceiling
(PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience
window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has
real range.
- F3: checkpoint restores put weights back but kept the rejected trajectory's
Adam moments, so the optimizer immediately pushed back toward the rolled-back
state (the restore->regress->restore oscillation). CNet::ResetOptimizerState()
zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta
untouched) on every mid-run restore, every boosted restart, and the
deploy-time restore that online learning continues from.
- F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk,
so the selection metric the checkpoint ranking and deploy gate read is a pure
function of the checkpoint instead of partly measuring BN drift. Defensive
unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and
the OOS continual-learning simulation stay adaptive by design.
Compiled clean (0 errors, 0 warnings) via the staged-tree recipe.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 10:54:09 -04:00
int m_restartBoostErasLeft ;
refactor(ai): nine class-imbalance inputs down to two
The imbalance section offered nine controls for one job. Audited against the
code, five of them did not do what their names said at the shipped defaults:
AILogitPriorStrength DEAD - Inference.mqh's post-hoc prior early-returns
whenever the adjusted loss is on, which is default.
OversampleParity DEAD in training - Training.mqh gated the replay loop
on !useLogitAdjustedLoss (correctly, citing Buda et
al. 2018). Live only in the online-learning path.
EnableMinorityReplay DEAD as replay. It survived ONLY as a focal-gamma
damper - "replay minority bars through pass-2
oversampling" was a focal-loss switch.
ConstrainReplay DEAD as a cap; it only chose damper 0.125 vs 0.25.
UseStaticPrior An exact duplicate of FreezePriorCalibration - the two
were OR'd together in the single place either is read.
So they were not five mechanisms fighting; they were one mechanism plus eight
knobs that mostly described machinery that no longer ran. That is worse than
a real conflict, because the log agreed with the names: the label-cache line
printed "reps up to 28x (90% parity) (seeding era 0's class-balance
oversampling)" on every run, describing an oversampling pass that had been
switched off. It is fixed here too - it cost this session a wrong diagnosis.
The one genuine redundancy was focal loss, running at gamma*0.125 alongside
the adjusted loss: two corrections on the same axis, the exact stacking
failure this file already cited Buda et al. for in two other places, damped
by a replay flag whose replay path was itself dead. Removed rather than
re-tuned. The plateau ladder is unaffected - its escape is the learning-rate
warm restart; the gamma anneal beside it only ever stepped toward zero.
WHAT REMAINS is logit-adjusted loss (Menon et al. 2021) plus a prior freeze:
LogitAdjustTau 0 = off; replaces the separate EnableLogitAdjusted-
Loss boolean, since a strength dial where 0 already
means off does not need an on/off switch beside it.
FreezePriorCalibration unchanged.
It is the only one of the six corrections with a consistency guarantee, and
it is consistent for exactly the balanced-error metric checkpoint selection
already ranks on - so the loss and the deploy decision optimize one thing.
The online continual-learning path keeps its own alpha-balanced focal weight,
now as constants pinned to the removed inputs' shipped defaults, so its
behaviour is unchanged. It legitimately needs its own correction:
ApplyLogitAdjustment() only runs inside a training run, so a deployed model
that was reloaded carries no logit offsets and would otherwise stream 31:1
data into itself uncorrected.
The weights-filename fingerprint is BYTE-IDENTICAL. The focal slot was a
double fed to a %d conversion and had always emitted a literal 0; the |MR:
segment is written as the constant its shipped defaults produced. Dropping
either would have re-keyed every model and forced a from-scratch retrain of
the one topology currently converged and trading.
Also removed as orphans: FOCAL_GAMMA_PRESET, MAX_OVERSAMPLE_REPLICAS,
OVERSAMPLE_PARITY_FRACTION, PLATEAU_GAMMA_STEP, and the now-unreachable
"neutralized by prior correction" diagnostic.
Both builds compile 0 errors, 0 warnings. No retrain forced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 11:46:57 -04:00
//--- m_focalGammaRuntime removed 2026-07-31 with focal loss itself - see the removal note at the
//--- former m_focalGamma above. The plateau ladder keeps its learning-rate warm restart, which was
//--- always the actual escape; the gamma anneal beside it stepped monotonically to zero anyway.
2026-07-14 22:36:27 -04:00
uint m_syncWaitStartTick ; // 0 = not waiting on history sync; else GetTickCount() when the wait began
//--- 3 no-op passes on a fresh start (see InitNeuralNetwork()/ResetWeights()), each its own separately-
//--- scheduled Train() call (not a tight in-process loop), so the broker/terminal's history sync gets
//--- several real, wall-clock-separated chances to finish before the era loop commits to a bar count.
int m_warmupPassesRemaining ;
feat(train): ONE pass over the held-out slice at deploy, on the restored checkpoint
The OOS slice is the newest history and the model never trains on it, while
online learning adapts to every bar resolving AFTER deployment. That leaves a gap
exactly at the handover, over the most regime-relevant data there is. This closes
it: select on validation, then refit on everything, which is standard practice.
Placed AFTER Net.RestoreWeights() and ResetOptimizerState() and BEFORE
PersistDeployedModel(), so it refines the weights that were actually SELECTED
rather than whatever the run happened to end on, and what it produces is what
gets written down.
THE COST IS REAL AND IS NOW STATED IN THE LOG. The deploy line promises "every
model reverts to the weights it held at the era whose combined vote scored best,
so the ensemble that trades is exactly the one that was measured". After this
pass that is no longer literally true, so the pass prints that the certified
numbers belong to the PRE-PASS weights and must be quoted that way. Set
EnableOosFinalPass=false to keep certified == traded exactly.
Guards:
* ONE-SHOT PER RUN, and the flag is set BEFORE the loop so no early return inside
it can leave the pass eligible to fire twice over bars it already trained on.
Reset at m_trainRunActive=true, because a retrain is a fresh selection and
earns a fresh pass.
* THE CONVERGED RATE, never a plateau-boosted one: m_modelEta can still carry
PLATEAU_RESTART_BOOST from an escape attempt, and this is a refinement of a
selected model, not another warm restart. g_eta is what backProp reads, so that
is what is capped and restored.
* OLDEST -> NEWEST. Series indices count backwards, so decreasing i moves forward
in time - the order the bars happened in.
* A failed feedForward is never followed by backProp; the output layer would
still hold the previous sample's activations and the update would be this bar's
label against another bar's prediction.
* m_oosFinalPassCutoff records the newest bar consumed and is deliberately NOT
cleared on a new run, so a later run can say plainly that its out-of-sample
window reaches back into bars this model has already seen.
Expect the gain to come from CURRENCY rather than finer weights: OOS precision
was measured flat from era 20 while in-sample error kept falling, so the data
this model can already see is exhausted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 11:05:43 -04:00
//--- ONE-SHOT GUARD for the deploy-time pass over the held-out slice. Set BEFORE the loop runs, so
//--- no early return inside it can leave the pass eligible to fire twice on one run. Reset only at
//--- the start of a new training run - a retrain is a new selection, so it earns a new pass.
bool m_oosFinalPassDone ;
//--- Newest bar the final pass consumed, 0 = none. Kept so a LATER run can say plainly that its
//--- out-of-sample window overlaps bars this model has already trained on, instead of quietly
//--- scoring against memorised data.
datetime m_oosFinalPassCutoff ;
2026-07-14 22:36:27 -04:00
bool m_labelCacheBuy [ ] ;
bool m_labelCacheSell [ ] ;
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- Bars-to-resolution of each cached label (idx - P2), stored under the SAME validity flag -
//--- the pool purge key reads it back as the earliest bar the label could have been known on.
int m_labelResolveAge [ ] ;
diag(label): expose bars-to-pivot - the confound the payoff test was missing
CORRECTION to what the payoff instrument was measuring. The 5-bar horizon looked
like the powered test and it is confounded.
SwingPivotDirectionLabel returns Buy when a swing LOW lands up to
PIVOT_LABEL_TOLERANCE_BARS bars AHEAD, and says the quiet part itself: gating on
where the pivot sits relative to entry "would drop exactly the bars where the
turn has not finished coming to us", and how much adverse move remains before
the turn "is a trade-management question".
So on a CORRECT Buy call price is often still falling for d more bars. A window
shorter than d measures the APPROACH, not the leg, and its negative contribution
is expected on the calls that are RIGHT. The tight null at 5 bars
(-0.012 +/- 0.074) is therefore not evidence of no payoff. Neither horizon is
both clean and powered: 5 bars is powered and confounded, 18-19 is clean and has
an SE of 0.277.
(idx - P1) was computed in the label and thrown away. Now cached beside
m_labelResolveAge under the same validity flag, and bucketed in the era verdict.
DELIBERATELY NOT USED AS A PER-CALL HORIZON, which is the trap sitting right
next to this: d exists only on bars the label found a pivot for, so a horizon
that varied with d would hand correct and incorrect calls different windows and
bias the comparison outright. The horizon stays fixed; d only buckets.
The bucket for "the label called no pivot here" is reported by name rather than
folded in, because it is the control the others are read against. Buckets 1..N
condition on the label, so they describe the MECHANISM, not what a book earns.
Reads: rising with d means the edge is in EARLY calls and the tolerance window
is spending it - fixable by reweighting the loss, not by a new label. Flat means
that hypothesis dies.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 08:16:47 -04:00
//--- BARS FROM THIS BAR TO THE PIVOT IT CALLS (idx - P1), or -1 where there is no pivot to call.
//--- A DIFFERENT QUANTITY from m_labelResolveAge (which is idx - P2) and from the lifespan
//--- (PIVOT_LABEL_TOLERANCE_BARS, a constant). Stored under the SAME validity flag as the label.
//---
//--- WHY IT IS KEPT: the label fires when a pivot lands up to PIVOT_LABEL_TOLERANCE_BARS bars
//--- AHEAD, so on a CORRECT call price may still be moving against the call for d more bars -
//--- SwingPivotDirectionLabel says exactly this ("the bars where the turn has not finished coming
//--- to us"). Any payoff measured over a window shorter than d is therefore measuring the
//--- APPROACH, not the leg, and its negative contribution is expected on the calls that are RIGHT.
//--- This is what lets the payoff diagnostic separate the two.
int m_labelBarsToPivot [ ] ;
2026-07-14 22:36:27 -04:00
bool m_labelCacheHasValue [ ] ;
int m_labelCacheBars ; // 0 = no cache built yet
datetime m_labelCacheAnchorTime ; // m_Time.GetData(0) at last (re)build - 2nd invalidation key
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- Resolves and caches the swing label for one bar - finality-gated, see the definition.
void AdvanceSwingLabelState ( int idx , int bars ) ;
//--- Independent-observation count behind `rawN` overlapping labels. See m_lastLabelLifespan for
//--- the measurement and for what an uncorrected n did to the operating point.
fix(topology): the capacity budget counted overlapping bars as independent examples
EstimatedInSampleBars() returned raw bars (11372 on SP500 H4) and every derived
capacity decision spent that: first-layer width, conv filters, LSTM hidden size.
But triple-barrier labels overlap - mean lifespan 9.4 bars - so the label cache line
on the same run already reports those bars are worth ~1210 independent observations.
Sizing a network against RAW bars while grading it against EFFECTIVE ones is two
subsystems disagreeing about one sample, and it disagreed in the dangerous direction
because the capacity side was the optimistic one: the warning's "roughly 1.1 weights
per training bar" is nearer 11 per independent observation.
EffectiveSampleSize() has existed since 2026-08-17 and is applied at eight sites, all
of them statistics. This adds the ninth, in the one place that decides how many
parameters get fitted. Applied inside EstimatedInSampleBars() rather than at the call
sites, because that function exists precisely so the three stages spend one budget.
SELF-ENABLING AND THEREFORE INERT WHERE IT MATTERS MOST, which is why this is two
changes and not one. MeanLabelLifespan() is 1.0 until a label cache has measured
something, so on a model's first build - before any label exists - the deflation is
correctly the identity: an unmeasured overlap must not invent a shrink. A fresh
attach constructs a fresh object, so its counters are zero too; only a mid-session
weights reset carries real evidence into a rebuild. That is deliberately safe (no
attach can now re-derive a narrower topology and discard trained weights) but it
would have left the first build - the case you most want the truth for - quoting the
flattering figure. So ReportDetectability now restates capacity against the effective
sample at the first moment L is real, for the topology already pinned. It re-sizes
nothing; it reports what was bought. Placed ABOVE that function's break-even guard on
purpose - a degenerate geometry is exactly when you want to know the net is
over-parameterised, and "it only fires for sane configs" is how the 2026-08-18
IS-error stop managed never to fire at all.
The warning also names its basis now (independent observations and L, or an explicit
"overlap NOT YET MEASURED, this is an UPPER BOUND"), so a flattering number can never
again read as a measured one.
Also factors FirstLayerFanIn() out of ComputeFirstLayerWidth so the capacity REPORT
charges for exactly what the capacity DECISION charged for - same reason
RequiredHorizonBars was factored out after the 2026-08-17 divergence - and makes
MeanLabelLifespan()/EffectiveSampleSize() const so the const budget path can call them.
Verified: no recursion (EstimatedInSampleBars -> EffectiveSampleSize ->
EstimatedInSampleBarsRaw, which computes from Bars() alone); both new StringFormat
sites hand-counted (basis 3/3 and 1/1, CAPACITY 10 specifiers / 10 arguments).
NOT COMPILED - user compiles in MetaEditor.
2026-08-19 18:43:48 -04:00
double EffectiveSampleSize ( double rawN ) const ;
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- Mean bars-to-resolution over the label cache, or 1.0 before anything has been measured
//--- (which makes EffectiveSampleSize the identity rather than a guess).
fix(topology): the capacity budget counted overlapping bars as independent examples
EstimatedInSampleBars() returned raw bars (11372 on SP500 H4) and every derived
capacity decision spent that: first-layer width, conv filters, LSTM hidden size.
But triple-barrier labels overlap - mean lifespan 9.4 bars - so the label cache line
on the same run already reports those bars are worth ~1210 independent observations.
Sizing a network against RAW bars while grading it against EFFECTIVE ones is two
subsystems disagreeing about one sample, and it disagreed in the dangerous direction
because the capacity side was the optimistic one: the warning's "roughly 1.1 weights
per training bar" is nearer 11 per independent observation.
EffectiveSampleSize() has existed since 2026-08-17 and is applied at eight sites, all
of them statistics. This adds the ninth, in the one place that decides how many
parameters get fitted. Applied inside EstimatedInSampleBars() rather than at the call
sites, because that function exists precisely so the three stages spend one budget.
SELF-ENABLING AND THEREFORE INERT WHERE IT MATTERS MOST, which is why this is two
changes and not one. MeanLabelLifespan() is 1.0 until a label cache has measured
something, so on a model's first build - before any label exists - the deflation is
correctly the identity: an unmeasured overlap must not invent a shrink. A fresh
attach constructs a fresh object, so its counters are zero too; only a mid-session
weights reset carries real evidence into a rebuild. That is deliberately safe (no
attach can now re-derive a narrower topology and discard trained weights) but it
would have left the first build - the case you most want the truth for - quoting the
flattering figure. So ReportDetectability now restates capacity against the effective
sample at the first moment L is real, for the topology already pinned. It re-sizes
nothing; it reports what was bought. Placed ABOVE that function's break-even guard on
purpose - a degenerate geometry is exactly when you want to know the net is
over-parameterised, and "it only fires for sane configs" is how the 2026-08-18
IS-error stop managed never to fire at all.
The warning also names its basis now (independent observations and L, or an explicit
"overlap NOT YET MEASURED, this is an UPPER BOUND"), so a flattering number can never
again read as a measured one.
Also factors FirstLayerFanIn() out of ComputeFirstLayerWidth so the capacity REPORT
charges for exactly what the capacity DECISION charged for - same reason
RequiredHorizonBars was factored out after the 2026-08-17 divergence - and makes
MeanLabelLifespan()/EffectiveSampleSize() const so the const budget path can call them.
Verified: no recursion (EstimatedInSampleBars -> EffectiveSampleSize ->
EstimatedInSampleBarsRaw, which computes from Bars() alone); both new StringFormat
sites hand-counted (basis 3/3 and 1/1, CAPACITY 10 specifiers / 10 arguments).
NOT COMPILED - user compiles in MetaEditor.
2026-08-19 18:43:48 -04:00
double MeanLabelLifespan ( void ) const ;
2026-08-22 00:24:45 -04:00
//--- Last era's deploy-gate arithmetic, published purely so the era line can state the bar
//--- rather than leave it implicit. -1 = not computed this era.
feat(measurement): fix zero-skill denominator, publish the deploy bar, measure lifespan per rung, add a MEASURE scale objective
The last run could not have demonstrated an edge either way, and nothing in the
log said so. Four changes so it does.
1. THE ZERO-SKILL LINE DIVIDED BY THE WRONG DENOMINATOR. m_oosWinLongTotal resets
every era; m_oosSamples only resets on a full model reset. So 'always-long %'
decayed as ~1/era: a run whose true rate is 37% printed 1.2% at era 33 and
0.0% at era 2219. This is the SAME bug already found and fixed for
logBuyPredPct thirty lines above ('era-15 Buy:2% that was really ~30%'), left
in the one line whose whole job is to be the reference every other number is
read against. Correct at era 1, wrong everywhere after - including the '62%
zero-skill' figure in the 2026-08-16 notes. Now per-era, and always-short is
finally readable.
2. THE DEPLOY GATE STATES ITS OWN BAR. 'edge -1pp' era after era cannot separate
'short by a hair' from 'short by an amount no strategy could cover'. The era
line now prints the required win rate, the SE, the effective n and the
lifespan it was deflated by; above 100% it says UNREACHABLE. At 4,738 OOS bars
and L=75.6 there are ~63 independent observations, putting the bar near 66% at
typical coverage.
3. LIFESPAN MEASURED PER RUNG. The first-passage cache already stores touch ages
at every ladder level, so each candidate geometry's resolution time is
readable without training on it - L-vs-width becomes a measurement across the
whole ladder in ONE run rather than a second chart. Each rung reports L,
n_eff, min provable edge and min provable EV.
4. SCALE OBJECTIVE IS PHASE-AWARE, defaulting to MEASURE. Width and detectability
are opposed: labels overlap by L, L grows like m*k = width^2 at fixed ratio,
so min provable EV ~ width^2 while the cost saving from width is only linear.
Doubling width quadruples the smallest EV you can prove. DEPLOY (widest that
clears reachability) is right once an edge is known; MEASURE (narrowest that
keeps round-trip spread under BARRIER_MAX_COST_FRACTION_PCT) is right while it
still has to be shown. The direction does not depend on the exponent, and
item 3 makes the exponent checkable.
Fixed in review: m_lastRungLifespan is cleared on every LadderWinShare entry or a
rejected rung reports the previous rung's lifespan as its own; per-rung
detectability is labelled IS-sample based (the deriver may not see the holdout),
so absolute figures are optimistic while the ranking is unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 10:37:05 -04:00
double m_lastEdgeFloorPct ;
double m_lastPrecSE ;
double m_lastEffN ;
refactor(pool): the cross-instrument gate owns a directory, not a model
PooledGate was three CExpertSignalAIBase method bodies in an #included
partial. It is now CPooledGate, a class the signal owns.
It needed NO data view. Diagnosing that first is the point: the module
reads a directory of CSV files and knows nothing about a model. The
only things it needs from its owner - the symbol's own numbers and the
ratio they were measured at - are arguments. Handing it a
CTrainingDataView would have been machinery for a dependency that does
not exist.
The owner fills SPoolRecord (the on-disk shape, which already existed)
because only it knows its symbol, its actual TargetRR and its label
lifespan. `id` is passed per call rather than bound, so there is no
init-order question about when the identity became available - m_symbol
is set by CExpertSignal::Init and ID by SetIdentity, at different
times.
m_poolWriteWarned was a one-shot latch living on the signal for a
warning only this module emits. It is m_writeWarned, private.
targetRR is now threaded into ReadPooledEvidence rather than read from
the owner. That is not plumbing for its own sake: a peer measured at a
different ratio has a different structural break-even, and only the
caller knows which ratio it is asking about.
Caught before compiling: I declared ReadPooledEvidence from memory as
(..., double &pooledEffN, const double targetRR). The real signature
ends in `string &detail`. Read the definition, aligned both ends.
Call sites in Training.mqh are untouched - PublishPoolRecord and
PooledGatePasses remain on the signal as the thin fillers that know its
geometry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 13:32:48 -04:00
//--- CROSS-INSTRUMENT CERTIFICATION - see Training\PooledGate.mqh for why the deploy bottleneck
//--- is certification rather than training, and why only the EVIDENCE pools while each symbol
//--- keeps its own model, geometry and chance rate. A collaborator: it owns a directory of CSV
//--- files and knows nothing about a model, so it takes numbers rather than a data view.
CPooledGate m_pooledGate ;
feat(training): wire TrainingPool into pass 2 - peer rows contribute gradient only
Peer rows join m_isTrainQueue as NEGATIVE sentinels before the shuffle, so they interleave with
this chart's samples instead of training in a block at one end. A block would be a curriculum:
whatever the optimizer saw last would decide where it landed.
TrainPoolStep is a separate path on purpose. Everything in pass 2's local branch after the
forward pass reaches for something indexed by a LOCAL bar - m_labelCache, m_winLongCache, the
excursion target, the arrow cache, m_Time - and a peer row has none of those. Sharing the path
would mean inventing values for all of them, which is how another instrument's outcomes end up
inside m_cumIsCorrect and the operating point gets fitted to them. The IS-vs-OOS gap is read as
THE overfitting signal, so polluting the IS side would not crash anything; it would just quietly
stop meaning what it says.
The purge key reuses the label walk's own two bounds - the horizon and NextScheduledCloseAll -
rather than approximating with a bar offset. A second horizon model here would drift from the
real one, and this project already measured that the close-all, not the nominal horizon, is what
actually terminates labels. Cutoff is the OLDEST OOS BAR'S TIME, in wall clock, because bar
indices cannot be compared across instruments that each have their own calendar.
Contribution happens while the window is still in TempData and before the forward pass
overwrites it, and is gated to direction models: the meta head trains a different target on a
wider input, which the fingerprint gate alone would NOT catch, since a meta model's fingerprint
matches its own peers perfectly well.
Use_Training_Pool ships false and does nothing until a second chart runs a matching fingerprint.
Compile-verified against a BASELINE of the same tree without the wiring: both produce 12
errors, all error 313 invalid-resource-path from #resource directives that cannot resolve in a
headless staged build (stock Controls res\*.bmp, plus the pre-existing Network.cl). Code errors
0, warnings 0, identical to baseline. Staging copy and junctions removed; the live .ex5 was
never touched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 14:00:21 -04:00
//--- CROSS-INSTRUMENT TRAINING ROWS - see Training\TrainingPool.mqh for the measurement that
//--- justifies it (+2.02pp at t_mkt 3.97, clearing its family-wise bar, replicated at D1).
//--- Collaborators on the same terms as m_pooledGate: they own a directory of row files and know
//--- nothing about a model, so they take numbers rather than a data view. Writer and reader are
//--- separate because they run at different points of the era and change for different reasons.
CTrainPoolWriter m_trainPoolWriter ;
CTrainPoolReader m_trainPoolReader ;
refactor(meta): remove meta-labeling entirely - RETRAIN-NEUTRAL
~2,300 lines. META had real, repeatedly measured ranking skill and ZERO
operating points that ever cleared break-even (0/350 H1 eras, 1/999 H4
pre-2-sigma, 0/8 pooled fitted points). The clinching arithmetic was edge x
width = 0.095 ATR/trade against spread 0.099 ATR/trade, and the
dose-response showed the high-conviction tail is temporally unstable -
the precision-vs-threshold slope flips sign between calib and test on 3 of
4 symbols, so no ex-ante threshold rule exists. It shipped default-off and
never gated a live entry. The self-measured tier weights are what actually
rank the vote, and all six H4 instruments converged on them alone.
RETRAIN-NEUTRAL, and that is the property that made this safe:
- The weights fingerprint emitted "|TGT:META2" or "|TGT:SWG1" from an
if/else. Every direction model already took the SWG1 arm, so
collapsing it to an unconditional append is byte-identical. No .nnw or
.cfg is orphaned or re-keyed.
- NetInputWidth() lost its "+ MetaDescWidth()" term. MetaDescWidth()
returned 0 for every direction model, so the input layer is unchanged.
- DbLegacyAiSlot()'s slot 5 was reachable only with all four Use_* NNs
off AND meta on - a config that never shipped. Every existing .db keeps
its filename.
Deleted outright: Signals/SignalMETA.mqh, Expert/Trading/MetaGate.mqh (the
directory is now empty), Expert/Training/{MetaCorpus,MetaCandidateStore,
MetaFamilies}.mqh, Tests/Test_MetaFamilies.mq5, Meta_Labeling_Design.md.
Unwound in place, the delicate part: Training.mqh carried four
IsMetaTarget() branches whose else-arm WRAPPED the direction body (pass 1
queueing, pass 2 backprop, pass 2.5 calibration, pass 3 OOS scoring). Each
wrapper is removed and the direction body promoted back to its original
nesting - the bodies were never re-indented when the wrappers were added,
so the promoted code is byte-identical to what ran before META existed.
Also gone: the ensemble verdict's meta-veto replay and its
approved/vetoed/unscored counters, the per-family/per-side OOS
decomposition arrays, the m_isTrainQueueCand parallel queue and its
lockstep shuffle, and the S2 era report.
Also removed: the CMetaGate abstraction and the live CheckOpenPosition
veto; m_gates plus AddFilter's non-voter routing and IsVotingSignal()
(META was the only non-voting child, so m_gates was always empty);
m_parentSignal/SetParentSignal (existed only to reach the root's gate);
SweepPrepare/SweepPrepareIndicator (only caller was the corpus sweep);
IsMetaTarget() from all four view interfaces and their adapters;
Use_MetaLabeling, EnableMETA, Meta_ExportDataset, m_trainTarget.
EvalShift is KEPT - HistoricalNetVote() uses it for the filtered overlay,
not just the corpus sweep; only its comment changed. The 2-output softmax
arm in NetForward.mqh is kept too: it costs nothing and is the reusable
binary-head path, now commented as unclaimed rather than as META's.
Compile-verified in _claude_stage: 0 errors, 0 warnings, matching the
pre-edit baseline.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 09:44:52 -04:00
//--- Off unless the operator asks for it.
bool TrainPoolEnabled ( void ) { return Use_Training_Pool ; }
feat(training): wire TrainingPool into pass 2 - peer rows contribute gradient only
Peer rows join m_isTrainQueue as NEGATIVE sentinels before the shuffle, so they interleave with
this chart's samples instead of training in a block at one end. A block would be a curriculum:
whatever the optimizer saw last would decide where it landed.
TrainPoolStep is a separate path on purpose. Everything in pass 2's local branch after the
forward pass reaches for something indexed by a LOCAL bar - m_labelCache, m_winLongCache, the
excursion target, the arrow cache, m_Time - and a peer row has none of those. Sharing the path
would mean inventing values for all of them, which is how another instrument's outcomes end up
inside m_cumIsCorrect and the operating point gets fitted to them. The IS-vs-OOS gap is read as
THE overfitting signal, so polluting the IS side would not crash anything; it would just quietly
stop meaning what it says.
The purge key reuses the label walk's own two bounds - the horizon and NextScheduledCloseAll -
rather than approximating with a bar offset. A second horizon model here would drift from the
real one, and this project already measured that the close-all, not the nominal horizon, is what
actually terminates labels. Cutoff is the OLDEST OOS BAR'S TIME, in wall clock, because bar
indices cannot be compared across instruments that each have their own calendar.
Contribution happens while the window is still in TempData and before the forward pass
overwrites it, and is gated to direction models: the meta head trains a different target on a
wider input, which the fingerprint gate alone would NOT catch, since a meta model's fingerprint
matches its own peers perfectly well.
Use_Training_Pool ships false and does nothing until a second chart runs a matching fingerprint.
Compile-verified against a BASELINE of the same tree without the wiring: both produce 12
errors, all error 313 invalid-resource-path from #resource directives that cannot resolve in a
headless staged build (stock Controls res\*.bmp, plus the pre-existing Network.cl). Code errors
0, warnings 0, identical to baseline. Staging copy and junctions removed; the live .ex5 was
never touched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 14:00:21 -04:00
//--- Gradient-only step for one adopted peer row. Deliberately does NOT touch m_labelCache, the
//--- excursion head, the arrow cache or any IS counter: see the dispatch comment in pass 2.
void TrainPoolStep ( const int poolIdx ) ;
//--- Latest time this bar's label could have become knowable, as the pool's purge key.
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- Deliberately the SAME bound the label itself uses - the bar its pivot pair committed on -
//--- because a second, approximate model here would drift from the real one, and the whole point
//--- of the key is that a peer row must not carry the future into this fit.
feat(training): wire TrainingPool into pass 2 - peer rows contribute gradient only
Peer rows join m_isTrainQueue as NEGATIVE sentinels before the shuffle, so they interleave with
this chart's samples instead of training in a block at one end. A block would be a curriculum:
whatever the optimizer saw last would decide where it landed.
TrainPoolStep is a separate path on purpose. Everything in pass 2's local branch after the
forward pass reaches for something indexed by a LOCAL bar - m_labelCache, m_winLongCache, the
excursion target, the arrow cache, m_Time - and a peer row has none of those. Sharing the path
would mean inventing values for all of them, which is how another instrument's outcomes end up
inside m_cumIsCorrect and the operating point gets fitted to them. The IS-vs-OOS gap is read as
THE overfitting signal, so polluting the IS side would not crash anything; it would just quietly
stop meaning what it says.
The purge key reuses the label walk's own two bounds - the horizon and NextScheduledCloseAll -
rather than approximating with a bar offset. A second horizon model here would drift from the
real one, and this project already measured that the close-all, not the nominal horizon, is what
actually terminates labels. Cutoff is the OLDEST OOS BAR'S TIME, in wall clock, because bar
indices cannot be compared across instruments that each have their own calendar.
Contribution happens while the window is still in TempData and before the forward pass
overwrites it, and is gated to direction models: the meta head trains a different target on a
wider input, which the fingerprint gate alone would NOT catch, since a meta model's fingerprint
matches its own peers perfectly well.
Use_Training_Pool ships false and does nothing until a second chart runs a matching fingerprint.
Compile-verified against a BASELINE of the same tree without the wiring: both produce 12
errors, all error 313 invalid-resource-path from #resource directives that cannot resolve in a
headless staged build (stock Controls res\*.bmp, plus the pre-existing Network.cl). Code errors
0, warnings 0, identical to baseline. Staging copy and junctions removed; the live .ex5 was
never touched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 14:00:21 -04:00
long TrainPoolResolvedMs ( const int entryIdx )
{
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
int age = ( entryIdx > = 0 & & entryIdx < ArraySize ( m_labelResolveAge ) ) ? m_labelResolveAge [ entryIdx ] : 0 ;
int resolvedIdx = entryIdx - MathMax ( age , 1 ) ;
if ( resolvedIdx < 0 )
resolvedIdx = 0 ;
return ( long ) m_Time . GetData ( resolvedIdx ) * 1000 ;
feat(training): wire TrainingPool into pass 2 - peer rows contribute gradient only
Peer rows join m_isTrainQueue as NEGATIVE sentinels before the shuffle, so they interleave with
this chart's samples instead of training in a block at one end. A block would be a curriculum:
whatever the optimizer saw last would decide where it landed.
TrainPoolStep is a separate path on purpose. Everything in pass 2's local branch after the
forward pass reaches for something indexed by a LOCAL bar - m_labelCache, m_winLongCache, the
excursion target, the arrow cache, m_Time - and a peer row has none of those. Sharing the path
would mean inventing values for all of them, which is how another instrument's outcomes end up
inside m_cumIsCorrect and the operating point gets fitted to them. The IS-vs-OOS gap is read as
THE overfitting signal, so polluting the IS side would not crash anything; it would just quietly
stop meaning what it says.
The purge key reuses the label walk's own two bounds - the horizon and NextScheduledCloseAll -
rather than approximating with a bar offset. A second horizon model here would drift from the
real one, and this project already measured that the close-all, not the nominal horizon, is what
actually terminates labels. Cutoff is the OLDEST OOS BAR'S TIME, in wall clock, because bar
indices cannot be compared across instruments that each have their own calendar.
Contribution happens while the window is still in TempData and before the forward pass
overwrites it, and is gated to direction models: the meta head trains a different target on a
wider input, which the fingerprint gate alone would NOT catch, since a meta model's fingerprint
matches its own peers perfectly well.
Use_Training_Pool ships false and does nothing until a second chart runs a matching fingerprint.
Compile-verified against a BASELINE of the same tree without the wiring: both produce 12
errors, all error 313 invalid-resource-path from #resource directives that cannot resolve in a
headless staged build (stock Controls res\*.bmp, plus the pre-existing Network.cl). Code errors
0, warnings 0, identical to baseline. Staging copy and junctions removed; the live .ex5 was
never touched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 14:00:21 -04:00
}
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- Fills this instrument's record from its own numbers and hands it over. Stays here because
//--- only this class knows its symbol and its label lifespan.
refactor(pool): the cross-instrument gate owns a directory, not a model
PooledGate was three CExpertSignalAIBase method bodies in an #included
partial. It is now CPooledGate, a class the signal owns.
It needed NO data view. Diagnosing that first is the point: the module
reads a directory of CSV files and knows nothing about a model. The
only things it needs from its owner - the symbol's own numbers and the
ratio they were measured at - are arguments. Handing it a
CTrainingDataView would have been machinery for a dependency that does
not exist.
The owner fills SPoolRecord (the on-disk shape, which already existed)
because only it knows its symbol, its actual TargetRR and its label
lifespan. `id` is passed per call rather than bound, so there is no
init-order question about when the identity became available - m_symbol
is set by CExpertSignal::Init and ID by SetIdentity, at different
times.
m_poolWriteWarned was a one-shot latch living on the signal for a
warning only this module emits. It is m_writeWarned, private.
targetRR is now threaded into ReadPooledEvidence rather than read from
the owner. That is not plumbing for its own sake: a peer measured at a
different ratio has a different structural break-even, and only the
caller knows which ratio it is asking about.
Caught before compiling: I declared ReadPooledEvidence from memory as
(..., double &pooledEffN, const double targetRR). The real signature
ends in `string &detail`. Read the definition, aligned both ends.
Call sites in Training.mqh are untouched - PublishPoolRecord and
PooledGatePasses remain on the signal as the thin fillers that know its
geometry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 13:32:48 -04:00
void PublishPoolRecord ( const double chancePct , const double winPct , const double effN )
{
SPoolRecord rec ;
rec . symbol = m_symbol . Name ( ) ;
//--- _Period, not Period(): inside a CExpertBase subclass the bare call resolves to the
//--- inherited SETTER bool CExpertBase::Period(ENUM_TIMEFRAMES) rather than the builtin.
rec . timeframe = ( int ) _Period ;
rec . chancePct = chancePct ;
rec . winPct = winPct ;
rec . effN = effN ;
rec . lifespanBars = MeanLabelLifespan ( ) ;
rec . eraCount = ( long ) m_eraCount ;
rec . stamp = 0 ; // stamped by the writer, so every record's clock is one clock
m_pooledGate . Publish ( ID , rec ) ;
}
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
bool PooledGatePasses ( string & report ) { return m_pooledGate . Passes ( report ) ; }
2026-08-22 00:24:45 -04:00
//--- Last era's pooled verdict, cached for the era line. Letting a cross-symbol result license a
//--- local deploy would ship a model that never cleared its own bar.
feat(gate): cross-instrument pooled certification
The deploy bottleneck is CERTIFICATION, not training. A 4,738-bar OOS window at
L=75.6 holds ~63 independent observations; certifying a 3pp edge at 2 sigma needs
~1,036. More bars of the same symbol barely help - they overlap. Other symbols do
not.
WHAT POOLS. Not win rates: symbols have different derived geometries, different
break-evens and different drifts, so averaging raw rates across them is
meaningless. What pools is each symbol's EXCESS OVER ITS OWN CHANCE RATE,
combined by inverse-variance weighting (fixed-effects meta-analysis). Each symbol
keeps its own model, geometry and chance rate; only the evidence is combined.
THE CORRELATION PROBLEM, bracketed rather than assumed away. SP500 and NAS100 are
~0.9 correlated and pooling them as independent inflates the evidence. Nothing
here can measure that without sharing return series, so instead of guessing a
correction the gate reports both ends:
SE_INDEP = sqrt(1/SUM(1/var_i)) all members independent
SE_CORR = SUM(w_i * sqrt(var_i)) all members perfectly correlated
The truth is always between. THE GATE USES SE_CORR, so a pass cannot be an
artifact of correlated instruments - that bound already assumes the worst. The
ratio is logged as the diversification credit the gate declines to claim, so the
cost of that conservatism is visible instead of hidden.
SCOPE, deliberately limited: the pooled result is REPORTED, never folded into
tradeableOK. The local gate certifies the model that actually trades this symbol;
the pool answers the different question of whether the strategy has an edge at
all. Letting a cross-symbol result license a local deploy would ship a model that
never cleared its own bar - so it cannot.
Mechanics: one file per instrument (no concurrent-write path to get wrong), every
FileOpen carrying FILE_SHARE_READ|FILE_SHARE_WRITE, records skipped rather than
reinterpreted on a version mismatch, 12h staleness cutoff so a stopped chart
cannot vote, and pooling refused below 3 instruments. Poolability requires
matching timeframe and ratio; differing SYMBOL is the entire point. Publishing is
unconditional - a pool that only hears from winners is a selection effect, not a
meta-analysis.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 10:45:37 -04:00
bool m_lastPoolPasses ;
string m_lastPoolReport ;
2026-08-22 00:24:45 -04:00
//--- full per-bar INPUT feature vector cache (everything BufferTempData() computes: ATR-
//--- normalized OHLC, time-of-day encoding, volume delta, AD indicator buffers, ...).
2026-07-14 22:49:14 -04:00
double m_featureCache [ ] ;
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
bool m_featureCacheHasValue [ ] ; // true once idx has a CACHED SUCCESS (f6150ee: only
// successes are ever cached - a miss is never stored,
// in any form; see BufferTempData's comment)
bool m_featureCacheValid [ ] ; // paired flag, always true when HasValue is true -
// kept for the (currently unreachable) cached-miss
// shape so the cache layout survives f6150ee
fix: a resumed model cached a cold ATR as permanent, so it never trained
BufferTempData cached EVERY failure - m_featureCacheHasValue[idx]=true
with m_featureCacheValid[idx]=false - and the cache never re-tries a
miss. So a single feature read taken before the terminal had finished
calculating the indicator buffers marked those bars unusable for the
rest of the process, even though the data arrived milliseconds later.
MT5 fills an indicator's buffers asynchronously after the handle is
created, and a cold ATR returns 0 for EVERY index, not just its warm-up
tail. BufferTempDataCompute rejects a bar with no ATR (correctly - the
price features would be meaningless), so the whole window failed, and
the whole cache was poisoned.
Only resumed models were hit, because only they read features that
early. Topology.mqh sets m_warmupPassesRemaining = netLoaded ? 0 : 3:
a fresh start sits through three separately-scheduled Train() calls
before anything touches a feature, which is exactly what those passes
are for. A resumed one skips them and TuneIndicatorsAndTrain drives
StartLabelCachePrebuild and the MI report from the first chart event.
Its rationale - "a restart already has a proven-synced history" - holds
for HISTORY and not for INDICATORS, which are recreated every process
start.
Downstream: BuildFeatureWindow failed on every bar of every era, so
add_loop never went true, so pass 2, pass 3, the era counter and the
checkpoint were all skipped and pass 1 swept 0->100% forever. The
"0 samples" MI report line at startup was the same failure, four
seconds earlier, already visible in the log.
- a miss is now cached only when it is PERMANENT; the two "not ready
yet" guards mark m_featureFailTransient and are recomputed on the
next visit. Steady-state cost is ~ind_Periods bars per era, not 54k.
- an era that discards itself now drops the feature cache before
restarting, so any remaining cause of this state self-heals instead
of looping.
Deleting the .nnw "fixed" this only by turning the model back into a
fresh one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:41:31 -04:00
//--- Set by BufferTempDataCompute when it rejected a bar because the data had not ARRIVED yet
2026-08-22 00:24:45 -04:00
//--- (price buffer EMPTY_VALUE, or an ATR the terminal has not finished calculating) as opposed
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- to the bar being genuinely unusable. Moved onto CFeatureBuilder as a real member (exclusive).
//--- WHICH BLOCK rejected the bar, and at which series index - stay HERE (Training.mqh reads both
//--- directly for the pass-1 stall report); CFeatureBuilder writes them via FeatureSetFailBlock()/
//--- FeatureSetFailIdx().
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
string m_featureFailBlock ;
int m_featureFailIdx ;
diag: name the cause when every feature window fails, and enforce the width contract
Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable
feature window, windows ok=0 failed=54681" and nothing else. That line
reads identically for a cold ATR, a conditionally-missing optional
feature block and an out-of-range index, so it cannot be diagnosed
without one restart per hypothesis.
Two changes:
1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit
exactly m_neuronsCount values on EVERY bar. A block that emits its
values on some bars and skips them on others (indicator, panel or
series unavailable for that bar) does not merely shorten the window -
it SHIFTS every feature after it into the wrong slot, and the net
then trains on silently misaligned inputs that still look like a
valid window to everything downstream. Now rejected, rolled back and
reported once, naming the optional blocks (XA / SPR / swing context)
as the ones carrying an availability test. Worth having independently
of the current stall.
2. BuildFeatureWindow records WHICH lookback slot rejected and how much
of the window was assembled, and the pass-1 stall report renders it:
"slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up
or history-edge read; "every lookback bar ACCEPTED and the window was
still short: 640 of 760" is a missing 6-value block.
No behaviour change on a healthy run: the width check is an equality
that already holds, and the diagnostics render only inside the
total-failure branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 10:31:56 -04:00
//--- Why the LAST BuildFeatureWindow failed, so the pass-1 stall report can name a cause instead of
//--- a count. Slot = which lookback position rejected (-1 = none did and the window was still
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- short); Total = how many values had been assembled when it gave up. Stay HERE for the same
//--- reason as m_featureFailBlock; CFeatureBuilder writes them via FeatureSetWindowFail().
diag: name the cause when every feature window fails, and enforce the width contract
Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable
feature window, windows ok=0 failed=54681" and nothing else. That line
reads identically for a cold ATR, a conditionally-missing optional
feature block and an out-of-range index, so it cannot be diagnosed
without one restart per hypothesis.
Two changes:
1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit
exactly m_neuronsCount values on EVERY bar. A block that emits its
values on some bars and skips them on others (indicator, panel or
series unavailable for that bar) does not merely shorten the window -
it SHIFTS every feature after it into the wrong slot, and the net
then trains on silently misaligned inputs that still look like a
valid window to everything downstream. Now rejected, rolled back and
reported once, naming the optional blocks (XA / SPR / swing context)
as the ones carrying an availability test. Worth having independently
of the current stall.
2. BuildFeatureWindow records WHICH lookback slot rejected and how much
of the window was assembled, and the pass-1 stall report renders it:
"slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up
or history-edge read; "every lookback bar ACCEPTED and the window was
still short: 640 of 760" is a missing 6-value block.
No behaviour change on a healthy run: the width check is an equality
that already holds, and the diagnostics render only inside the
total-failure branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 10:31:56 -04:00
int m_windowFailSlot ;
int m_windowFailTotal ;
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- m_featureWidthWarned/m_featureHealthReported moved onto CFeatureBuilder as real members
//--- (exclusive, ctor-init-list only elsewhere).
fix(altdata): median-fill instead of zero-fill, and a one-shot feature-vector autopsy
ALT-DATA AUDIT. The files themselves are healthy - all six symbols, 6,073 daily
rows, 2010-01-01 to 2026-08-17, no constant or degenerate columns, sane tails
(mac_cpi/mac_unemp flat ~47d is monthly data behaving correctly). The problem is
not the data, it is what happens where the data ISN'T.
CAltDataPanel::Features() returned an all-ZERO vector for any bar older than the
file's first row, and left blank cells at 0 too. Both were deliberate ('the block
is additive context and must degrade, never reject the bar') and that reasoning
holds for the CHANGE columns - but half these features are LEVELS: vix, ivol,
mac_y10, mac_cpi, mac_unemp, eia_util. For a level, 0 is not a missing reading,
it is an impossible one far outside the series' range. VIX does not visit zero.
And the spike lands in exactly the wrong place. Every alt file starts 2010-01-01
while the charts run far deeper - USDJPY H4 reaches ~1994, roughly HALF its
history - so 'alt block is all zeros' is precisely the predicate 'this bar is
older than 2010'. The IS/OOS split is chronological, so that predicate covers
~half of IS and none of OOS: an in-sample feature guaranteed to be useless
out-of-sample, and a bimodal input for the first BatchNorm to normalise. Not a
lookahead leak - a distribution corruption, which is quieter and was never
reported anywhere.
Now filled with the column MEDIAN over the covered range. A constant cannot leak
whatever its source - it takes the same value on every pre-coverage bar, so it
carries no information about which of those bars won - which is what makes a
median computed over later data legitimate here. Median not mean because the
series are skewed. Blank cells get the same treatment (eia_stk_idx1y alone has
181 blanks in 6,073 rows) and the count is now logged at load.
THE BACKOFF WAS ALREADY THERE AND WAS DEAD. Training.mqh arms m_coldSweepTick on
m_featureFailTransient, but only the open/ATR guards ever set that flag, so
f0cf659's cold ADMovingAverage looked PERMANENT and the sweep re-ran at full
speed forever. Setting the flag in the indicator guards revives the mechanism
that was already designed for this; no second backoff was needed and the one I
first wrote has been removed in favour of it.
SELF-HEALING, as asked. ReportFeatureHealth() runs once, the first time pass 1
produces usable windows, samples ~400 bars spread across the whole training range
and names every feature slot that is CONSTANT or mostly-zero, tagging alt-block
slots as alt[i]. Both of today's failures were the same shape - a block silently
produces nothing while every downstream number stays plausible - and neither an
accuracy figure nor a model can tell 'this feature is always 0' from 'this
feature is genuinely 0 here'. Evenly spaced sampling so a block that dies only in
deep history is caught as surely as one dead everywhere. A report, not a gate:
a rare-flag feature can be legitimately constant, and refusing to train would
turn a diagnostic into an outage.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 12:53:51 -04:00
//--- One-shot feature-vector autopsy - see ReportFeatureHealth() for the two silent 2026-08-17
//--- failures it exists to catch. Runs the first time pass 1 produces usable windows.
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
void ReportFeatureHealth ( int bars ) { m_featureBuilder . ReportFeatureHealth ( bars ) ; }
bool BufferTempDataCompute ( int idx ) { return m_featureBuilder . BufferTempDataCompute ( idx ) ; }
2026-07-19 11:04:38 -04:00
//--- Nearest confirmed (non-repainting) ZigZag pivot at or after fromIdx - see this method's
//--- definition comment and m_useSwingContext's declaration comment for the repainting-embargo
//--- rationale callers must apply to fromIdx before calling this.
bool FindConfirmedZigZagPivot ( int fromIdx , int & pivotIdx , double & pivotPrice , bool & pivotIsLow ) ;
2026-07-14 22:49:14 -04:00
bool EnsureBarCachesCapacity ( int bars ) ;
fix(labels): a closed candle shifts the cache, it does not invalidate it
Series indices are relative to now, so one new bar moves every cached bar's
index by one. EnsureBarCachesCapacity answered that by wiping the label cache,
the excursion caches, the ladder and the feature cache and rebuilding the whole
prebuild from scratch - on any timeframe where a bar closes before a run
finishes, the labels were being recomputed continuously and the training set
never held still.
The labels do not change when a candle closes. ShiftBarCaches moves every
per-bar cache up by the number of new bars, marks only those newest bars as
unfilled, and leaves the rest exactly as computed. CFirstPassageLadder gets a
matching Shift (resizing directly rather than through Allocate, which zeroes the
ages this is preserving).
Refuses, falling back to the full rebuild, when a prebuild is mid-flight: its
cursor is an index into the array being moved.
Compile-verified in the staging copy: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 18:36:16 -04:00
bool ShiftBarCaches ( const int bars , const int delta ) ;
2026-07-14 22:36:27 -04:00
//--- Eager label-cache pre-build + true-label tally, run once per fresh start (see
2026-08-22 00:24:45 -04:00
//--- m_warmupPassesRemaining) BEFORE era 0's real training loop begins.
2026-07-14 22:36:27 -04:00
bool m_labelCachePrebuilt ; // true once the one-time pre-scan has completed
bool m_labelPrebuildActive ; // true while a chunked pre-scan is in progress
bool m_prebuildSeedPending ; // true: era 0's era-start reset must NOT stomp the
// prebuild-seeded m_prevEraTrue* counts with the
// still-empty live tally (see Train()'s era-start block)
int m_labelPrebuildBars ;
int m_labelPrebuildOosCutoff ;
int m_labelPrebuildIndex ;
int m_labelPrebuildBuyCount ;
int m_labelPrebuildSellCount ;
int m_labelPrebuildNeutralCount ;
void StartLabelCachePrebuild ( void ) ;
void AdvanceLabelCachePrebuild ( void ) ;
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
//--- Evaluation-only continual-learning OOS simulation and the one-shot pattern-database backfill
//--- walk - both now live on COnlineLearning (m_onlineLearning); these stay one-line forwards at
//--- their original position so every internal caller (Training.mqh) is unchanged.
void StartOosContinualSimulation ( int bars , int oosCutoff ) { m_onlineLearning . StartOosContinualSimulation ( bars , oosCutoff ) ; }
void AdvanceOosSimulationChunk ( void ) { m_onlineLearning . AdvanceOosSimulationChunk ( ) ; }
void StartPatternDatabaseBackfill ( int bars , int totalIter , int oosCutoff ) { m_onlineLearning . StartPatternDatabaseBackfill ( bars , totalIter , oosCutoff ) ; }
void AdvancePatternDatabaseBackfill ( void ) { m_onlineLearning . AdvancePatternDatabaseBackfill ( ) ; }
2026-07-14 22:36:27 -04:00
//--- same resumability problem one level up: TuneIndicatorsAndTrain()'s own trial loop calls
//--- Train() per trial and used to assume each call ran an entire trial to completion synchronously
int m_tuneTrialIndex ; // -1 = no multi-trial tuning run in progress
double m_tuneBestOosForecast ;
bool m_tuneLastTrialWasWin ;
bool m_tuneHaveBestCheckpoint ;
datetime m_tuneStartTrainBar ;
perf(autotune): replace the genetic search with a filter score - hours to seconds
MEASURED COST OF THE GA, which is what retired it. Per generation:
rung 0: 8 cand x 3 seeds x 3 eras = 72 eras
rung 1: 4 cand x 3 seeds x 8 eras = 96
rung 2: 2 cand x 3 seeds x 20 eras = 120
= 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's
real training began. Against the observed era times on SP500 H1:
PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22)
CONV 41.3 s/era -> 13.2 h
LSTM 150.4 s/era -> 48.1 h
HYBRID 154.6 s/era -> 49.5 h
Two days to tune is not a first-run experience, and it is the phase in
which the panel goes quiet, which is what made it look like a hang.
It also bought nothing. The space is 90 points (10 MA periods x 9 MA
types), so 1152 evaluations revisited each point ~13 times; and rungs of
3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run
proves it: every finalist scored 25.0-25.9% balanced accuracy - below the
33.3% one-class floor, i.e. indistinguishable noise - and the search then
"deployed the winner" of that.
THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full
training run to choose a feature's period is a wrapper method paying
wrapper prices for a decision that does not need one. The reference book
does not do this: ch. 3.3 selects inputs by measuring each candidate
indicator's CORRELATION with the target and dropping the ones with none,
with no network involved.
So: rank candidates by the MUTUAL INFORMATION between the resulting
feature vector and the triple-barrier label. MI rather than correlation
because the label is 3-class categorical and the features are not
monotonically related to it. Equal-FREQUENCY binning (rank-based),
because these features are ATR-normalised and heavy-tailed - fixed-width
bins put nearly everything in one bucket and report ~0 information for a
genuinely useful feature.
Scoring is arithmetic over the feature cache, so it costs seconds and its
cost is independent of topology: LSTM now tunes as fast as the MLP.
Coordinate sweep, not product sweep - cost is the SUM of per-parameter
candidate counts, so enabling every indicator stays affordable - with a
second pass that breaks early once nothing moves.
Sampling is IS-ONLY. Letting the OOS window influence which indicator
settings ship would mean the holdout had been used for selection and had
stopped being a holdout.
HONEST LIMIT, recorded because it is the price: MI is marginal, so a
parameter that only pays off in combination with another can be missed
(Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it
replaces was ranking pure noise at 48 h a run, this is strictly better.
Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/
GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga*
members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget.
AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28
read sites all permanently inert. That is not a tidy-up: the `if
(!m_evalMode)` guard on UpdateClassPriors is exactly what silently
disabled the imbalance correction for entire runs two commits ago. Dead
machinery that still reads like live machinery is this codebase's most
expensive recurring bug, and leaving 28 more instances of it would have
been indefensible.
The panel's tuning-progress state goes too - tuning no longer takes long
enough to need one.
Both builds compile 0 errors / 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
//=== Filter-based indicator auto-tuner (see TuneIndicatorsByFilter) =============================
2026-08-22 00:24:45 -04:00
//--- Replaced a genetic + successive-halving search on 2026-08-01. See TuneIndicatorsByFilter()
//--- for the measurements and the honest limit.
perf(autotune): replace the genetic search with a filter score - hours to seconds
MEASURED COST OF THE GA, which is what retired it. Per generation:
rung 0: 8 cand x 3 seeds x 3 eras = 72 eras
rung 1: 4 cand x 3 seeds x 8 eras = 96
rung 2: 2 cand x 3 seeds x 20 eras = 120
= 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's
real training began. Against the observed era times on SP500 H1:
PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22)
CONV 41.3 s/era -> 13.2 h
LSTM 150.4 s/era -> 48.1 h
HYBRID 154.6 s/era -> 49.5 h
Two days to tune is not a first-run experience, and it is the phase in
which the panel goes quiet, which is what made it look like a hang.
It also bought nothing. The space is 90 points (10 MA periods x 9 MA
types), so 1152 evaluations revisited each point ~13 times; and rungs of
3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run
proves it: every finalist scored 25.0-25.9% balanced accuracy - below the
33.3% one-class floor, i.e. indistinguishable noise - and the search then
"deployed the winner" of that.
THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full
training run to choose a feature's period is a wrapper method paying
wrapper prices for a decision that does not need one. The reference book
does not do this: ch. 3.3 selects inputs by measuring each candidate
indicator's CORRELATION with the target and dropping the ones with none,
with no network involved.
So: rank candidates by the MUTUAL INFORMATION between the resulting
feature vector and the triple-barrier label. MI rather than correlation
because the label is 3-class categorical and the features are not
monotonically related to it. Equal-FREQUENCY binning (rank-based),
because these features are ATR-normalised and heavy-tailed - fixed-width
bins put nearly everything in one bucket and report ~0 information for a
genuinely useful feature.
Scoring is arithmetic over the feature cache, so it costs seconds and its
cost is independent of topology: LSTM now tunes as fast as the MLP.
Coordinate sweep, not product sweep - cost is the SUM of per-parameter
candidate counts, so enabling every indicator stays affordable - with a
second pass that breaks early once nothing moves.
Sampling is IS-ONLY. Letting the OOS window influence which indicator
settings ship would mean the holdout had been used for selection and had
stopped being a holdout.
HONEST LIMIT, recorded because it is the price: MI is marginal, so a
parameter that only pays off in combination with another can be missed
(Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it
replaces was ranking pure noise at 48 h a run, this is strictly better.
Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/
GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga*
members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget.
AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28
read sites all permanently inert. That is not a tidy-up: the `if
(!m_evalMode)` guard on UpdateClassPriors is exactly what silently
disabled the imbalance correction for entire runs two commits ago. Dead
machinery that still reads like live machinery is this codebase's most
expensive recurring bug, and leaving 28 more instances of it would have
been indefensible.
The panel's tuning-progress state goes too - tuning no longer takes long
enough to need one.
Both builds compile 0 errors / 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
bool m_tuneFilterDone ; // the one-shot filter pass has run for this model
//--- Mutual information between one feature column and the 3-class label, and the whole-vector score.
double FeatureColumnMI ( const double & vals [ ] , const int & labels [ ] , int n ) ;
2026-08-22 00:24:45 -04:00
//--- Returns the MEAN per-feature marginal MI. "0.001 nats" means nothing on its own; "0.1% of
//--- the label's entropy" is a magnitude anyone can act on.
fix(autotune): MI scorer read an array nobody filled; add the permutation floor
THE TUNER WAS A SILENT NO-OP. Every chart logged
auto-tune complete - 17 candidate settings scored in ~139s,
feature/label mutual information 0.0000 -> 0.0000 nats (no improvement)
0.0000 is not a weak result, it is a broken measurement: finite-sample MI
is biased UPWARD, so even pure noise scores above zero. Cause:
ScoreCurrentParamsByMI called BufferTempDataCompute(), which APPENDS the
bar's features to TempData and never touches m_featureCache - only the
caching wrapper BufferTempData() writes that array. It then read
m_featureCache, which ReInitADIndicators had just invalidated. Every
column came back constant, FeatureColumnMI returned 0 for all of them,
and all 17 candidates tied at exactly zero. 139 s per chart to return the
settings it started with.
Now reads the values back out of TempData, where they actually land. And
an exactly-zero best score is called out as a fault rather than reported
as "no improvement", because that is what it is.
ADDED: a PERMUTATION BASELINE, which is the diagnostic this project has
been missing. MI's finite-sample bias is ~(bins-1)(classes-1)/(2n) nats -
at these sample sizes the same order as any real edge in this domain - so
a raw MI figure is uninterpretable on its own. Shuffling the labels
destroys every genuine association while leaving sample size, binning and
class proportions intact, so the score it produces IS this dataset's
noise floor, measured rather than approximated. The log now reads
feature/label information - X nats against a shuffled-label floor of Y
and says outright whether the features carry usable information about the
target. It needs no training, no topology and no convergence, so unlike
every accuracy number in this codebase it cannot be confounded by an
optimizer or an objective. If the score sits on the floor, no change of
architecture can help - which is the question the last three days of
zero-edge results have been circling.
DEPLOY FLOOR: `dirPrecPct > chancePrecPct` passed anything above chance by
any amount. At ~11,000 directional calls the standard error of the
precision estimate is ~0.4pp, so that gate was accepting sub-one-sigma
noise - the perceptron deployed at edge +0pp on 2026-08-01. Now requires
EDGE_MIN_SIGMAS (2.0) standard errors above chance, computed from the
actual call count, so the bar scales with the evidence instead of needing
a hand-picked constant.
Recorded with it, because it is why chance is the right reference at all:
under a driftless random walk P(touch +k*ATR before -m*ATR) = m/(m+k),
and the break-even win rate for a k:m reward:risk trade is ALSO m/(m+k).
The label's own base rate IS the break-even rate, at every SL/TP setting.
So "beats chance" and "is profitable" are the same test, and no choice of
SL/TP can manufacture an edge - only prediction can.
Both builds compile 0 errors / 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:05:50 -04:00
double ScoreCurrentParamsByMI ( bool shuffleLabels = false ) ;
2026-08-22 00:24:45 -04:00
//--- The same work split in two, so the permutation test can extract the sample ONCE and reuse
//--- it for every null draw. The sampled range is trimmed by MiShiftPad() at both ends - a FIXED
//--- amount, never by |offset| - so every build enumerates the same bars in the same order and
//--- two builds can be compared row by row.
2026-08-24 21:01:08 -04:00
int BuildMiSample ( double & cols [ ] , int & labels [ ] , int labelBarOffset = 0 ) ;
2026-08-22 00:24:45 -04:00
//--- Bars trimmed from each end of every MI sample. Must cover the largest offset any caller
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- asks for: the alignment scan's MI_ALIGN_MAX_SHIFT and the positive control's quarter of the
//--- mean label resolution.
2026-08-02 08:12:47 -04:00
int MiShiftPad ( void ) const
{
2026-08-24 21:01:08 -04:00
return MathMax ( MI_ALIGN_MAX_SHIFT , LabelResolutionBars ( ) / 4 ) ;
2026-08-02 08:12:47 -04:00
}
diag(autotune): five permutations was still a coin flip - use a real test
The 5-draw z-score shipped an hour ago disproved itself on its first run.
All four charts scored the IDENTICAL 0.00401 nats on identical features
and identical labels - and reported z of +1.3, +2.0, +4.0 and +4.7. Two
"AT THE NOISE FLOOR", two "a real association", same data. The entire
swing came from estimating the null's spread from five draws, where the
standard deviation of the standard-deviation estimate is ~35%: the
denominator was noisier than the effect it was judging.
Replaced with an empirical permutation test. 200 draws, p counted by rank
with the +1/(B+1) correction (Phipson & Smyth 2010) so p is never
reported as exactly zero - no normality assumption and no spread to
estimate. The strongest single column is tested against the null
distribution OF THE MAXIMUM, which corrects for scoring 26 features at
once by construction and is far less conservative than Bonferroni.
Affordable because BuildMiSample is now split out of ScoreCurrentParamsByMI
and runs ONCE for the whole test - every draw reuses that sample and costs
a relabel plus 26 histogram passes, not 2000 feature extractions. The
coordinate sweep still calls the combined form, which is correct there:
each candidate changes the indicator settings, so its features really do
have to be re-extracted.
The verdict line keeps both questions apart and prints both answers: the
p-value for "is it real", the excess as a percentage of H(Y) for "is it
big enough to trade". At n=2000 those can disagree, and collapsing them
into one word is how a worthless effect gets called a discovery.
Compiles 0 errors / 0 warnings. Build tag permtest-v1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:45:46 -04:00
double ScoreMiSample ( const double & cols [ ] , int & labels [ ] , int n , bool shuffleLabels ) ;
2026-08-01 14:01:32 -04:00
//--- The permutation test + verdict, split out of the tuner so it is NOT gated on era 0 with it - see
//--- the definition. Read-only; runs once per attach, whether or not the sweep did.
void ReportFeatureLabelInformation ( void ) ;
2026-08-22 00:24:45 -04:00
//--- Smallest BarsCalculated() across the ENABLED tunable indicators, or -1 when none is on. The
//--- tuner reports this so "the parameter change did not reach the features" can be told apart
//--- from "it reached them but they weren't ready".
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
int TunableBarsCalculated ( void ) { return m_featureBuilder . TunableBarsCalculated ( ) ; }
2026-08-22 00:24:45 -04:00
//--- Same number, plus HOW MANY tunable indicators were actually consulted.
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
int TunableBarsCalculated ( int & enabled ) { return m_featureBuilder . TunableBarsCalculated ( enabled ) ; }
fix(indicators+panel): the dead handle is MEASURED now - recreate it; and order the ensemble panel by member, not by who published first
THE ANSWER, off the instrumentation added in be39674, first run:
ConvLSTM [HYB-2484]: TUNABLE INDICATOR REPORTS NO CALCULATED BARS - 1 tunable
indicator(s) enabled and the least-ready answers BarsCalculated()=-1 ...
Per-indicator depth: price=33982 MA=-1 ZigZag=33982 ATR=33982
MA=-1 with price, ZigZag and ATR all at full depth. **The handle is INVALID, not
short.** Same line on USDJPY (price=50179 MA=-1). Depth was never the problem;
the previous session's five theories were all answering the wrong question.
And it is per-member, not per-chart: LSTM-2484 ran the 34-candidate auto-tune on
that same chart at 15:24:18 and went on to train normally (feature health, 51
features, excursion head) reading the same indicator. Only ConvLSTM's handle -
the last member constructed - was dead. WHY is still not established. All four
members request ADMovingAverage with identical params, so MT5 hands them the SAME
refcounted handle, and the tuner's inner loop is Create-then-IndicatorRelease over
exactly that shared handle; that is the obvious suspect and it is NOT yet proven,
so this commit does not act on it.
1. IndicatorDepthReport() NOW PRINTS HANDLE NUMBERS, not just depths.
"MA=-1" says the handle is dead. "MA=-1(h12)" against another member's "MA=33982
(h12)" says it is the SAME handle and someone released it; "(h-1)" says it was
never created. That is the difference between a refcount bug and a creation
failure and it is one field. This is the measurement the shared-handle suspicion
needs before anyone acts on it.
2. RECREATE A DEAD HANDLE INSTEAD OF SWEEPING AGAINST IT.
A member that cannot read its own indicator must rebuild it. RepairDeadIndicatorHandles()
re-Creates only the ENABLED tunables reporting BarsCalculated() < 0 - a merely COLD
indicator (valid handle, 0 bars) is left alone to warm up the normal way. It does
NOT release first: -1 means the terminal no longer knows the handle, so there is
nothing to give back, and MT5 recycles handle VALUES so releasing a stale one could
decrement whatever now owns that number. 30s cooldown, because every ServableBars()
consumer reaches it including live inference on every tick. The feature cache is
dropped with it, and the log names before/after depths.
Cause-agnostic on purpose. Whatever is killing the handle, sweeping 50,163 bars
against a buffer that answers EMPTY_VALUE at every index - then discarding the era
and doing it again - is not a recovery.
3. THE SWEEP NOW HOLDS ON A DEAD HANDLE.
ServableBars() keeps answering `want` (its contract; live inference and online
learning have their own refusal paths and a 0 there reads as "no history at all").
SettledBars() - the training sweep's entry, the one caller that can afford to wait -
returns 0 instead, so Train() holds and reports rather than burning a full-history
pass it is guaranteed to throw away. A recreated handle is cold, so it primes
through the existing settle path on the next call. If the repair fails the member
holds indefinitely and says so every minute, and be39674's barrier liveness escape
releases the rest of the ensemble after 12 minutes - which is the correct
degradation and is exactly what the log shows happening.
4. THE PANEL ROWS WERE ORDERED BY WHO PUBLISHED FIRST.
Reported on XAUUSD: LSTM, ConvLSTM, Perceptron, Convolutional instead of
Perceptron, Convolutional, LSTM, ConvLSTM. ClaimEnsemblePanelSlot() handed out the
next free row on each member's FIRST PublishStatus() call, so the order was a race -
the members busy sweeping published before the ones sitting idle at the era barrier,
and be39674 sharpened it by (correctly) making a held member stop writing the terse
line. Rows are now keyed to m_ensembleIndex, the registration/construction order,
which is fixed for the life of the chart. Claimed on every publish rather than once,
so it is idempotent and refreshes the tag for a member whose ID was not final when it
first published (the config-tag suffix is appended during InitIndicators, after
EnsembleMember() registers). Unclaimed rows are skipped by the render and excluded
from the model count, so a member that has not published yet leaves no gap and shifts
nobody.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:40:00 -04:00
//--- Re-Create any ENABLED tunable indicator whose handle the terminal no longer recognises
//--- (BarsCalculated() < 0). Returns true when something was actually rebuilt.
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
bool RepairDeadIndicatorHandles ( void ) { return m_featureBuilder . RepairDeadIndicatorHandles ( ) ; }
2026-08-22 00:24:45 -04:00
//--- `want`, clamped to what the indicators can actually serve. THE single gate in front of
perf(tester): skip the signal DB in tester/optimizer, drop ExportFeaturesOnly
Two removals of work that a backtest was paying for and never using.
1. SignalDatabaseActive() gates the signal DB off in tester/optimizer.
A backtest opened the fingerprinted SQLite DB under FILE_COMMON - and so
did every parallel optimization agent, against the same file, with the
per-tick journal Update() behind them. Measured 2026-08-25 on a 12-agent
SP500 H4 run: zero passes completed in 75 minutes.
It bought nothing, for a reason specific to this EA's current shape: the
DB's only effect on a trading decision is ApplyPatternWeight overriding a
filter's module weight, and that is declined for any self-ranking filter
(CExpertSignalCustom's !filter.SelfRanked() guard). The AI members
self-rank once their tiers are measured, and the classic votes that DID
consume the ranking are gone - so a tester run's DB was written and never
read. Skipping it changes no decision.
One predicate, not two inline guards: OnInit asks the question twice
(InitDatabaseAndJournal, then VerifyDatabaseTransactionCycle) and a run
where those disagreed would try to open a database it never initialised.
The tester now takes journal.InitTrackingOnly(), so close detection,
MAE/MFE and the expectancy-stop feed still run - only the SQLite half is
dropped, and Update() already skipped its INSERT when there is no DB.
Caveat recorded at the predicate: if a future filter consumes DB ranking
WITHOUT self-ranking, this needs revisiting - a backtest would then stop
reproducing live.
2. ExportFeaturesOnly and its two exporters are gone.
Research-only CSV dumps (feature matrix + a hardcoded 8-symbol x 5-TF raw
rates grid), superseded by the research/ python path that reads its own
data. Removed the input, m_exportFeaturesOnly, the setter, both method
declarations, ExportFeatureMatrix()/ExportRawRates() (111 lines in
AutoTune.mqh), the OnTick early-return, and the ctor initialiser.
The config-lock bypass it owned collapses to the plain tester test:
`if(!inTesterOrOpt && !AcquireConfigLock())`. Shared helpers it called -
ServableBars, EnsureBarCachesCapacity, ResizeBuffers, RefreshData - all
have other callers and are untouched.
Compile-verified in _claude_stage: 0 errors, 0 warnings, identical to the
baseline taken before either edit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 09:32:44 -04:00
//--- every ResizeBuffers() call site (train, live inference, chart rescan).
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
int ServableBars ( int want , string context ) { return m_featureBuilder . ServableBars ( want , context ) ; }
2026-08-22 00:24:45 -04:00
//--- ServableBars() with a WAIT in front of it, for the paths that can afford one (the training
//--- sweep and the label prebuild). Returns >0 = the depth to use, or 0 = "not settled, come
//--- back later".
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
int SettledBars ( int want , string context ) { return m_featureBuilder . SettledBars ( want , context ) ; }
2026-08-22 00:24:45 -04:00
//--- Per-indicator BarsCalculated(), for the cap/priming/stall lines. It answers that directly.
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
string IndicatorDepthReport ( void ) { return m_featureBuilder . IndicatorDepthReport ( ) ; }
fix(indicators): a dead handle and a priming one both read -1, so the repair report proved nothing
The detector claimed "-1 means an INVALID HANDLE, 0 means created-but-never-
calculated". This run disproved it with our own instrumentation: the repair
line prints only when Create() RETURNED TRUE, and the depth it read
microseconds later was
BEFORE: MA=-1(h13) | AFTER: MA=-1(h13)
A freshly created, valid handle read -1 - the value the model says is
impossible for one. So BarsCalculated() < 0 does not mean "dead"; it also
covers "valid, not calculated yet", and the trigger cannot separate them.
Consequences, all fixed here:
- The AFTER depth was re-read synchronously, when it can only be -1 or 0, so
every repair looked like a failure and the line was unreadable either way.
It now reports the handle NUMBER across the recreate instead. A changed
number proves a new instance; SAME means MT5 handed back the same
refcounted one, so it was never dead.
- IndicatorDepthReport printed the handle number for MA alone. Every tunable
gets one now, through a single IndicatorDepthField() - nine near-identical
StringFormat calls collapse to one.
- The comment justifying "never release before re-creating" rested on the
claim just disproved. The decision stands, the reason is restated: given
the ambiguity, releasing is the dangerous half - a recycled number would
decrement whatever owns it now and CAUSE this outage - while re-creating a
live handle only leaks a reference on a path that fires a few times a
session.
The before-handle is captured on its own line, never as a sibling argument to
the Init* call: MQL5 does not define argument evaluation order.
Behaviour is otherwise unchanged - same trigger, same cooldown, same
recreate. Only what gets reported changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 13:38:02 -04:00
//--- One field of that report, and one line of the repair report. Both print the handle NUMBER:
//--- a depth of -1 cannot separate "freed under this member" from "created but not calculated yet",
//--- and the number can.
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
string IndicatorDepthField ( const string name , const int depth , const int handle )
{ return m_featureBuilder . IndicatorDepthField ( name , depth , handle ) ; }
int NoteHandleMove ( const string name , const int oldHandle , const int newHandle , string & moves )
{ return m_featureBuilder . NoteHandleMove ( name , oldHandle , newHandle , moves ) ; }
2026-08-01 14:01:32 -04:00
bool m_miReportDone ;
2026-08-02 12:25:20 -04:00
//--- Eras the MI report has waited for the cross-asset panel to exist, so it describes the SAME
//--- feature vector training uses. Bounded, so a terminal that never syncs the reference symbols
//--- still gets its diagnostics rather than silently getting none.
int m_miReportDeferrals ;
feat(pool,mi): one feature layout fleet-wide, and the keep-screen stops self-disabling on a cold start
TWO CHANGES, BOTH RETRAIN-FORCING BY INTENT.
1. SP500 was training alone, and one alt-data column was the reason.
The alt block's width joins the model fingerprint, and the pool reader only
adopts peer rows whose fingerprint and width match. The exporter gives each
instrument the series that apply to it - FX 15 columns, metals/oil 14, SP500
13 - so the fleet ran as three incompatible pools:
EURUSD/USDJPY/USDCAD adopt ~57-60k peer rows each
XAUUSD/XTIUSD adopt 6.4k / 20.3k
SP500 "EVERY peer file was REJECTED, so this chart is
training alone" - 0 rows
SP500 therefore trained on 2279 independent observations against a 600-wide
input with its first layer floored at 16, printing its own "expect
overfitting" warning. It is the one chart with no pool and the worst
capacity ratio in the fleet by a factor of three.
Fresh models now pin ALTDATA_FLEET_COLUMNS - the 12-column intersection -
instead of their own file header. An existing model still adopts its .cfg
pin, so this re-keys nothing that is already trained.
Intersection rather than union: filling an absent series with its median
makes that column constant per instrument, which lets a pooled model
identify the source instrument and stop learning the shared mechanism. It
is also 6 columns narrower. Cost is six columns whose retained information
is UNMEASURED - the keep-screen reports a bitmask nothing has mapped back
to names.
2. The MI keep-screen disabled itself for the whole run on any cold start.
ReportFeatureLabelInformation set m_miReportDone on ENTRY. On a cold start
the label cache is allocated before it is filled, so BuildMiSample finds no
row carrying a resolved label and returns 0 - a sixth exit, and the only
one the 8c1266d instrumentation did not cover, which is why it printed
nothing. observed then stayed -1, the permutation loop never iterated, and
the report emitted "-1.00000 nats over 0 permutations" beside a plausible
"strongest single feature 0.05979" that was a STALE m_miBestColumn from an
earlier scoring call. The first ensemble member propagated the latch to
g_ensembleChartMiReportDone and silenced every member on the chart.
The flag now latches only once a measurement exists. A short sample is
reported as a deferral naming the two numbers that identify it (cached bars
vs bars carrying a resolved label) and retried, up to
MI_REPORT_MAX_ATTEMPTS.
Build tag -> fleet-pool-v1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 17:41:14 -04:00
//--- Eras on which the screen ran and could not build a usable sample. Distinct from the
//--- deferrals above, which count eras it declined to run at all (cross-asset not ready).
int m_miReportAttempts ;
//--- How many cached bars actually carry a resolved label. The MI screen's sample is drawn only
//--- from these, so on a cold start the cache can be large and this zero - which is the whole
//--- cold-start failure, and printing the two side by side is what makes it self-evident.
int LabelCacheResolvedCount ( void ) const
{
int n = 0 , total = MathMin ( m_labelCacheBars , ArraySize ( m_labelCacheHasValue ) ) ;
for ( int i = 0 ; i < total ; i + + )
if ( m_labelCacheHasValue [ i ] )
n + + ;
return n ;
}
diag(autotune): one label shuffle cannot settle the no-edge question
The permutation baseline added in 018afb1 came back on all four charts as
0.00401 nats against floors of 0.00267 / 0.00298 / 0.00318 - three draws
whose spread is as wide as the excess being judged, because one shuffle
is one sample from the null, not the null. That is not enough to retire a
topology on.
Now MI_NOISE_PERMUTATIONS draws, reported as mean +/- sd with a z-score,
plus two numbers the mean over 26 columns cannot express:
- the STRONGEST single feature's MI, against its own shuffled value.
One informative column among 25 useless ones is precisely the case
the mean hides, and precisely the case worth finding.
- the excess as a percentage of H(Y). At these sample sizes a z-score
can be comfortably significant while the effect is worthless, so
"is it real" and "is it big enough to matter" are asked separately
and answered separately.
The verdict line also now states the measure's limit every time rather
than only when the news is bad: this is a MARGINAL, PER-BAR statistic and
the network reads m_historyBars bars jointly, so it can prove signal
exists but never that it does not. It rules out a per-feature edge - and
therefore any indicator retuning - not an edge that lives in a
combination or across time.
Compiles 0 errors / 0 warnings, standard and Market.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:32:12 -04:00
double m_miBestColumn ;
double m_miLabelEntropy ;
feat(features): per-column MI keep-screen (report only)
Step 1 of the prune, stopping deliberately short of pruning - two blockers make
an immediate mask the wrong move, and this is the measurement that decides
whether pruning is worth doing at all.
WHY NOT PRUNE YET:
* the screen runs with cross-asset ABSENT - its own log line says the numbers
"describe a NARROWER vector than training will use". A mask built from it
would have no evidence either way about the cross-asset block.
* a per-chart mask FRAGMENTS THE POOL. The mask must participate in the
fingerprint, and the pool only accepts peers with an identical feature
layout. Pooling is currently the only thing keeping the FX trio off the
capacity floor - the three pool-poor charts (SP500, XAUUSD, XTIUSD) are
exactly the three still floored. Six per-chart masks = six pool groups of
one, and pruning could cost more capacity than it buys.
WHAT THIS ADDS: the per-column MI was always computed inside ScoreMiSample and
thrown away except for the sum and the max. It is retained now, and the same
permutation draws that build the headline null also accumulate a PER-COLUMN null,
which is what a per-column p-value needs - distinct from the null-of-the-max,
which answers the single family-wise question "is the strongest column real".
Selection uses Benjamini-Hochberg at q=0.10, NOT the family-wise bar. FWER
controls the chance of one false positive, which is right for a verdict and far
too conservative for selection - it would discard every genuinely weak-but-useful
feature. BH bounds the expected SHARE of kept columns that are noise, which is
what a feature set cares about.
The report prints the decision in capacity units: columns kept, the resulting
input width, and the first-layer budget before and after against the 16-wide
floor. 3 of 52 is not a feature set; 45 of 52 is not worth a fingerprint re-key.
The cross-asset caveat prints itself when it applies.
Context that makes this worth doing at all: under the pivot-event label the MI
screen now reads "above the noise floor - a real association" - mean 4x the null
(p=0.005), strongest column 7.7x the null-max, excess 0.80% of label entropy,
against 1.3x / 1.15x / ~0.1% under the old label. The noise-floor verdict that
closed several earlier directions was a property of the OLD label.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 15:43:59 -04:00
//--- PER-COLUMN MI from the most recent ScoreMiSample() call, indexed 0..m_neuronsCount-1. The
//--- per-column loop always existed inside ScoreMiSample and threw every value away except the sum
//--- and the max; retaining it is what lets the screen say WHICH columns carry the association
//--- rather than only that one of them does.
double m_miColumn [ ] ;
diag(autotune): a positive control, and a scan that separates "no signal"
from "signal knocked out of step"
Four architecturally different networks landed on the same precision -
Buy 23-25% against a 25.4% base rate, Sell 19-22% against 22.0% - while
making completely different calls (HYBRID votes Sell on 69% of bars, PAI
on 41%). Precision equal to the base rate is what INDEPENDENCE looks
like, and precision under independence is fixed by the label
distribution, not by the architecture, so all four converging on it is
arithmetic rather than coincidence. Accuracy meanwhile tracks coverage
exactly as independence predicts (31.1/30.3/25.0 predicted vs
31.8/28.9/24.6 observed for PAI/CONV/HYB).
But "no information in the data" and "information destroyed upstream of
every topology" produce that identical picture, and the MI test alone
cannot tell them apart either. Two additions:
POSITIVE CONTROL. Three "measurements" in this codebase have turned out
to be silent no-ops that produced plausible numbers - the MI scorer
reading an array nobody filled, the eval-mode guard that switched off the
imbalance correction, the alternation gate whose premise was never true.
So the estimator now has to prove it responds to a signal known to be
present before any floor reading is believed: the label of a neighbouring
sample row, ~19 bars away and far inside the 128-bar barrier horizon, so
the two outcome windows overlap heavily and MUST be associated. Same
binning, same estimator. Near the floor => every MI figure is void.
ALIGNMENT SCAN. Re-scores against the label taken from bar i+k for k in
-5..+5. A peak at k != 0 is a feature/label misalignment - an off-by-one
in the label index, a horizon applied to the wrong bar, a feature window
that lags what it claims - which would destroy the information before any
topology saw it and would look identical in every accuracy number this EA
prints. A flat profile says the features simply do not carry this target.
The sampled range is trimmed by |k| at both ends so a shift is measured
rather than an edge effect, and both bars must carry a real label.
Also: BuildMiSample publishes its stride instead of the report
recomputing that arithmetic (it would drift), and the control sizes its
buffers from its own sample count rather than the caller's.
Compiles 0 errors / 0 warnings, standard and Market.
Build tag mi-control-align-v1. Redeploy only - no retrain, no model
deletion; the diagnostic runs on resumed models.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:10 -04:00
//--- bars between two consecutive MI sample rows, set by BuildMiSample - see its note.
int m_miStrideBars ;
2026-08-24 21:01:08 -04:00
//--- independent label blocks the permutation null was built from (rows within one label resolution
fix(diag): the symbol sweep was measuring its own sampling, not the market
Twelve cells came back with higher-timeframe "signal" 5-9x anything on
H1, at p=0.005. It was an artifact, and the sweep's own columns gave it
away: excess tracked the sampling STRIDE almost monotonically, and the
three D1 cells - stride collapsed to 1-5 bars against a 128-bar horizon,
i.e. ~99% window overlap - were the three highest. Three flaws, all the
same family: comparing numbers without the spread that belongs to them.
1. THE NULL ASSUMED INDEPENDENCE THE LABELS DO NOT HAVE. Triple-barrier
labels overlap; two rows less than one horizon apart share most of their
outcome window. A free Fisher-Yates shuffle destroys that dependence
along with the association, making the null far narrower than the truth
and handing out significance that isn't there - Lopez de Prado ch. 4
arriving through the back door of the significance test. Now permutes
contiguous BLOCKS of at least one horizon, so the null keeps the
autocorrelation and the p-value means what it says. It degrades honestly:
severe overlap leaves few blocks, the null widens, nothing reaches
significance. The block count is now printed, because THAT - not the row
count - is the sample size a p-value rests on, and a warning fires under
30 blocks so "not significant" is not misread as "no signal" when it
means "not enough independent history to tell".
2. THE POSITIVE CONTROL'S STRENGTH DEPENDED ON THE DATASET. It paired
each row's label with the NEXT SAMPLE ROW's, whose distance is the
stride - so on M5, where stride ran 160-717 bars against a 128-bar
horizon, it was pairing two windows that never overlap. All three M5
cells duly reported a FAILED estimator and voided their own results with
nothing wrong. A control whose strength varies with the cell cannot
certify the cell. Now pinned to a quarter of the horizon, where ~75%
overlap is guaranteed by construction.
3. THE LOOKAHEAD VERDICT HAD NO MARGIN. It flagged 7 of 12 cells on gaps
of 0.00008-0.00040 nats against a measured null sd of ~0.00030 - noise,
every one. Now requires 3 sd, the same discipline the deploy floor
applies to precision.
Compiles 0 errors / 0 warnings, standard and Market. Build tag
blockperm-v1. Supersedes every number from the sweep.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:11:40 -04:00
//--- move together, so THIS - not the row count - is the sample size the p-value really rests on).
int m_miNullBlocks ;
perf(autotune): replace the genetic search with a filter score - hours to seconds
MEASURED COST OF THE GA, which is what retired it. Per generation:
rung 0: 8 cand x 3 seeds x 3 eras = 72 eras
rung 1: 4 cand x 3 seeds x 8 eras = 96
rung 2: 2 cand x 3 seeds x 20 eras = 120
= 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's
real training began. Against the observed era times on SP500 H1:
PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22)
CONV 41.3 s/era -> 13.2 h
LSTM 150.4 s/era -> 48.1 h
HYBRID 154.6 s/era -> 49.5 h
Two days to tune is not a first-run experience, and it is the phase in
which the panel goes quiet, which is what made it look like a hang.
It also bought nothing. The space is 90 points (10 MA periods x 9 MA
types), so 1152 evaluations revisited each point ~13 times; and rungs of
3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run
proves it: every finalist scored 25.0-25.9% balanced accuracy - below the
33.3% one-class floor, i.e. indistinguishable noise - and the search then
"deployed the winner" of that.
THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full
training run to choose a feature's period is a wrapper method paying
wrapper prices for a decision that does not need one. The reference book
does not do this: ch. 3.3 selects inputs by measuring each candidate
indicator's CORRELATION with the target and dropping the ones with none,
with no network involved.
So: rank candidates by the MUTUAL INFORMATION between the resulting
feature vector and the triple-barrier label. MI rather than correlation
because the label is 3-class categorical and the features are not
monotonically related to it. Equal-FREQUENCY binning (rank-based),
because these features are ATR-normalised and heavy-tailed - fixed-width
bins put nearly everything in one bucket and report ~0 information for a
genuinely useful feature.
Scoring is arithmetic over the feature cache, so it costs seconds and its
cost is independent of topology: LSTM now tunes as fast as the MLP.
Coordinate sweep, not product sweep - cost is the SUM of per-parameter
candidate counts, so enabling every indicator stays affordable - with a
second pass that breaks early once nothing moves.
Sampling is IS-ONLY. Letting the OOS window influence which indicator
settings ship would mean the holdout had been used for selection and had
stopped being a holdout.
HONEST LIMIT, recorded because it is the price: MI is marginal, so a
parameter that only pays off in combination with another can be missed
(Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it
replaces was ranking pure noise at 48 h a run, this is strictly better.
Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/
GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga*
members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget.
AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28
read sites all permanently inert. That is not a tidy-up: the `if
(!m_evalMode)` guard on UpdateClassPriors is exactly what silently
disabled the imbalance correction for entire runs two commits ago. Dead
machinery that still reads like live machinery is this codebase's most
expensive recurring bug, and leaving 28 more instances of it would have
been indefensible.
The panel's tuning-progress state goes too - tuning no longer takes long
enough to need one.
Both builds compile 0 errors / 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
void TuneIndicatorsByFilter ( void ) ;
feat: add unified MA type support to indicator tuner
Add `MA_TYPE_PRESETS` enum covering advanced (ALMA, DEMA, ZLEMA, T3, Kalman) and standard (SMA, EMA, SMMA, LWMA) moving averages. Integrate `maType` and `bestMaType` into `CADIndicatorTuner` struct, update flatten/unflatten routines, and bump `AD_TUNE_PARAM_COUNT` to 33. This allows the auto-tuner to search over MA type alongside period, improving feature discovery.
2026-07-23 15:02:09 -04:00
//================================================================================================
2026-07-14 22:36:27 -04:00
void FinalizeTrainRun ( void ) ;
2026-07-25 16:39:11 -04:00
//--- The "this is now THE model" persistence sequence, shared by every deploy path so they can't
2026-08-22 00:24:45 -04:00
//--- drift apart: weights (carrying the current m_trainingComplete flag), the pure-MQL5
//--- inference self-check, the calibration sidecar, and the EMA shadow.
2026-07-25 16:39:11 -04:00
void PersistDeployedModel ( void ) ;
feat(train): ONE pass over the held-out slice at deploy, on the restored checkpoint
The OOS slice is the newest history and the model never trains on it, while
online learning adapts to every bar resolving AFTER deployment. That leaves a gap
exactly at the handover, over the most regime-relevant data there is. This closes
it: select on validation, then refit on everything, which is standard practice.
Placed AFTER Net.RestoreWeights() and ResetOptimizerState() and BEFORE
PersistDeployedModel(), so it refines the weights that were actually SELECTED
rather than whatever the run happened to end on, and what it produces is what
gets written down.
THE COST IS REAL AND IS NOW STATED IN THE LOG. The deploy line promises "every
model reverts to the weights it held at the era whose combined vote scored best,
so the ensemble that trades is exactly the one that was measured". After this
pass that is no longer literally true, so the pass prints that the certified
numbers belong to the PRE-PASS weights and must be quoted that way. Set
EnableOosFinalPass=false to keep certified == traded exactly.
Guards:
* ONE-SHOT PER RUN, and the flag is set BEFORE the loop so no early return inside
it can leave the pass eligible to fire twice over bars it already trained on.
Reset at m_trainRunActive=true, because a retrain is a fresh selection and
earns a fresh pass.
* THE CONVERGED RATE, never a plateau-boosted one: m_modelEta can still carry
PLATEAU_RESTART_BOOST from an escape attempt, and this is a refinement of a
selected model, not another warm restart. g_eta is what backProp reads, so that
is what is capped and restored.
* OLDEST -> NEWEST. Series indices count backwards, so decreasing i moves forward
in time - the order the bars happened in.
* A failed feedForward is never followed by backProp; the output layer would
still hold the previous sample's activations and the update would be this bar's
label against another bar's prediction.
* m_oosFinalPassCutoff records the newest bar consumed and is deliberately NOT
cleared on a new run, so a later run can say plainly that its out-of-sample
window reaches back into bars this model has already seen.
Expect the gain to come from CURRENCY rather than finer weights: OOS precision
was measured flat from era 20 while in-sample error kept falling, so the data
this model can already see is exhausted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 11:05:43 -04:00
//--- See EnableOosFinalPass. Returns how many bars it trained on; 0 when it did nothing.
int OosFinalPass ( void ) ;
2026-07-22 13:33:56 -04:00
//--- On hitting the per-run era cap: asks the operator whether to keep training (true) or deploy
//--- the best checkpoint and stop (false). Headless (tester/optimizer) can't show a dialog, so it
//--- returns false. See m_maxErasPerRun's declaration comment.
bool PromptContinuePastEraCap ( double bestOos ) ;
2026-07-14 22:36:27 -04:00
//--- variables
//--- training control, driven by the control panel (Warrior_EA.mq5); Train()/OnTickHandler
//--- poll these rather than being torn down/rebuilt, so pausing/stopping never loses in-memory state
bool m_trainingPaused ; // true: Train() blocks between eras until unpaused
bool m_trainingStopRequested ; // true: OnTickHandler stops scheduling new training passes
2026-08-22 00:24:45 -04:00
//--- true for ANY Strategy Tester run - a single backtest AND every optimization pass
//--- (MQL_TESTER): the run must NEVER train. Training + online continual learning happen only on
//--- a live chart, where this is false.
feat: add unified MA type support to indicator tuner
Add `MA_TYPE_PRESETS` enum covering advanced (ALMA, DEMA, ZLEMA, T3, Kalman) and standard (SMA, EMA, SMMA, LWMA) moving averages. Integrate `maType` and `bestMaType` into `CADIndicatorTuner` struct, update flatten/unflatten routines, and bump `AD_TUNE_PARAM_COUNT` to 33. This allows the auto-tuner to search over MA type alongside period, improving feature discovery.
2026-07-23 15:02:09 -04:00
bool m_inferenceOnly ;
2026-08-22 00:24:45 -04:00
//--- true only when the current Net weights came from a saved .nnw on disk, not from a freshly-
//--- built random topology.
2026-07-27 11:28:23 -04:00
bool m_modelLoadedFromDisk ;
2026-08-22 00:24:45 -04:00
//--- Set by EnforceTopologyContract() when a just-loaded .nnw was built by a superseded
//--- architecture that cannot be repaired in place (currently: a different conv receptive field,
//--- whose weight tensor is a different SHAPE).
feat(ai): real conv receptive field + the reference's channel pool
CONV's convolution used window = step = one bar, which is a per-bar
projection - a 1x1 conv with a temporal receptive field of ONE BAR. It never
mixed information across time, so "convolutional" described the layer type
and nothing about what it computed. Same finding that sank HYBRID's LSTM.
Pooling was removed on 2026-07-29 for being misconfigured against the conv
output's memory layout. That removal was right; leaving the conv at a
one-bar window was not. The two belong together: the NeuroNet_DNG reference
(references\MQL5\Experts\EDL\Trajectory.mqh layers 2-5, kernels
byte-identical to ours) pairs conv(window=2, step=1, window_out=4) with
pool(window=4, step=4), and the pool only earns its place because a conv
with a real receptive field sits above it.
The input is bar-major (BufferTempData appends m_neuronsCount contiguous
features per bar), so a flat window of k*m_neuronsCount spans exactly k
bars - the receptive field needed NO kernel change. The conv output is
position-major, so window == step == window_out is a clean
max-over-channels, which is what the reference does and what the existing
pool kernels already implement correctly.
New chain at H1 defaults (420 = 20 bars x 21):
conv1 w=42 s=21 out=8 -> 19 pos x 8 = 152
pool w=8 s=8 -> 19
conv2 w=2 s=1 out=8 -> 18 pos x 8 = 144 (effective field: 3 bars)
We deliberately stop before the reference's SECOND pool: a channel pool
emits one scalar per position, so a trailing pool would hand the dense stack
18 values and force it to fan out 18 -> 64. That is a bottleneck below every
learnable layer - the same class of mistake the 2026-07-29 removal was about.
Fixes a latent sizing bug this exposed: CNet's conv/pool position cursor
tracked sliding POSITIONS, but a conv's real width is units_count *
window_out. Any pool stacked on a conv would therefore have sized against a
width window_out times too small and silently built the wrong shape. Both
branches now read the built layer's actual Neurons(), which is what the
batch-norm branch already did for the same reason.
Also closes the architecture-pinning trap: a .nnw persists the window each
conv was built with, so an existing CONV/HYBRID model would have loaded
cleanly and gone on training under the OLD architecture. The conv weight
tensor is (window+1)*window_out, so this cannot be repaired in place -
EnforceTopologyContract now detects it, reports both shapes, and retrains.
Conv chain shape is derived in one place (ConvReceptiveFieldBars /
ConvFirstStagePositions / HasSecondConvStage / ConvOutputPositions /
ConvOutputWidth) and consumed by AddConvStage, LstmFanIn and the startup
config line, so what is built and what is logged cannot drift.
Both builds compile 0 errors, 0 warnings. Forces a CONV and HYBRID retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 20:05:37 -04:00
bool m_topologySuperseded ;
feat(topology): re-derive capacity once when the training pool appears
A cold fleet start sizes every model BEFORE any chart has published a pool
file, so the first layer is budgeted as if the chart trains alone and then
pinned to .cfg. This is not a rare race - it is what happens EVERY time the
feature layout changes, because that invalidates the pool and forces a wipe.
Correcting it by hand needs a two-phase start: run the fleet to fill the pool,
stop, wipe the weights while KEEPING the pool, restart so derivation sees it.
That is not something an unattended fleet can do for itself, and getting it
wrong is silent - the models simply stay narrow.
TuneIndicatorsAndTrain now notices that the pool has appeared and re-derives
once, reusing ResetWeights() - the existing tested path that re-measures all
four sizes, rebuilds and rewrites the .cfg. No second copy of that logic.
Bounded on every axis that could make it a loop:
- once per model (the flag is set BEFORE the reset, because ResetWeights
zeroes m_eraCount and the model would otherwise re-qualify forever)
- only while era <= CAPACITY_RESIZE_MAX_ERA, so the discarded eras are worth
nothing
- only on CAPACITY_RESIZE_MIN_GROWTH real growth
- only if the recomputed width actually differs; if it does not, the check
settles itself rather than re-running the census every era
Safe against the one thing that would make it self-defeating: the derived width
is NOT part of BuildModelFingerprint, so a model that resizes does not leave
the pool it resized for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 21:42:35 -04:00
//--- CAPACITY RESIZE (2026-08-26). What TopologyPooledIndependentBars() returned at the moment
//--- this model's first-layer width was derived, and whether the one-shot re-derive has run.
//---
//--- ComputeFirstLayerWidth budgets against own bars PLUS the training pool. On a COLD FLEET START
//--- every chart derives its topology before any chart has published a pool file - measured
//--- 18:13:21 against a first publish at 18:13:48 - so all six size as if training alone and pin
//--- that. It is not a rare race: it is what happens EVERY time the feature layout changes, since
//--- that invalidates the pool and forces a full wipe. Correcting it by hand needs a two-phase
//--- start (run, stop, wipe weights but keep the pool, restart), which is not something an
//--- unattended fleet can do for itself.
double m_poolObsAtDerivation ;
bool m_capacityResizeDone ;
2026-08-22 00:24:45 -04:00
//--- true once ValidateCpuInference() has confirmed this model's pure-MQL5 forward pass matches
//--- the compute backend's within tolerance (see CNet::SetCpuInference). Measured at deploy on
//--- the chart (where a backend exists to compare against), never in the tester itself.
2026-07-24 11:52:19 -04:00
bool m_mqlInferenceValidated ;
2026-07-14 22:36:27 -04:00
string m_fileName ;
string m_folderPath ;
//--- which file Train()'s Net.Save() calls (and this method's own Net.Load()) actually target:
//--- the shared FILE_COMMON production weights normally, or a LOCAL per-agent cache file when
//--- running inside the Strategy Tester/optimizer (see InitNeuralNetwork) so that repeated
//--- optimization passes with an unchanged topology can reuse an already-trained model instead of
//--- re-running every era from scratch, without ever touching the live production .nnw/.cfg.
string m_activeFileName ;
bool m_activeFileCommon ;
2026-07-18 14:56:41 -04:00
//--- user-settable via Inputs.mqh's TrainingOptimizer (SGD or ADAM), read into this member at
2026-08-22 00:24:45 -04:00
//--- construction.
2026-07-14 22:36:27 -04:00
int m_optimizationAlgo ;
int m_historyBars ;
2026-08-22 00:24:45 -04:00
//--- Input-window derivation for a NEW model (existing models adopt theirs from the .cfg):
//--- median confirmed swing leg from raw highs/lows - strict local extrema over
fix(topology): size the network against observations, not bars
The capacity budget is stated in weights per INDEPENDENT observation
and divides by the mean label lifespan to get there. It never once
did: EstimatedInSampleBars() deflates via m_labelOverlap, but it is
only ever called from InitNeuralNetwork, where the label cache does
not exist yet (that same function sets m_labelCachePrebuilt = false
a few lines below), so MeanLifespan() returned its "nothing measured"
default of 1.0 at every call. Every fresh model was sized as though
its labels did not overlap - over-budgeting the first dense layer by
a factor of L, which is several rungs of a power-of-two ladder. The
"expect overfitting, reduce the feature set or pool instruments"
warning is the branch that should fire on H1 and structurally could
not.
Fixed at the source rather than by reordering the boot sequence (the
prebuild is chunked across Train() calls and cannot complete inside
init): MeasureSwingGeometry() walks the ZigZag ONCE at init and
answers both questions from it - the median leg gives the window,
and the leg series gives the mean label lifespan analytically.
SwingPivotDirectionLabel resolves bar i when the SECOND pivot after
it commits, so a bar d bars before pivot P waits d + (the leg
leaving P); summed over every bar of every leg that is exactly the
mean the label walk accumulates.
That also closes the coherence gap the swing target opened: the
window was measured with a private +/-12-bar fractal while the label
aimed at ZigZag(12,5,3) pivots, so it was sized against a leg
distribution the label never used. One pivot source now, the
label's.
Also:
- ResetWeights() re-derives the shape. It rebuilt from the members a
history-starved init had pinned and re-saved them - so the "let
history download, then reset from the panel" advice in both
fallback warnings did nothing at all.
- The CAPACITY line prints the measured lifespan beside the one the
topology was sized for, and warns when they differ by more than a
ladder rung. That is the check that makes the estimator falsifiable.
- Topology reads the view's symbol, not _Symbol (latent for pooling).
- Unmeasured geometry defaults to HISTORY_BARS_FALLBACK, never 1.0:
under-sizing is recoverable, over-sizing silently is not.
Compile: 0 errors, 0 warnings (stage).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 21:21:05 -04:00
//--- Median ZigZag leg - the LABEL'S own pivots - snapped down to {12,16,20,24,32}. The same walk
//--- measures the mean label lifespan the capacity budget divides by, so this must be called
//--- BEFORE ComputeFirstLayerWidth/ComputeLstmHiddenSize, exactly as InitNeuralNetwork orders it.
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- Body in Expert\Topology\Topology.mqh.
int DeriveHistoryBars ( void ) { return m_topology . DeriveHistoryBars ( ) ; }
2026-07-14 22:36:27 -04:00
int m_outputNeuronsCount ;
int m_minNeuronsCount ;
int m_initialNeuronsCount ;
int m_neuronsCount ;
double m_neuronsReduction ;
int m_hiddenLayersCount ;
2026-07-22 22:51:04 -04:00
//--- LSTM-only recurrent hidden-unit count - see LstmHiddenSize's declaration comment
//--- (Variables\Inputs.mqh). Harmless, unused constant contribution to m_fingerprint for MLP/CONV.
int m_lstmHiddenSize ;
//--- CONV-only convolutional output-filter count - see ConvFilterCount's declaration comment
//--- (Variables\Inputs.mqh). Harmless, unused constant contribution to m_fingerprint for MLP/LSTM.
int m_convFilterCount ;
2026-07-14 22:36:27 -04:00
int m_minTrainYear ;
bool m_isInitialized ;
fix(deinit): a full model write was running ahead of the cheap cleanup
"Abnormal termination" is back, and this time it is not the arrows. The
timing names the culprit exactly:
16:02:31.547 OnDeinit: shutting down
16:02:36.003 Abnormal termination <- 4.46 s, MetaTrader gave up
16:02:36.226 chart signals - persisted <- cleanup finished 0.2 s LATE
OnDeinit called StopTraining() BEFORE the chart cleanup. StopTraining()
finalises an in-flight run, and FinalizeTrainRun() restores the best
checkpoint and then persists it - a full ~1MB model write per signal. So
the expensive step ran ahead of the cheap bounded one, which is precisely
the inversion the shutdown ordering exists to prevent. The previous fix
put PersistWeightsOnShutdown last and missed that StopTraining smuggles a
second save in at the front.
Two changes:
Cleanup now runs FIRST, then StopTraining, then the weight save. The
visible teardown is cheap and bounded, so it always completes even when
everything after it is killed.
And the deploy-persist inside FinalizeTrainRun is suppressed during
shutdown. RestoreWeights() is an in-MEMORY swap, so the best checkpoint
is already the live net by that line, and PersistWeightsOnShutdown writes
exactly those weights moments later. The old path wrote the same model
twice per signal - eight full writes across four charts - for no benefit.
A user-pressed Stop still persists immediately, because nothing else
would.
Compiles 0 errors / 0 warnings. Build tag deinit-order-v2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 16:06:40 -04:00
//--- true once OnDeinit has begun - see MarkShutdown()/FinalizeTrainRun().
bool m_shutdownInProgress ;
2026-07-14 22:36:27 -04:00
int m_fractalPeriods ;
2026-08-20 09:54:23 -04:00
//--- The AI's four "market models", in the role a classic signal's geometric m_pattern_N members
2026-08-22 00:24:45 -04:00
//--- fill: ConfidenceTierFor() buckets a fire's RAW confidence into one of four equal bands
//--- between the head's structural floor (1/3 softmax, 0.5 regression) and 1.0, and the fire
//--- votes at that tier's weight.
2026-07-23 08:21:41 -04:00
int m_pattern_0 , m_pattern_1 , m_pattern_2 , m_pattern_3 ;
feat(ensemble): per-NN inputs replace the preset selector - the meta head becomes the vote's gate
User design (2026-08-19): 'remove the enum menu that selects neural networks... individual
inputs for every NN just like classic signals... the META NN should be integrated into the
voting decision pipeline when enabled... as a bonus meta labelling is applied to enabled NNs.'
- AI_CHOICE is GONE (tombstoned per the stale-.set doctrine). Use_MLP/Use_CONV/Use_LSTM/
Use_CONVLSTM are ordinary bools like the classic votes; the ensemble arithmetic adapts to
any subset because the consensus divisor is the enabled capable weight. Two or more
enabled = ensemble (|ENS1 token + joint gate, exactly the old AI_HYBRID fingerprints, so
existing weight files keep loading); one = the old solo preset; none = classic-only.
- Use_MetaLabeling un-couples META from the direction NNs (the old selector made them
mutually exclusive). S3 ships: CSignalMETA::LiveMetaGate scores each vote-cleared entry
(shared window at bar 1 + proposal descriptor: side, net vote, live geometry, spread/ATR;
pattern one-hot ZEROED - ranking, not calibrated probability, documented in the body) and
vetoes below the cost-adjusted break-even. Entries only; fail-open everywhere, loudly.
- COEXISTENCE HAZARDS closed: VoteCapableWeight()=0 and ProspectiveVote()=false for the
meta target - solo-only until today, a trained META would otherwise sit in the consensus
divisor as a permanent abstainer and shrink every vote by its module weight.
- CERTIFIED == TRADED: the ensemble era verdict replays the identical veto through the same
g_warriorMetaGate pointer over its OOS fired bars (bar re-resolved from the row's own
time; fail-open counted as fires and reported: 'metaGate: N approved, M vetoed, K
unscored'). The overlay deliberately does NOT replay it (veto-filter-in-replay class,
calendar-cliff precedent) - documented at the sweep site. Solo charts' own gate does not
model the veto - the standing solo-gate caveat, documented at the input.
- DB continuity: the pattern/journal DB fingerprint's first slot was (int)AIType;
DbLegacyAiSlot() maps every legacy-expressible config to its OLD value (new 2-3 member
subsets get 100+bitmask, outside the legacy range) so no existing database re-keys.
filterID becomes the enabled roster via one EnabledNNSummary().
- HUD: the meta line shows the gate (armed/(trn), last P vs BE, ok/veto tally); the
armed/disarmed announcement fires on state change via one latch (MetaGateArmedNow), not
only when an entry happens to be proposed.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 13:01:02 -04:00
//--- True when this signal runs as one of the ensemble's members - see EnsembleMember().
refactor(arch): a read-only training-data view, so modules stop being #included code
The AIBase\*.mqh files are not modules. They are method bodies of one
3,400-line class, textually #included after its declaration. Every one
of them can touch every member of every other, which is why "move this
out" has so far meant "move the whole class".
Introduce the seam that ends that:
CTrainingDataView abstract - the ONLY thing a training-side
collaborator may see: a feature row, a label, an
outcome, an excursion, the shape they share, and
the identity to log under.
CAIBaseTrainingData the adapter. MQL5 gives a class exactly one base
and CExpertSignalAIBase is already a
CExpertSignalCustom, so it cannot implement the
view itself. It owns one of these instead.
Data*() on the the published read API the adapter forwards to.
signal MQL5 has no `friend`, so reaching in from outside
was never an option - and making it explicit is
the point rather than a workaround.
Every row accessor OWNS ITS BOUNDS TEST and answers false for a bar it
has nothing for. Thirty-odd call sites currently carry their own
ArraySize() guard; one that forgets reads past a cache that is shorter
than the bar count for the whole warm-up. The -2.0 "never scored"
sentinel on the arrow cache is folded in the same way, so it can no
longer be mistaken for a small confidence.
Nothing uses it yet - this is the seam only, kept as its own commit so
the pattern compiles before 951 lines of Baselines move onto it. The
pattern is the stdlib's own: abstract base with =0 (Canvas\DX\DXObject),
concrete override, forward-declared owner pointer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:53:00 -04:00
//--- This class, seen as a CTrainingDataView. Owned by value: it is a pure forwarder with no
//--- state of its own beyond the back-pointer, so there is nothing to allocate or free.
CAIBaseTrainingData m_trainingData ;
refactor(baselines): the first real module - a class, not an #included partial
Baselines was 951 lines of CExpertSignalAIBase method bodies in a file
that only looked like a module. It is now CBaselineComparator: a class
the signal OWNS, which reads a CTrainingDataView and prints. It does
not name the signal anywhere in its code.
What the seam forced out into the open:
- Thirty-odd ArraySize() bounds tests, each carried by its caller, are
now one test per accessor next to the data. The two `hasValueN` and
one `arrowN` locals are gone with them.
- The -2.0 "never scored" sentinel on the arrow cache was tested at the
call site. It is now inside DataDirectionalCall, where it cannot be
read as a small confidence.
- DoubleToSignal needs m_outputNeuronsCount, so a raw double could not
be turned into a side by any reader. The view answers
DirectionalCall(bar, isBuy, magnitude) instead - the conversion
happens where the head width lives, and the module no longer needs
ENUM_SIGNAL at all.
- m_baselineDone was a latch on the signal for a decision only this
module makes. It is m_done, private, where it belongs.
Correction to my own earlier claim: I said Baselines had nine exclusive
members "polluting the signal class". It had none. m_x, m_f, m_ngrad,
m_AvgCE and the rest are FIELDS OF ALGLIB REPORT OBJECTS (state.m_x,
mrep.m_AvgCE) that my `\bm_\w+` scan matched after the dot. The module
needs no private state but its view pointer and that latch - which is
why it came out this cleanly.
The include sits below the g_ens* vote globals and the Alglib headers
it reads, because unlike the AIBase\*.mqh partials this is a real class
declaration compiled where it stands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 12:00:18 -04:00
//--- NON-NN BASELINES on this model's own matrix. A collaborator, not a mixin: it sees the view
//--- and nothing else, so it can be read, changed or dropped without touching this class.
CBaselineComparator m_baselines ;
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
//--- This class, seen as a CChartView. Owned by value for the same reason m_trainingData is.
CAIBaseChartView m_chartView ;
//--- Arrows, the status panel and the HUD line. A collaborator, not a mixin: it sees the view
//--- and nothing else - see Expert\Chart\ChartUI.mqh.
CChartUI m_chartUI ;
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
//--- This class, seen as a CPersistenceView (read+write, unlike CChartView). Owned by value for
//--- the same reason m_chartView is.
CAIBasePersistenceView m_persistenceView ;
//--- .cfg/.stats sidecars, CPU-inference validation, net-load retry. STATELESS - see
//--- Expert\Persistence\ModelPersistence.mqh's class comment.
CModelPersistence m_modelPersistence ;
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
//--- This class, seen as a COnlineLearningView. Owned by value for the same reason m_chartView is.
CAIBaseOnlineLearningView m_onlineLearningView ;
//--- Continual learning, the EMA shadow net, the OOS continual-learning simulation and the
//--- pattern-database backfill walk - see Expert\OnlineLearning\OnlineLearning.mqh's class
//--- comment. STATEFUL, like m_excursionHead: owns the shadow net and every walk's resume state.
COnlineLearning m_onlineLearning ;
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- This class, seen as a CTopologyView. Owned by value for the same reason m_chartView is.
CAIBaseTopologyView m_topologyView ;
//--- The fingerprint, the derived shape and BuildFreshTopology - see
//--- Expert\Topology\Topology.mqh's class comment. STATELESS, like m_modelPersistence.
CTopology m_topology ;
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- This class, seen as a CFeaturesView. Owned by value for the same reason m_chartView is.
CAIBaseFeaturesView m_featuresView ;
//--- Indicator lifecycle + the per-bar input feature vector - see Expert\Features\FeatureBuilder.mqh's
//--- class comment. STATEFUL, like m_excursionHead: owns the 10 feature-only indicator handles
//--- (m_Volumes/m_MA/m_RSI/m_MACDFeature/m_Ichimoku/the 5 AD* indicators) and the depth-probe/
//--- handle-repair/spread-series state directly.
CFeatureBuilder m_featureBuilder ;
2026-08-24 04:39:17 -04:00
//--- This class, seen as a CConfigLockView. Owned by value for the same reason m_chartView is.
CAIBaseConfigLockView m_configLockView ;
//--- The per-config chart lock - see Expert\ConfigLock\ConfigLock.mqh's class comment. STATEFUL,
//--- like m_excursionHead: owns m_configLockName directly.
CConfigLock m_configLock ;
2026-08-15 16:50:36 -04:00
bool m_ensembleMember ;
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
//--- Slot in g_warriorEnsemble (registration order, -1 = not an ensemble member). Doubles as the
//--- bit index in the combined-vote masks and the cursor index - see the registry's header comment.
int m_ensembleIndex ;
2026-08-15 16:54:43 -04:00
//--- This member's slot in the combined ensemble panel; claimed lazily on first publish (-1 = none).
int m_ensemblePanelSlot ;
2026-08-25 22:51:50 -04:00
//--- Dedup/throttle for the SOLO status label, mirroring PublishEnsembleStatus's own two guards
//--- (StatusLabel.mqh) - the ensemble panel had both, the solo path had neither: SetStatusLabel()
//--- was being called unconditionally on every PublishStatus(), unthrottled, on every tick,
//--- including inside the tester where nothing is ever drawn.
string m_soloStatusLastText ;
uint m_soloStatusLastRender ;
2026-08-22 00:24:45 -04:00
//--- functions Creates the OHLC + ZigZag indicators the feature builder reads. Called by
//--- InitNeuralNetwork(), never by the framework - the PUBLIC InitIndicators() override below is
//--- the framework entry point.
refactor(signals): AI signal files are identity + topology, nothing else
Every AI signal repeated the same five-line InitIndicators override that
did nothing but call InitNeuralNetwork. The cause was an access mismatch,
not a design: CExpertSignalCustom declares InitIndicators public, the AI
base redeclared it PROTECTED, and each subclass had to redeclare it
public to be reachable by CExpert. Worse, the base's own override does a
different job entirely - it creates the OHLC/ZigZag feature indicators -
and InitNeuralNetwork called it back scope-qualified to stop the virtual
dispatch landing in the subclass. Two jobs, one virtual name, and a
recursion trap held off by a scope qualifier.
The feature-indicator step is now InitFeatureIndicators() (protected,
non-virtual, named for what it does) and the AI base carries the single
public InitIndicators override. CONV/HYBRID/LSTM/PAI/META drop their
copies and are now purely identity plus topology, which is the classic
signal file's shape.
Comment pass on ExpertSignalAIBase.mqh, -100 lines with every constant
and every measured number kept. Three claims in the tier block were
stale and inverted - it named CalibratedConfidenceMagnitude() as the
tiering input where the code deliberately uses the RAW magnitude, and it
described the signal DB as re-ranking each tier when ApplyPatternWeight
declines the DB from the end of era 1. Also dropped a paragraph whose
subject was a previous version of the comment, and moved two notes down
onto the constants they document (CONV_COMPRESSION_DIVISOR was 16 lines
and three unrelated defines away from its own text).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 08:57:54 -04:00
bool InitFeatureIndicators ( CIndicators * indicators ) ;
2026-07-23 08:21:41 -04:00
//--- sets ID/m_id/m_folderPath/m_fileName/m_pattern_count from the subclass constructor - defaults
//--- to 4 (the confidence tiers - see m_pattern_0's declaration comment), not 1
void SetIdentity ( string id , string shortId , int patternCount = 4 ) ;
2026-07-14 22:36:27 -04:00
//--- hook for neuron-type-specific layers (Conv+Pool, LSTM, ...); default is a plain perceptron (no-op)
virtual bool AddCustomLayers ( CArrayObj * topology ) { return true ; }
2026-08-22 00:24:45 -04:00
//--- Reusable front-end stages, composed by the AddCustomLayers() overrides. (They had already
//--- drifted - HYBRID guarded the LSTM step with MathMax(1,...) and CSignalLSTM did not, so a
//--- historyBars of 1 gave the two a different step.)
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- Bodies in Expert\Topology\Topology.mqh.
bool AddConvStage ( CArrayObj * topology ) { return m_topology . AddConvStage ( topology ) ; }
bool AddLstmStage ( CArrayObj * topology ) { return m_topology . AddLstmStage ( topology ) ; }
2026-08-22 00:24:45 -04:00
//--- Which front-end stages this subclass's AddCustomLayers() actually appends. A virtual rather
//--- than a type-enum check, so a future composition cannot silently get the wrong answer.
2026-07-30 15:20:30 -04:00
virtual bool UsesConvStage ( void ) const { return false ; }
virtual bool UsesLstmStage ( void ) const { return false ; }
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- Labels.mqh: the swing-pivot direction label - THE training target for direction models.
2026-08-24 18:26:25 -04:00
ENUM_SIGNAL SwingPivotDirectionLabel ( int idx ) ;
feat: S2 meta-labeling head - binary trade-quality model over the classic-candidate corpus
The NN now has a target that is not per-bar direction (closed, best-of-999
p=1.0000): P(win | this journaled candidate, at the EA's own SL/TP, net of
cost). One net for all 52 pattern-sides, AIType=AI_META.
- NetForward.mqh: the host-side softmax+CE gradient generalized total==3 ->
2||3 on both backprop paths; a 2-class softmax IS a logistic head, and no
compute backend changes.
- SignalMETA.mqh (new): corpus loaded read-only from the LARGEST signal DB on
disk (decoupled from the config fingerprint that burned four S1 runs); the
GMT->server offset is measured PER ROW against entryPrice vs bar open
(DST-immune, histogram logged); a window-span regime filter drops the
pre-2017 daily-backfill rows; 31-feature setup descriptor appended at the
input (26 one-hot + side + tanh netVote + SL/TP ATR + spread/ATR).
- Training.mqh: candidate-queued pass 1, binary-target pass 2, per-candidate
calibration (2.5) and OOS (3) walks. Counter mapping win->Buy / loss->Sell
lets checkpoint selection, the edge floor, the plateau ladder and the
family-wise deploy gate run UNCHANGED: precision reads as win rate among
traded candidates, chance as the base win rate, recalls as sensitivity/
specificity. Era-end META line: coverage x (p - break-even) vs the null.
- Labels are the side-conditional triple-barrier win caches - never the DB's
stop-and-reverse outcome. Logit adjustment deliberately skipped (~40% base
rate). Live inference + online learning guarded off until S3.
- Fingerprint: conditional |TGT:META1; State\META\ folder + 2-output filename
slot keep meta models fully separate from direction models.
Compiles clean (0 errors, 0 warnings). S2 run = attach a chart with
AIType=AI_META; S3 wires the votes via the per-side hooks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 06:52:31 -04:00
//--- The one true input width every feedForward guard compares against.
refactor(meta): remove meta-labeling entirely - RETRAIN-NEUTRAL
~2,300 lines. META had real, repeatedly measured ranking skill and ZERO
operating points that ever cleared break-even (0/350 H1 eras, 1/999 H4
pre-2-sigma, 0/8 pooled fitted points). The clinching arithmetic was edge x
width = 0.095 ATR/trade against spread 0.099 ATR/trade, and the
dose-response showed the high-conviction tail is temporally unstable -
the precision-vs-threshold slope flips sign between calib and test on 3 of
4 symbols, so no ex-ante threshold rule exists. It shipped default-off and
never gated a live entry. The self-measured tier weights are what actually
rank the vote, and all six H4 instruments converged on them alone.
RETRAIN-NEUTRAL, and that is the property that made this safe:
- The weights fingerprint emitted "|TGT:META2" or "|TGT:SWG1" from an
if/else. Every direction model already took the SWG1 arm, so
collapsing it to an unconditional append is byte-identical. No .nnw or
.cfg is orphaned or re-keyed.
- NetInputWidth() lost its "+ MetaDescWidth()" term. MetaDescWidth()
returned 0 for every direction model, so the input layer is unchanged.
- DbLegacyAiSlot()'s slot 5 was reachable only with all four Use_* NNs
off AND meta on - a config that never shipped. Every existing .db keeps
its filename.
Deleted outright: Signals/SignalMETA.mqh, Expert/Trading/MetaGate.mqh (the
directory is now empty), Expert/Training/{MetaCorpus,MetaCandidateStore,
MetaFamilies}.mqh, Tests/Test_MetaFamilies.mq5, Meta_Labeling_Design.md.
Unwound in place, the delicate part: Training.mqh carried four
IsMetaTarget() branches whose else-arm WRAPPED the direction body (pass 1
queueing, pass 2 backprop, pass 2.5 calibration, pass 3 OOS scoring). Each
wrapper is removed and the direction body promoted back to its original
nesting - the bodies were never re-indented when the wrappers were added,
so the promoted code is byte-identical to what ran before META existed.
Also gone: the ensemble verdict's meta-veto replay and its
approved/vetoed/unscored counters, the per-family/per-side OOS
decomposition arrays, the m_isTrainQueueCand parallel queue and its
lockstep shuffle, and the S2 era report.
Also removed: the CMetaGate abstraction and the live CheckOpenPosition
veto; m_gates plus AddFilter's non-voter routing and IsVotingSignal()
(META was the only non-voting child, so m_gates was always empty);
m_parentSignal/SetParentSignal (existed only to reach the root's gate);
SweepPrepare/SweepPrepareIndicator (only caller was the corpus sweep);
IsMetaTarget() from all four view interfaces and their adapters;
Use_MetaLabeling, EnableMETA, Meta_ExportDataset, m_trainTarget.
EvalShift is KEPT - HistoricalNetVote() uses it for the filtered overlay,
not just the corpus sweep; only its comment changed. The 2-output softmax
arm in NetForward.mqh is kept too: it costs nothing and is the reusable
binary-head path, now commented as unclaimed rather than as META's.
Compile-verified in _claude_stage: 0 errors, 0 warnings, matching the
pre-edit baseline.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 09:44:52 -04:00
int NetInputWidth ( void ) const { return ( int ) m_historyBars * m_neuronsCount ; }
2026-07-30 15:20:30 -04:00
//--- AddConvStage runs BEFORE AddLstmStage wherever both are present (HYBRID), so the LSTM is fed the
//--- conv feature map rather than the raw flattened input.
bool HasConvBeforeLstm ( void ) const { return UsesConvStage ( ) & & UsesLstmStage ( ) ; }
feat(ai): real conv receptive field + the reference's channel pool
CONV's convolution used window = step = one bar, which is a per-bar
projection - a 1x1 conv with a temporal receptive field of ONE BAR. It never
mixed information across time, so "convolutional" described the layer type
and nothing about what it computed. Same finding that sank HYBRID's LSTM.
Pooling was removed on 2026-07-29 for being misconfigured against the conv
output's memory layout. That removal was right; leaving the conv at a
one-bar window was not. The two belong together: the NeuroNet_DNG reference
(references\MQL5\Experts\EDL\Trajectory.mqh layers 2-5, kernels
byte-identical to ours) pairs conv(window=2, step=1, window_out=4) with
pool(window=4, step=4), and the pool only earns its place because a conv
with a real receptive field sits above it.
The input is bar-major (BufferTempData appends m_neuronsCount contiguous
features per bar), so a flat window of k*m_neuronsCount spans exactly k
bars - the receptive field needed NO kernel change. The conv output is
position-major, so window == step == window_out is a clean
max-over-channels, which is what the reference does and what the existing
pool kernels already implement correctly.
New chain at H1 defaults (420 = 20 bars x 21):
conv1 w=42 s=21 out=8 -> 19 pos x 8 = 152
pool w=8 s=8 -> 19
conv2 w=2 s=1 out=8 -> 18 pos x 8 = 144 (effective field: 3 bars)
We deliberately stop before the reference's SECOND pool: a channel pool
emits one scalar per position, so a trailing pool would hand the dense stack
18 values and force it to fan out 18 -> 64. That is a bottleneck below every
learnable layer - the same class of mistake the 2026-07-29 removal was about.
Fixes a latent sizing bug this exposed: CNet's conv/pool position cursor
tracked sliding POSITIONS, but a conv's real width is units_count *
window_out. Any pool stacked on a conv would therefore have sized against a
width window_out times too small and silently built the wrong shape. Both
branches now read the built layer's actual Neurons(), which is what the
batch-norm branch already did for the same reason.
Also closes the architecture-pinning trap: a .nnw persists the window each
conv was built with, so an existing CONV/HYBRID model would have loaded
cleanly and gone on training under the OLD architecture. The conv weight
tensor is (window+1)*window_out, so this cannot be repaired in place -
EnforceTopologyContract now detects it, reports both shapes, and retrains.
Conv chain shape is derived in one place (ConvReceptiveFieldBars /
ConvFirstStagePositions / HasSecondConvStage / ConvOutputPositions /
ConvOutputWidth) and consumed by AddConvStage, LstmFanIn and the startup
config line, so what is built and what is logged cannot drift.
Both builds compile 0 errors, 0 warnings. Forces a CONV and HYBRID retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 20:05:37 -04:00
//--- Conv chain shape - see the definitions above AddConvStage. Every consumer reads these rather
//--- than re-deriving the arithmetic, so the built topology and the logged shape cannot disagree.
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- Bodies in Expert\Topology\Topology.mqh.
int ConvReceptiveFieldBars ( void ) const { return m_topology . ConvReceptiveFieldBars ( ) ; }
int ConvFirstStagePositions ( void ) const { return m_topology . ConvFirstStagePositions ( ) ; }
bool HasSecondConvStage ( void ) const { return m_topology . HasSecondConvStage ( ) ; }
int ConvOutputPositions ( void ) const { return m_topology . ConvOutputPositions ( ) ; }
int ConvOutputWidth ( void ) const { return m_topology . ConvOutputWidth ( ) ; }
fix(ai): stop the shutdown save from resurrecting reset weights; size HYBRID's LSTM to its real fan-in
ResetWeights already deletes the whole model set - .nnw, .cfg, _ckpt.tmp,
.stats, _shadow.nnw - and clears both the .arrows sidecar and the drawn
chart objects. What undid it was PersistWeightsOnShutdown: detaching the EA
after a reset but before an era completed re-created a .nnw from the
freshly-built, never-run net, so the next attach loaded an era-0 stub
instead of starting clean. For LSTM/HYBRID that stub is worse than nothing -
a layer that has never run a forward pass has m_iInputs<=0, so Save omits
every LSTM buffer (see 413ff7e). Skip the save when no era completed and no
model was loaded; that is exactly the post-reset and first-attach state.
Also sweep _shadowclone.tmp, which the reset did not cover.
Separately, ComputeLstmHiddenSize budgeted every topology against the
flattened input (historyBars x neuronsCount). True for LSTM, wrong for
HYBRID, where AddConvStage runs first and the LSTM is fed the conv feature
map - historyBars x convFilterCount, 160 rather than 420 at H1 defaults.
The quadratic is dominated by the inputs term, so overstating the fan-in
2.6x cost a full ladder step (16 units where the budget affords 32). New
virtual HasConvBeforeLstm() feeds LstmFanIn(), so composition decides this
rather than an AIType check. desc.window is advisory only - CNet never
passes it to the layer - but is now truthful for the same reason.
Derived values stay out of the weights-filename fingerprint and are adopted
from the .cfg, so existing models keep their saved width; only fresh ones
pick up the corrected budget.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 13:06:09 -04:00
//--- Actual input width the LSTM block sees, which is NOT always the flattened input.
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
int LstmFanIn ( void ) const { return m_topology . LstmFanIn ( ) ; }
2026-07-30 15:20:30 -04:00
//--- " | conv 21->8 x20 bars | lstm 160->32" for the startup config line; "" when neither applies.
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
string FrontEndConfigSummary ( void ) const { return m_topology . FrontEndConfigSummary ( ) ; }
feat(ai): batch normalization between dense layers
The only bounded stage in the entire forward path was the sigmoid
classification head - every hidden stage is PRELU. That is a network with
no internal scale control, and the failure ordered exactly by depth: on
SP500 H1 the shallow perceptron held ~52% balanced accuracy while the
deepest topology sat on the 33.3% one-class floor, with the per-bar logit
spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the
evidence tilt fell under the class-prior tilt. That is the signature of
internal covariate shift, which chapter 6.1 of the reference book is
entirely about and which the NeuroNet_DNG engine addresses with a layer
this project never had.
Two mechanisms make this the right fix rather than more hyperparameter
nudging:
- it decouples WEIGHT_DECAY from the learned function (van Laarhoven
2017) - with a normalized layer downstream, decay can no longer grind
the discriminative signal away, it only rescales the effective
learning rate;
- it is the precondition for ever running an unbounded logit head here.
The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because
nothing upstream constrained scale.
Implementation notes:
- CNeuronBatchNormOCL computes host-side rather than as a fourth copy of
a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math
is elementwise O(n); this way it behaves identically on all four
compute tiers, needs no DLL rebuild, and cannot drift between
backends. Same precedent as the softmax+CCE gradient and the
per-sample loss weighting, both computed in MQL5 for that reason.
- Statistics are exponential moving, not a stored mini-batch: training
is pure online SGD, one update per sample, so there is no batch to
average over. BatchNormWindow is an EMA window length.
- gamma/beta are excluded from weight decay, deliberately - decaying
gamma toward zero is the exact pathology being fixed.
- The layer self-sizes from whatever sits below it, because a conv/pool
stage's output width is derived inside the CNet constructor and is not
knowable to the topology builder.
- Checkpoint capture/restore/blend carry gamma/beta and the running
statistics alongside the dense matrix, so the plateau ladder cannot
restore a mismatched pair.
- SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the
weight-carrying penultimate layer; with normalization enabled that is
the batch-norm layer, so the cold-start bias seed would have silently
stopped being applied.
- Refuses to build, loudly, if a topology asks for normalization with no
compute backend at all - rather than quietly training a different
architecture than the one requested.
EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are
inputs so the effect can be A/B'd without a recompile. Both feed the
weights-filename fingerprint, appended conditionally so existing non-BN
configs keep their fingerprints and are not forced to retrain.
Verified: analytic gradients match finite differences to 1.5e-7 relative
over 200 random cases; a faithful port of the full forward/backward chain
collapses to the 33.3% floor by era 4 without this layer and holds
36-43% with it. Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
//--- Appends a batch-normalization layer, or does nothing (returning success) when EnableBatchNorm is
//--- off. `units` is advisory only - CNet sizes the layer from whatever sits below it, because a conv
//--- or pool stage's output width is derived inside the CNet constructor and is not knowable here.
//--- See AI\NeuronBatchNorm.mqh for what the layer does and why it exists.
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- Body in Expert\Topology\Topology.mqh.
bool AddBatchNormStage ( CArrayObj * topology , int units ) { return m_topology . AddBatchNormStage ( topology , units ) ; }
2026-07-14 22:36:27 -04:00
//--- hardcoded activation for the common tapering Dense hidden-layer stack built by
2026-08-22 00:24:45 -04:00
//--- BuildFreshTopology() (below AddCustomLayers, above the output layer).
2026-07-14 22:36:27 -04:00
virtual ENUM_ACTIVATION HiddenLayerActivation ( void ) { return PRELU ; }
2026-08-22 00:24:45 -04:00
//--- Single source of truth for the output head's activation. As two separate literals, changing
//--- the head silently did nothing to any existing model. Regression: TANH, whose [-1,1] maps
//--- onto the Sell/Neutral/Buy convention.
2026-07-29 12:00:40 -04:00
ENUM_ACTIVATION OutputLayerActivation ( void ) const { return ( m_outputNeuronsCount = = 1 ) ? TANH : SIGMOID ; }
2026-08-22 00:24:45 -04:00
//--- Width of the first dense layer, DERIVED rather than configured. That is ~8 parameters per
//--- sample, and it EXPANDS a set of highly correlated inputs instead of compressing them. MUST
//--- be called before the fingerprint is built and never again (see the note on fingerprint-
//--- feeding members at the top of this file).
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- Body in Expert\Topology\Topology.mqh.
int ComputeFirstLayerWidth ( void ) const { return m_topology . ComputeFirstLayerWidth ( ) ; }
fix(topology): the capacity budget counted overlapping bars as independent examples
EstimatedInSampleBars() returned raw bars (11372 on SP500 H4) and every derived
capacity decision spent that: first-layer width, conv filters, LSTM hidden size.
But triple-barrier labels overlap - mean lifespan 9.4 bars - so the label cache line
on the same run already reports those bars are worth ~1210 independent observations.
Sizing a network against RAW bars while grading it against EFFECTIVE ones is two
subsystems disagreeing about one sample, and it disagreed in the dangerous direction
because the capacity side was the optimistic one: the warning's "roughly 1.1 weights
per training bar" is nearer 11 per independent observation.
EffectiveSampleSize() has existed since 2026-08-17 and is applied at eight sites, all
of them statistics. This adds the ninth, in the one place that decides how many
parameters get fitted. Applied inside EstimatedInSampleBars() rather than at the call
sites, because that function exists precisely so the three stages spend one budget.
SELF-ENABLING AND THEREFORE INERT WHERE IT MATTERS MOST, which is why this is two
changes and not one. MeanLabelLifespan() is 1.0 until a label cache has measured
something, so on a model's first build - before any label exists - the deflation is
correctly the identity: an unmeasured overlap must not invent a shrink. A fresh
attach constructs a fresh object, so its counters are zero too; only a mid-session
weights reset carries real evidence into a rebuild. That is deliberately safe (no
attach can now re-derive a narrower topology and discard trained weights) but it
would have left the first build - the case you most want the truth for - quoting the
flattering figure. So ReportDetectability now restates capacity against the effective
sample at the first moment L is real, for the topology already pinned. It re-sizes
nothing; it reports what was bought. Placed ABOVE that function's break-even guard on
purpose - a degenerate geometry is exactly when you want to know the net is
over-parameterised, and "it only fires for sane configs" is how the 2026-08-18
IS-error stop managed never to fire at all.
The warning also names its basis now (independent observations and L, or an explicit
"overlap NOT YET MEASURED, this is an UPPER BOUND"), so a flattering number can never
again read as a measured one.
Also factors FirstLayerFanIn() out of ComputeFirstLayerWidth so the capacity REPORT
charges for exactly what the capacity DECISION charged for - same reason
RequiredHorizonBars was factored out after the 2026-08-17 divergence - and makes
MeanLabelLifespan()/EffectiveSampleSize() const so the const budget path can call them.
Verified: no recursion (EstimatedInSampleBars -> EffectiveSampleSize ->
EstimatedInSampleBarsRaw, which computes from Bars() alone); both new StringFormat
sites hand-counted (basis 3/3 and 1/1, CAPACITY 10 specifiers / 10 arguments).
NOT COMPILED - user compiles in MetaEditor.
2026-08-19 18:43:48 -04:00
//--- Expected in-sample training rows for the configured study period, split and timeframe, in
2026-08-22 00:24:45 -04:00
//--- INDEPENDENT observations. See the definition for why a raw bar count was the wrong unit to
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- size a network in. Body in Expert\Topology\Topology.mqh.
double EstimatedInSampleBars ( void ) const { return m_topology . EstimatedInSampleBars ( ) ; }
fix(topology): the capacity budget counted overlapping bars as independent examples
EstimatedInSampleBars() returned raw bars (11372 on SP500 H4) and every derived
capacity decision spent that: first-layer width, conv filters, LSTM hidden size.
But triple-barrier labels overlap - mean lifespan 9.4 bars - so the label cache line
on the same run already reports those bars are worth ~1210 independent observations.
Sizing a network against RAW bars while grading it against EFFECTIVE ones is two
subsystems disagreeing about one sample, and it disagreed in the dangerous direction
because the capacity side was the optimistic one: the warning's "roughly 1.1 weights
per training bar" is nearer 11 per independent observation.
EffectiveSampleSize() has existed since 2026-08-17 and is applied at eight sites, all
of them statistics. This adds the ninth, in the one place that decides how many
parameters get fitted. Applied inside EstimatedInSampleBars() rather than at the call
sites, because that function exists precisely so the three stages spend one budget.
SELF-ENABLING AND THEREFORE INERT WHERE IT MATTERS MOST, which is why this is two
changes and not one. MeanLabelLifespan() is 1.0 until a label cache has measured
something, so on a model's first build - before any label exists - the deflation is
correctly the identity: an unmeasured overlap must not invent a shrink. A fresh
attach constructs a fresh object, so its counters are zero too; only a mid-session
weights reset carries real evidence into a rebuild. That is deliberately safe (no
attach can now re-derive a narrower topology and discard trained weights) but it
would have left the first build - the case you most want the truth for - quoting the
flattering figure. So ReportDetectability now restates capacity against the effective
sample at the first moment L is real, for the topology already pinned. It re-sizes
nothing; it reports what was bought. Placed ABOVE that function's break-even guard on
purpose - a degenerate geometry is exactly when you want to know the net is
over-parameterised, and "it only fires for sane configs" is how the 2026-08-18
IS-error stop managed never to fire at all.
The warning also names its basis now (independent observations and L, or an explicit
"overlap NOT YET MEASURED, this is an UPPER BOUND"), so a flattering number can never
again read as a measured one.
Also factors FirstLayerFanIn() out of ComputeFirstLayerWidth so the capacity REPORT
charges for exactly what the capacity DECISION charged for - same reason
RequiredHorizonBars was factored out after the 2026-08-17 divergence - and makes
MeanLabelLifespan()/EffectiveSampleSize() const so the const budget path can call them.
Verified: no recursion (EstimatedInSampleBars -> EffectiveSampleSize ->
EstimatedInSampleBarsRaw, which computes from Bars() alone); both new StringFormat
sites hand-counted (basis 3/3 and 1/1, CAPACITY 10 specifiers / 10 arguments).
NOT COMPILED - user compiles in MetaEditor.
2026-08-19 18:43:48 -04:00
//--- The same figure BEFORE the overlap deflation, for reports that want to show both. Never size
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- anything from this one - that was the bug. Body in Expert\Topology\Topology.mqh.
double EstimatedInSampleBarsRaw ( void ) const { return m_topology . EstimatedInSampleBarsRaw ( ) ; }
fix(topology): the capacity budget counted overlapping bars as independent examples
EstimatedInSampleBars() returned raw bars (11372 on SP500 H4) and every derived
capacity decision spent that: first-layer width, conv filters, LSTM hidden size.
But triple-barrier labels overlap - mean lifespan 9.4 bars - so the label cache line
on the same run already reports those bars are worth ~1210 independent observations.
Sizing a network against RAW bars while grading it against EFFECTIVE ones is two
subsystems disagreeing about one sample, and it disagreed in the dangerous direction
because the capacity side was the optimistic one: the warning's "roughly 1.1 weights
per training bar" is nearer 11 per independent observation.
EffectiveSampleSize() has existed since 2026-08-17 and is applied at eight sites, all
of them statistics. This adds the ninth, in the one place that decides how many
parameters get fitted. Applied inside EstimatedInSampleBars() rather than at the call
sites, because that function exists precisely so the three stages spend one budget.
SELF-ENABLING AND THEREFORE INERT WHERE IT MATTERS MOST, which is why this is two
changes and not one. MeanLabelLifespan() is 1.0 until a label cache has measured
something, so on a model's first build - before any label exists - the deflation is
correctly the identity: an unmeasured overlap must not invent a shrink. A fresh
attach constructs a fresh object, so its counters are zero too; only a mid-session
weights reset carries real evidence into a rebuild. That is deliberately safe (no
attach can now re-derive a narrower topology and discard trained weights) but it
would have left the first build - the case you most want the truth for - quoting the
flattering figure. So ReportDetectability now restates capacity against the effective
sample at the first moment L is real, for the topology already pinned. It re-sizes
nothing; it reports what was bought. Placed ABOVE that function's break-even guard on
purpose - a degenerate geometry is exactly when you want to know the net is
over-parameterised, and "it only fires for sane configs" is how the 2026-08-18
IS-error stop managed never to fire at all.
The warning also names its basis now (independent observations and L, or an explicit
"overlap NOT YET MEASURED, this is an UPPER BOUND"), so a flattering number can never
again read as a measured one.
Also factors FirstLayerFanIn() out of ComputeFirstLayerWidth so the capacity REPORT
charges for exactly what the capacity DECISION charged for - same reason
RequiredHorizonBars was factored out after the 2026-08-17 divergence - and makes
MeanLabelLifespan()/EffectiveSampleSize() const so the const budget path can call them.
Verified: no recursion (EstimatedInSampleBars -> EffectiveSampleSize ->
EstimatedInSampleBarsRaw, which computes from Bars() alone); both new StringFormat
sites hand-counted (basis 3/3 and 1/1, CAPACITY 10 specifiers / 10 arguments).
NOT COMPILED - user compiles in MetaEditor.
2026-08-19 18:43:48 -04:00
//--- Width of the vector the first dense layer actually sees: the front-end stage's output where
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- one exists, the flattened window otherwise. Body in Expert\Topology\Topology.mqh.
int FirstLayerFanIn ( void ) const { return m_topology . FirstLayerFanIn ( ) ; }
2026-08-22 00:24:45 -04:00
//--- Conv output-filter count and LSTM hidden width, DERIVED for the same reason the first-layer
//--- width is. Both MUST be called before the fingerprint is built and never again: they assign
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- fingerprint-feeding members (see the note at the top of this file). Bodies in
//--- Expert\Topology\Topology.mqh.
int ComputeConvFilterCount ( void ) const { return m_topology . ComputeConvFilterCount ( ) ; }
int ComputeLstmHiddenSize ( void ) const { return m_topology . ComputeLstmHiddenSize ( ) ; }
2026-08-22 00:24:45 -04:00
//--- Dense-taper DEPTH, derived 2026-07-30 from the two endpoints the taper connects. Reads
feat(nn): derive dense depth, train on all history, pin the shape in .cfg
Completes the derived-topology work. Three inputs removed.
AIType loses its depth suffix - AI_MLP/AI_CONV/AI_LSTM/AI_HYBRID, five
entries instead of eight. Depth is now derived from the two endpoints
the taper already has to connect (derived first-layer width, output-tied
final width) at a 2x per-layer compression target, clamped [2..5].
Asking a user to pick a layer count while the code derives the widths
those layers taper between was asking for half a decision: at 64 units
tapering to 12, four layers compress by 1.4x per step and five by 1.3x,
so the extra depth bought no abstraction. On the shipping H1/10y default
the derivation lands on 3 layers - the depth that actually won Run 2.
StudyPeriods removed. There is no case for training on less data than
the broker provides at a ~6% directional base rate; the honest
generalization read comes from the OOS holdout, not from withholding
history. Training now starts at the earliest available bar, floored by
MinTrainYear, which answers a different question (excluding dubious
pre-history) and stays.
That required closing the hazard the old code documented: the capacity
budget now MEASURES the symbol's real bar count, and a topology derived
from a measurement would widen as history downloads. Both ends are now
pinned. Every derived value left the weights-filename fingerprint -
keying a filename on a measured quantity means the EA looks for a file
that does not exist, starts from era 0 and orphans a trained model,
silently, because a missing cache is the normal first-run state. The
shape lives in the .cfg instead, where LoadAndCompare now ADOPTS the
four derived fields rather than diffing them; a mismatch there would
discard a fully-trained model over nothing the user did. Two fields
appended to the .cfg for the conv/LSTM stages, length-guarded on read
because FileReadInteger past EOF returns 0 with no error.
ForceHiddenLayers, a compile-time constant like DebuggingMode, pins
depth for diagnostic comparisons. It joins the fingerprint only when
non-zero, so forced depths get their own files - sequential comparisons
only, not simultaneous from one .ex5.
Derived shape, H1/10y defaults (21 features x 20 bars): first layer 64,
3 dense, 8 conv filters, 16 LSTM units. The LSTM block halves from
~58k to ~28k weights.
Both builds compile 0 errors, 0 warnings. Re-keys existing models.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 10:05:40 -04:00
//--- m_initialNeuronsCount, so it MUST be called after ComputeFirstLayerWidth and before the
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- fingerprint - see the note on fingerprint-feeding members at the top of this file. Body in
//--- Expert\Topology\Topology.mqh.
int ComputeHiddenLayerCount ( void ) const { return m_topology . ComputeHiddenLayerCount ( ) ; }
2026-07-29 12:00:40 -04:00
//--- Re-assert everything about a just-loaded net that lives in the FILE but is owned by the CODE.
//--- Call after every successful Net.Load(); no-ops (and stays silent) when the file already agrees.
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
void EnforceTopologyContract ( void ) { m_modelPersistence . EnforceTopologyContract ( ) ; }
2026-07-14 22:36:27 -04:00
//--- common network bootstrap: indicators, topology build/load, training-file bookkeeping
bool InitNeuralNetwork ( CIndicators * indicators ) ;
2026-08-19 23:50:30 -04:00
//--- The retrain-affecting configuration, as one string whose hash names the .nnw/.cfg pair.
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- Body and the two rules that govern what may enter it: Expert\Topology\Topology.mqh.
string BuildModelFingerprint ( void ) { return m_topology . BuildModelFingerprint ( ) ; }
2026-08-24 04:39:17 -04:00
//--- Exclusive per-config claim, so two charts can never train into one set of model files. Body
//--- on CConfigLock (m_configLock) - see Expert\ConfigLock\ConfigLock.mqh's class comment.
bool AcquireConfigLock ( void ) { return m_configLock . Acquire ( ) ; }
void ReleaseConfigLock ( void ) { m_configLock . Release ( ) ; }
feat(panel): one live vote line, no stale era count, no per-model HUD
Two chart-display fixes reported after watching a converged 4-model
ensemble: the ensemble panel's trailing "(era 69, 4 models,
DEPLOYING)" was frozen at whatever era the ensemble happened to
deploy on, and the separate top-right HUD (one line per model, raw
B/S/N + weight + era + error) was clutter once the vote itself is
what matters.
Root cause of the freeze: g_ensembleVoteLine is written once per era,
at pass-3 completion. A deployed/converged ensemble runs no further
eras (ScheduleTrainingIfNeeded's trainingComplete branch skips
Train() entirely), so that line could never update again - the era
count and "DEPLOYING" marker were permanent set-dressing from the
deploying era, not a live reading.
- EnsembleScoreCombinedVote() drops the era/DEPLOYING tail once
g_ensDeployApproved - nothing left there worth freezing.
- UpdateVoteReadout() (the aggregate "VOTE ..." line, previously its
own top-right chart object) now writes g_liveVoteLine instead of
drawing anything. Both status-label builders - PublishEnsembleStatus
for the ensemble panel, PublishStatus's choke point for the solo
panel - append it as one line, refreshed every tick/timer exactly
as the old HUD was, so the live vote replaces the frozen era tail
in the same visual slot.
- RefreshVoteReadout()'s per-member loop (DisplayHudLine, one
ObjectLabel per model) is deleted outright rather than folded in -
the operator asked for the aggregate only, "without telling me each
individual network".
Follow-on dead-code removal, since DisplayHudLine was the only
caller: the DispProb/DispSignal/MetaGateArmedNow/MetaHasScore/
MetaLastP/MetaLastBe/MetaApproved/MetaVetoed leg of IChartView (and
its AIBaseChartView/AIBaseChartViewImpl/ExpertSignalAIBase forwards)
had no other reader. The underlying data survives untouched -
m_metaTelemetry is still populated live by SignalMETA.mqh,
m_dispSignal still feeds ProspectiveVote - only the chart-view
forwarding that existed solely to reach the deleted HUD is gone.
Compile: 0 errors, 0 warnings (stage).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 22:27:41 -04:00
//--- Chart arrows, persistence and the status panel all live in CChartUI now - see
//--- Expert\Chart\ChartUI.mqh. These stay as thin forwards, called by name from Training.mqh and
//--- this file's own live-tick path; none of their signatures changed.
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
void DrawObject ( datetime time , double signal , double close ) { m_chartUI . DrawObject ( time , signal , close ) ; }
void DeleteObject ( datetime time ) { m_chartUI . DeleteObject ( time ) ; }
2026-07-21 00:03:45 -04:00
//--- Time-ordered NMS sweep over m_arrowSignalCache: prunes each same-direction run down to its
//--- earliest bar (deleting redundant neighbors within m_signalClusterWindow). Run once per era end.
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
void PruneDirectionalClusters ( int bars ) { m_chartUI . PruneDirectionalClusters ( bars ) ; }
2026-08-22 00:24:45 -04:00
//--- Whether BOTH directions can currently be traded, which is the precondition for the
//--- alternation rule in the NMS paths: with only one side enabled there is no opposite signal
//--- to wait for, so requiring alternation would suppress everything after the first call.
2026-08-10 14:26:12 -04:00
bool BothDirectionsTradeable ( void ) const { return true ; }
2026-07-21 00:03:45 -04:00
//--- Live newest-bar NMS accept test (time-keyed, idempotent per bar time - see m_signalClusterWindow).
2026-07-21 12:30:29 -04:00
bool NmsLiveAccept ( datetime barTime , ENUM_SIGNAL dir , double conf )
2026-07-21 00:03:45 -04:00
{
if ( m_signalClusterWindow < = 0 )
return true ;
2026-07-21 12:30:29 -04:00
if ( dir ! = Buy & & dir ! = Sell )
return true ;
// Idempotent re-eval of the same bar (RefreshLatestSignal can run more than once per bar).
if ( dir = = Buy & & m_nmsLiveBuyTime = = barTime )
return m_nmsLiveBuyAccept ;
if ( dir = = Sell & & m_nmsLiveSellTime = = barTime )
return m_nmsLiveSellAccept ;
2026-07-21 00:03:45 -04:00
long minGap = ( long ) m_signalClusterWindow * PeriodSeconds ( ) ;
2026-07-21 12:30:29 -04:00
datetime lastSame = ( dir = = Buy ) ? m_nmsLiveBuyTime : m_nmsLiveSellTime ;
bool accept ;
feat(signal): make the signal cooldown tunable, and add a hard any-direction gate
The declustering the charts needed already existed - NmsLiveAccept, per-direction
run-collapse plus cross-direction resolution plus strict alternation - and it was
already set to 10 bars. It could not be TUNED: SignalClusterWindow was a compile-
time const, so finding the right value needed a rebuild. That is the actual gap.
Now three inputs, as enum dropdowns:
Signal_CooldownScope per-direction, or a hard any-direction gate on top
Signal_CooldownBars SCB_OFF..SCB_50, default 10
Signal_CooldownMinutes SCM_OFF..SCM_1440, overrides bars when set
Minutes resolve against the CHART period and round UP, so a cooldown asked for in
wall-clock is never silently shorter than requested and survives a timeframe
change.
SCB_/SCM_ prefixes are deliberately unique. M15/M30/M60 are ALREADY members of
NF_LOOKBACK_PRESETS, and MQL5 binds a duplicated enum member to the first-declared
enum silently - the obvious names would have compiled straight into the news
filter's values.
THE ANY-DIRECTION GATE IS ADDITIVE, NOT A REPLACEMENT, and the first cut of this
had it backwards. Measured on the live log: the current rules draw 222 arrows over
4999 bars, while a BARE 10-bar cooldown permits up to 454 - because ALTERNATION is
what declutters today, not the window. Swapping the rules out would have roughly
doubled the clutter it was asked to remove. Layered, it can only ever suppress
more. Suppressed bars still advance the per-direction last-SEEN cursors, so a run
straddling the boundary does not restart as if it were fresh.
Applied at all THREE sites that must agree - live inference, OOS pass-3 scoring
and the chart renderer. Their own comments say why: an arrow set that does not
obey the same rule as the traded set shows calls the EA would never take.
Also corrects a stale comment that called this window "display only". It is not:
when it suppresses, the live path zeroes the signal outright - no arrow, no vote,
no position. Training never sees it, so these cost no retrain and are correctly
absent from the fingerprint.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 09:48:28 -04:00
//--- RULE 0 - HARD ANY-DIRECTION COOLDOWN, applied BEFORE and IN ADDITION TO rules 1-3, never
//--- instead of them. A kept signal of either direction silences the next `window` bars
//--- outright. Suppressing here still advances the per-direction last-SEEN state below, so a
//--- run that straddles the cooldown boundary does not restart as if it were fresh.
bool coolBlocked = ( m_signalCooldownScope = = SIGNAL_COOLDOWN_ANY_SIGNAL & &
m_nmsLiveKeptTime ! = 0 & &
( long ) ( barTime - m_nmsLiveKeptTime ) < = minGap ) ;
if ( coolBlocked )
{
if ( dir = = Buy )
{
m_nmsLiveBuyTime = barTime ;
m_nmsLiveBuyAccept = false ;
}
else
{
m_nmsLiveSellTime = barTime ;
m_nmsLiveSellAccept = false ;
}
return false ;
}
2026-07-21 12:30:29 -04:00
// 1) Same-direction contiguous collapse: suppress if within the window of the previous SEEN
// same-direction bar (advance last-seen below either way, so a whole run collapses to one).
if ( lastSame ! = 0 & & ( long ) ( barTime - lastSame ) < = minGap )
accept = false ;
else
{
// 2) Cross-direction resolution vs the last KEPT opposite signal: keep the stronger side.
accept = true ;
if ( m_nmsLiveKeptTime ! = 0 & & m_nmsLiveKeptDir ! = dir & &
( long ) ( barTime - m_nmsLiveKeptTime ) < = minGap )
{
if ( conf > m_nmsLiveKeptConf )
DeleteObject ( m_nmsLiveKeptTime ) ; // this bar is stronger: remove the weaker opposite arrow
else
accept = false ; // the kept opposite is stronger: suppress this bar
}
2026-08-22 00:24:45 -04:00
//--- 3) ALTERNATION. Rule 1 only collapses a same-direction run inside the window; past
//--- it, a second Buy is emitted with no Sell in between, giving Buy/Buy/Buy/Sell.
2026-08-10 14:26:12 -04:00
if ( accept & & BothDirectionsTradeable ( ) & & m_nmsLiveKeptTime ! = 0 & & m_nmsLiveKeptDir = = dir )
accept = false ;
2026-07-21 12:30:29 -04:00
}
2026-07-21 00:03:45 -04:00
if ( dir = = Buy )
{
m_nmsLiveBuyTime = barTime ;
2026-07-21 12:30:29 -04:00
m_nmsLiveBuyAccept = accept ;
2026-07-21 00:03:45 -04:00
}
2026-07-21 12:30:29 -04:00
else
2026-07-21 00:03:45 -04:00
{
m_nmsLiveSellTime = barTime ;
2026-07-21 12:30:29 -04:00
m_nmsLiveSellAccept = accept ;
2026-07-21 00:03:45 -04:00
}
2026-07-21 12:30:29 -04:00
if ( accept )
{
m_nmsLiveKeptTime = barTime ;
m_nmsLiveKeptDir = dir ;
m_nmsLiveKeptConf = conf ;
}
return accept ;
2026-07-21 00:03:45 -04:00
}
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
int PurgeChart ( void ) { return m_chartUI . PurgeChart ( ) ; }
2026-07-14 22:36:27 -04:00
ENUM_SIGNAL DoubleToSignal ( double value ) ;
2026-08-22 00:24:45 -04:00
//--- Shared status-label formatting for all three of Train()'s era passes (pass 1 sequential
//--- scan/ display, pass 2 shuffled backProp, pass 3 post-training OOS scoring) - see
//--- m_isTrainQueue's and m_isPass2Active's declaration comments for why the era loop is now
//--- three passes instead of one.
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
void UpdateTrainingStatusLabel ( const string & progressLine , double neuron0 , double neuron1 , double neuron2 , double signalValue , bool forceRefresh = false )
{ m_chartUI . UpdateTrainingStatusLabel ( progressLine , neuron0 , neuron1 , neuron2 , signalValue , forceRefresh ) ; }
//--- Forced panel refresh from CChartUI's own last-cached values - see its declaration comment.
void RefreshStatusLabel ( void ) { m_chartUI . RefreshStatusLabel ( ) ; }
//--- The throttle tick and the last-values cache this used to keep now live on CChartUI, next to
//--- the panel text they feed - see its declaration comments.
//--- Latest OOS Buy/Sell recall (-1 = n/a), read by CChartUI through ChartOosRecallPct() so the panel can show it
2026-08-20 09:54:23 -04:00
//--- on every call rather than only at era end. On-chart because blended accuracy is what a trader
//--- sees by default, and a model can look good on it purely by calling Neutral often.
2026-07-22 22:51:04 -04:00
int m_lastBuyRecallPct , m_lastSellRecallPct ;
2026-08-24 04:04:28 -04:00
//--- THE single derivation of the 3-class argmax rule - strict majority, ties resolve to
//--- Neutral. ApplyClassificationSoftmax()/AdjustedSignalFromSoftmax()/DirectionalMargin() all
//--- derive their decision from this one test instead of each re-deriving the comparison.
ENUM_SIGNAL Argmax3 ( double pBuy , double pSell , double pNeutral ) ;
2026-08-22 00:24:45 -04:00
//--- Turns the head's 3 SIGMOID values into a softmax distribution in place and returns the
//--- signed dPrevSignal convention (+P(buy), -P(sell), exactly 0.0 for neutral). Max-subtracted
//--- before exp() for stability.
2026-07-14 22:36:27 -04:00
double ApplyClassificationSoftmax ( void ) ;
2026-07-23 19:36:34 -04:00
//--- Post-hoc logit adjustment / prior correction: reads the raw softmax probabilities
2026-08-22 00:24:45 -04:00
//--- ApplyClassificationSoftmax() just left in TempData[0..2] and returns the PRIOR-CORRECTED
//--- signed decision (same +P'(buy)/-P'(sell)/0-neutral convention).
2026-07-23 19:36:34 -04:00
double AdjustedSignalFromSoftmax ( void ) ;
feat(hud): per-member neuron lines + a vote label that moves as the nets learn
Both 2026-08-19 reports were the same staleness: every source behind the
label was an ERA artifact (live cache refills at pass-3 completion, the
snapshot copies once per era, dPrevSignal is the frozen purge-band edge
bar) - so the readout stepped at era cadence at best, stayed glued to
one direction, and lagged the era counter.
DisplayInference(): throttled (4s, 1s across an era boundary),
SIDE-EFFECT-FREE forward of the current decision bar (window ending on
bar 1, same question the live path asks) through the LEARNER net.
Batch-norm running stats are bracketed frozen/RESTORED via the new
CNet::GetBatchNormFrozen() + CNeuronBatchNormOCL::StatsFrozen() - restore,
not unfreeze, because a display tick can land between pass-3 chunks whose
whole scan holds them frozen. Writes nothing a trading or training path
reads (dPrevSignal, NMS state, tallies, watermarks all untouched;
RefreshLatestSignal is not reusable here precisely because it writes all
of them). LSTM safe by construction: h/c zeroed per forward.
ProspectiveVote() reads the fresh forward as its FIRST source; the
era-artifact chain becomes the fallback (meta head, warm-up, window
holes).
DisplayHudLine(): the reference library's training label, per ensemble
member - name, output activations (softmax probs or raw scalar), the
decision, its weighted vote (the exact consensus numerator term), era,
recent average error, "(trn)" while not vote-capable. Rendered under the
vote line in RefreshVoteReadout BEFORE the live-vote defer (member lines
are telemetry, not tradable readings), coloured by the member's own
direction in muted tones - the vote line's strict
green-only-when-it-would-trade rule is untouched.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 08:48:34 -04:00
//--- Throttled, side-effect-free forward of the CURRENT decision bar for the HUD - body and the
//--- full why in AIBase\Inference.mqh. True when m_dispProbs/m_dispSignal hold a usable read.
bool DisplayInference ( void ) ;
feat: fitted directional confidence threshold - selectivity gets a mechanism
The training loss and the selection metric wanted different things and only
the second one knew it. Logit-adjusted cross-entropy has no term for "how
often should I trade", so the head calls a direction on 87-91% of bars. The
selection metric is precision x coverage credit, saturating at the coverage
floor - above the floor extra calls earn NOTHING and only precision counts.
So selection wanted few good calls, the loss produced many mediocre ones, and
all selection could do was pick the least-bad era out of what it was handed.
Nothing pushed the model toward selectivity.
This gives the decision RULE the policy instead of distorting the loss (which
is estimating class probabilities correctly, and a probability estimate should
not be bent to encode a trading policy - Elkan 2001: estimate, then choose the
operating point separately). AdjustedSignalFromSoftmax now abstains unless the
winning direction's softmax margin over its best rival clears a fitted
threshold. Margin, not the winning probability: the latter moves with overall
calibration rather than with how close the decision actually was.
Fitted on IS, applied to OOS and live. Pass 2 already forward-passes every IS
sample, so the margin histogram is harvested there for free (primary
occurrences only, so the oversampled replay queue cannot skew the operating
point); the fit runs at the end of pass 2, BEFORE pass 3, so the deploy gate
grades the thresholded model on bars the threshold never saw. Fitting on
pass 3's own predictions would be choosing the operating point on the data
being graded - the best-of-N error corrected in five other places here.
Objective: maximise IS directional precision subject to still clearing the
SAME coverage floor the deploy gate uses (base rate x 0.25, re-derived
locally so the two cannot drift apart). Swept top-down in one pass; ties go
to the LOWER threshold, since equal precision for less coverage is strictly
worse. Under DIR_CONF_MIN_FIT_CALLS (200) it runs unthresholded rather than
on a guess.
The threshold is part of the MODEL, not the run: captured with
Net.CaptureWeights(), restored with the weights at both restore sites, and
appended to the .cfg under the same length-guard convention so a deployed
model reloads at the operating point its gate actually cleared. A pre-2026-08-09
.cfg reads 0.0, which is exactly the behaviour it was trained under.
Per-era line now prints "@margin>=X.XX" next to coverage, so a coverage drop
can be attributed to the operating point rather than guessed at.
Both build variants compile 0 errors / 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:04:37 -04:00
//--- Margin between the winning class and its best rival, from the softmax already in TempData.
2026-08-22 00:24:45 -04:00
//--- Returns <0 when the winner is Neutral (not a directional call, so no operating point
//--- applies) or when the outputs are unreadable.
feat: fitted directional confidence threshold - selectivity gets a mechanism
The training loss and the selection metric wanted different things and only
the second one knew it. Logit-adjusted cross-entropy has no term for "how
often should I trade", so the head calls a direction on 87-91% of bars. The
selection metric is precision x coverage credit, saturating at the coverage
floor - above the floor extra calls earn NOTHING and only precision counts.
So selection wanted few good calls, the loss produced many mediocre ones, and
all selection could do was pick the least-bad era out of what it was handed.
Nothing pushed the model toward selectivity.
This gives the decision RULE the policy instead of distorting the loss (which
is estimating class probabilities correctly, and a probability estimate should
not be bent to encode a trading policy - Elkan 2001: estimate, then choose the
operating point separately). AdjustedSignalFromSoftmax now abstains unless the
winning direction's softmax margin over its best rival clears a fitted
threshold. Margin, not the winning probability: the latter moves with overall
calibration rather than with how close the decision actually was.
Fitted on IS, applied to OOS and live. Pass 2 already forward-passes every IS
sample, so the margin histogram is harvested there for free (primary
occurrences only, so the oversampled replay queue cannot skew the operating
point); the fit runs at the end of pass 2, BEFORE pass 3, so the deploy gate
grades the thresholded model on bars the threshold never saw. Fitting on
pass 3's own predictions would be choosing the operating point on the data
being graded - the best-of-N error corrected in five other places here.
Objective: maximise IS directional precision subject to still clearing the
SAME coverage floor the deploy gate uses (base rate x 0.25, re-derived
locally so the two cannot drift apart). Swept top-down in one pass; ties go
to the LOWER threshold, since equal precision for less coverage is strictly
worse. Under DIR_CONF_MIN_FIT_CALLS (200) it runs unthresholded rather than
on a guess.
The threshold is part of the MODEL, not the run: captured with
Net.CaptureWeights(), restored with the weights at both restore sites, and
appended to the .cfg under the same length-guard convention so a deployed
model reloads at the operating point its gate actually cleared. A pre-2026-08-09
.cfg reads 0.0, which is exactly the behaviour it was trained under.
Per-era line now prints "@margin>=X.XX" next to coverage, so a coverage drop
can be attributed to the operating point rather than guessed at.
Both build variants compile 0 errors / 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:04:37 -04:00
double DirectionalMargin ( void ) ;
fix: the operating point was fitted on bars the net had memorized
FitDirConfThreshold harvested its margin histogram from pass 2's own
backprop samples. Pairing every fit against the same era's OOS result
shows what that measured:
PAI era 1 IS 25% cov @ 66.1% (-0.8pp) -> OOS 64% (-3pp) gap +2.1pp
PAI era 76 IS 90% cov @ 79.6% (+12.7pp) -> OOS 65% (-2pp) gap +14.6pp
LSTM era 9 IS 77% cov @ 81.6% (+14.6pp) -> OOS 63% (-4pp) gap +18.6pp
The gap grows monotonically while OOS stays flat, so within a handful of
eras the curve stops describing behaviour on unseen bars. That is fatal
here specifically, because the objective branches on the SIGN of
(p - break-even): the memorized curve reads +12pp at 95% coverage, so
coverage x (p - p0) correctly maximises coverage and returns ~0.02 - fire
on every bar. The "p < p0 -> get more selective" branch, which is the
actual regime and the entire point of 983a6a3, could never fire because IS
never showed p < p0.
Carve a calibration slice out of the IS span - DIR_CONF_CALIB_PCT_OF_IS,
purged from backprop by one label horizon on BOTH sides (the far-side
purge is not optional: without it the newest training bars carry labels
partly decided by price action inside the slice, putting the memorization
straight back into the curve). Score it in a new chunked pass 2.5, after
pass 2 has trained and before pass 3 grades - the only position where the
histogram is simultaneously not-trained-on, not-graded, and current with
the weights it will be applied to.
Costs 15% of the training data. Worth it beyond honesty: the deploy gate
needs dirPrecPct > chance + EDGE_MIN_SIGMAS*SE, and a threshold pinned
near zero dilutes any edge concentrated in the confident bars across every
bar the model calls, driving dirPrecPct toward chance by construction. A
threshold that can be selective is the only mechanism by which a small,
concentrated edge could ever clear that gate.
Also: a sparse histogram now KEEPS the previous threshold instead of
resetting to 0.0. A failed measurement must not decay to the most exposed
setting in the range.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 15:58:18 -04:00
//--- Reset / accumulate / fit, in the order the calibration walk calls them. See
//--- DIR_CONF_THRESHOLD_BINS and DIR_CONF_CALIB_PCT_OF_IS.
feat: fitted directional confidence threshold - selectivity gets a mechanism
The training loss and the selection metric wanted different things and only
the second one knew it. Logit-adjusted cross-entropy has no term for "how
often should I trade", so the head calls a direction on 87-91% of bars. The
selection metric is precision x coverage credit, saturating at the coverage
floor - above the floor extra calls earn NOTHING and only precision counts.
So selection wanted few good calls, the loss produced many mediocre ones, and
all selection could do was pick the least-bad era out of what it was handed.
Nothing pushed the model toward selectivity.
This gives the decision RULE the policy instead of distorting the loss (which
is estimating class probabilities correctly, and a probability estimate should
not be bent to encode a trading policy - Elkan 2001: estimate, then choose the
operating point separately). AdjustedSignalFromSoftmax now abstains unless the
winning direction's softmax margin over its best rival clears a fitted
threshold. Margin, not the winning probability: the latter moves with overall
calibration rather than with how close the decision actually was.
Fitted on IS, applied to OOS and live. Pass 2 already forward-passes every IS
sample, so the margin histogram is harvested there for free (primary
occurrences only, so the oversampled replay queue cannot skew the operating
point); the fit runs at the end of pass 2, BEFORE pass 3, so the deploy gate
grades the thresholded model on bars the threshold never saw. Fitting on
pass 3's own predictions would be choosing the operating point on the data
being graded - the best-of-N error corrected in five other places here.
Objective: maximise IS directional precision subject to still clearing the
SAME coverage floor the deploy gate uses (base rate x 0.25, re-derived
locally so the two cannot drift apart). Swept top-down in one pass; ties go
to the LOWER threshold, since equal precision for less coverage is strictly
worse. Under DIR_CONF_MIN_FIT_CALLS (200) it runs unthresholded rather than
on a guess.
The threshold is part of the MODEL, not the run: captured with
Net.CaptureWeights(), restored with the weights at both restore sites, and
appended to the .cfg under the same length-guard convention so a deployed
model reloads at the operating point its gate actually cleared. A pre-2026-08-09
.cfg reads 0.0, which is exactly the behaviour it was trained under.
Per-era line now prints "@margin>=X.XX" next to coverage, so a coverage drop
can be attributed to the operating point rather than guessed at.
Both build variants compile 0 errors / 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:04:37 -04:00
void ResetDirConfHistogram ( void ) ;
void AccumulateDirConfSample ( double margin , bool wasCorrect , bool isPrimaryBar ) ;
void FitDirConfThreshold ( void ) ;
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- The purge width around every held-out slice: the label's own measured mean resolution lag,
//--- because that is how far a label can leak across a split boundary.
int LabelResolutionBars ( void ) const { return ( int ) MathMax ( 1.0 , MathCeil ( MeanLabelLifespan ( ) ) ) ; }
fix: the operating point was fitted on bars the net had memorized
FitDirConfThreshold harvested its margin histogram from pass 2's own
backprop samples. Pairing every fit against the same era's OOS result
shows what that measured:
PAI era 1 IS 25% cov @ 66.1% (-0.8pp) -> OOS 64% (-3pp) gap +2.1pp
PAI era 76 IS 90% cov @ 79.6% (+12.7pp) -> OOS 65% (-2pp) gap +14.6pp
LSTM era 9 IS 77% cov @ 81.6% (+14.6pp) -> OOS 63% (-4pp) gap +18.6pp
The gap grows monotonically while OOS stays flat, so within a handful of
eras the curve stops describing behaviour on unseen bars. That is fatal
here specifically, because the objective branches on the SIGN of
(p - break-even): the memorized curve reads +12pp at 95% coverage, so
coverage x (p - p0) correctly maximises coverage and returns ~0.02 - fire
on every bar. The "p < p0 -> get more selective" branch, which is the
actual regime and the entire point of 983a6a3, could never fire because IS
never showed p < p0.
Carve a calibration slice out of the IS span - DIR_CONF_CALIB_PCT_OF_IS,
purged from backprop by one label horizon on BOTH sides (the far-side
purge is not optional: without it the newest training bars carry labels
partly decided by price action inside the slice, putting the memorization
straight back into the curve). Score it in a new chunked pass 2.5, after
pass 2 has trained and before pass 3 grades - the only position where the
histogram is simultaneously not-trained-on, not-graded, and current with
the weights it will be applied to.
Costs 15% of the training data. Worth it beyond honesty: the deploy gate
needs dirPrecPct > chance + EDGE_MIN_SIGMAS*SE, and a threshold pinned
near zero dilutes any edge concentrated in the confident bars across every
bar the model calls, driving dirPrecPct toward chance by construction. A
threshold that can be selective is the only mechanism by which a small,
concentrated edge could ever clear that gate.
Also: a sparse histogram now KEEPS the previous threshold instead of
resetting to 0.0. A failed measurement must not decay to the most exposed
setting in the range.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 15:58:18 -04:00
//--- CALIBRATION BAND BOUNDS, in pass-1 bar indices (0 = newest bar, so LARGER index = OLDER).
//--- The era's bars lay out, newest to oldest:
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
int CalibPurgeBars ( void ) const { return LabelResolutionBars ( ) ; }
fix: the operating point was fitted on bars the net had memorized
FitDirConfThreshold harvested its margin histogram from pass 2's own
backprop samples. Pairing every fit against the same era's OOS result
shows what that measured:
PAI era 1 IS 25% cov @ 66.1% (-0.8pp) -> OOS 64% (-3pp) gap +2.1pp
PAI era 76 IS 90% cov @ 79.6% (+12.7pp) -> OOS 65% (-2pp) gap +14.6pp
LSTM era 9 IS 77% cov @ 81.6% (+14.6pp) -> OOS 63% (-4pp) gap +18.6pp
The gap grows monotonically while OOS stays flat, so within a handful of
eras the curve stops describing behaviour on unseen bars. That is fatal
here specifically, because the objective branches on the SIGN of
(p - break-even): the memorized curve reads +12pp at 95% coverage, so
coverage x (p - p0) correctly maximises coverage and returns ~0.02 - fire
on every bar. The "p < p0 -> get more selective" branch, which is the
actual regime and the entire point of 983a6a3, could never fire because IS
never showed p < p0.
Carve a calibration slice out of the IS span - DIR_CONF_CALIB_PCT_OF_IS,
purged from backprop by one label horizon on BOTH sides (the far-side
purge is not optional: without it the newest training bars carry labels
partly decided by price action inside the slice, putting the memorization
straight back into the curve). Score it in a new chunked pass 2.5, after
pass 2 has trained and before pass 3 grades - the only position where the
histogram is simultaneously not-trained-on, not-graded, and current with
the weights it will be applied to.
Costs 15% of the training data. Worth it beyond honesty: the deploy gate
needs dirPrecPct > chance + EDGE_MIN_SIGMAS*SE, and a threshold pinned
near zero dilutes any edge concentrated in the confident bars across every
bar the model calls, driving dirPrecPct toward chance by construction. A
threshold that can be selective is the only mechanism by which a small,
concentrated edge could ever clear that gate.
Also: a sparse histogram now KEEPS the previous threshold instead of
resetting to 0.0. A failed measurement must not decay to the most exposed
setting in the range.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 15:58:18 -04:00
int CalibLoIndex ( int oosCutoff ) const { return oosCutoff + CalibPurgeBars ( ) ; }
//--- Zero (an empty band) whenever the era is too short to carve one without eating the training set;
//--- callers must treat that as "no calibration this era" and leave the threshold where it is.
int CalibBandBars ( int totalIter , int oosCutoff ) const
{
int isSpan = totalIter - CalibLoIndex ( oosCutoff ) - CalibPurgeBars ( ) ;
if ( isSpan < = 0 )
return 0 ;
return ( int ) ( isSpan * ( DIR_CONF_CALIB_PCT_OF_IS / 100.0 ) ) ;
}
int CalibHiIndex ( int totalIter , int oosCutoff ) const
{ return CalibLoIndex ( oosCutoff ) + CalibBandBars ( totalIter , oosCutoff ) ; }
2026-07-23 19:36:34 -04:00
//--- EMA-updates the persisted true class base rates (m_priorBuy/Sell/Neutral) from a just-finished
//--- era's true class counts. No-op on an empty/degenerate tally.
void UpdateClassPriors ( long buyCnt , long sellCnt , long neutralCnt ) ;
feat(ai): logit-adjusted loss, replacing oversampling and the post-hoc prior
Menon et al. 2021 (ICLR), "Long-tail learning via logit adjustment": add
tau*log(prior_c) to each class logit inside the training gradient. Softmax
CE on adjusted logits is consistent for BALANCED error - the metric
checkpoint selection already ranks on - so the loss and the deploy decision
finally optimize the same thing.
The engine already computed a true softmax + categorical-CE gradient and
wrote it over the per-neuron sigmoid delta, so this is an offset added to
three logits in the two places that gradient is built (backProp scalar path
and backPropOCL). No backend, kernel or DLL change; the forward pass and
every inference path are untouched, which is the point - the network learns
to absorb the offset, so its raw argmax becomes the balanced-optimal
decision with nothing applied at inference.
Replaces rather than stacks. Minority replay is disabled while this is on,
and the post-hoc inference prior is forced off. Stacking is not a
theoretical worry: simulated on the measured 1118/1119/34298 distribution
in the weak-signal regime, plain CE collapses to Neutral (33.4% balanced,
Buy 0%), replay reaches 48.1%, logit adjustment 50.9% with better balance -
and BOTH together score 45.4% with Neutral recall at 0%, worse than either
alone. Buda et al. 2018 predicts exactly that.
Motivation from the six-chart run: every topology took one direction to
~50% recall and abandoned the other, the direction chosen arbitrarily (the
batch-norm control went Buy 1% / Sell 42%, the inverse of the other five).
One era in 1,301 cleared the per-class recall floor.
Fingerprinted conditionally, so the converged 60.7% models on disk keep
their filenames and stay loadable as the fallback.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 19:05:14 -04:00
//--- Installs tau*log(prior_c) on Net from the freshly measured priors. Called once per era
//--- start, straight after UpdateClassPriors, so the offsets track the same distribution the
//--- era is scored against. No-op (and actively clears stale offsets) when the input is off.
void ApplyLogitAdjustment ( void ) ;
2026-07-23 19:36:34 -04:00
//--- Small binary sidecar (fileName + ".stats") persisting the calibration state that must survive a
//--- restart for live trading to behave like training: the true class priors and m_confidenceCalScale.
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
bool SaveModelStats ( string fileName , bool common ) { return m_modelPersistence . SaveModelStats ( fileName , common ) ; }
bool LoadModelStats ( string fileName , bool common ) { return m_modelPersistence . LoadModelStats ( fileName , common ) ; }
2026-08-22 00:24:45 -04:00
//--- Deploy-time (chart, backend present) self-check: runs the just-saved deployed model through
//--- both the backend and a temporary pure-MQL5 (CNet::SetCpuInference) clone on the same input
//--- window and returns true only if the outputs match within CPU_INFERENCE_MAX_DIFF.
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
bool ValidateCpuInference ( void ) { return m_modelPersistence . ValidateCpuInference ( ) ; }
2026-07-25 01:07:21 -04:00
//--- Build the panel's "Buy/Sell accuracy: IS x% OOS y%" line (directional win-rate, Neutral excluded)
//--- from the cumulative counts (m_cumIsCorrect etc.); returns "...: measuring..." until at least one
//--- directional call has been validated. Shared by the training and live/complete simple panels.
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
string ComputeCompoundedAccuracyLine ( void ) { return m_chartUI . ComputeCompoundedAccuracyLine ( ) ; }
2026-08-22 00:24:45 -04:00
//--- Persist/restore the drawn directional arrows (the "WarSig_" objects) to a sidecar file so
//--- they survive an EA remove/re-add, recompile, or restart WITHOUT a retrain - the chart
//--- objects are destroyed on unload (destructor PurgeChart) and OnInit has no other way to
//--- bring them back.
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
bool SaveChartSignals ( bool pruneChartObjects = true ) { return m_chartUI . SaveChartSignals ( pruneChartObjects ) ; }
void LoadChartSignals ( void ) { m_chartUI . LoadChartSignals ( ) ; }
fix(chart): arrows survived the EA that drew them - persist, then clear
Reported: on deinit the panel and status label go, the signal arrows stay.
Two independent causes, both fixed here.
1. It was partly deliberate. ShutdownChartCleanup carried a second
behaviour selected by a `preserveChartArrows` flag derived from the
deinit reason: on RECOMPILE / PARAMETERS / CHARTCHANGE / TEMPLATE the
arrows were left on the chart on purpose, to avoid a reload flicker.
That branch IS the reported symptom, an operator cannot tell it apart
from a cleanup that failed, and it was outright wrong whenever the
reload changed the config - REASON_PARAMETERS means exactly that, and
the preserved arrows then belonged to a model the chart no longer
runs, with nothing marking them stale. It is gone, along with the flag
and m_purgeChartOnDestruct. One path now: persist, clear, restore on
the next attach.
2. Whatever remains was unfalsifiable. PurgeChart was a single
ObjectsDeleteAll(prefix) whose return value was discarded, with no
caller ever looking at the chart again - so "the arrows are still
there" and "the arrows were never there" produced identical evidence,
which is why the report survived three sessions. It now verifies:
after the bulk delete it walks the OBJ_ARROW-typed list (a handful of
objects, not the whole chart), deletes any surviving WarSig_ by name,
and says so. Costs one typed scan when the bulk delete works, which is
the normal case; names the root cause when it does not.
Every failure mode of SaveChartSignals was also silent - it returned void
and had three bare early returns. It returns bool now, logs the open
error with the filename, and the shutdown purge is CONDITIONAL on it: for
a converged model the chart objects are the only copy of its signal
history (nothing redraws them - the renderer runs per training era and a
deployed model has none left), so a chart left littered because the disk
write failed beats a clean chart bought by destroying the history. Either
way the log now says which happened.
Also states the user's rule once, where arrows come back rather than
across InitNeuralNetwork's several exits: no weights loaded for this
config => clear the sidecar and start visually clean. A fresh run must
not inherit calls it never made, and the first save would otherwise adopt
them (the sidecar is rebuilt by scanning the chart).
Compiles 0 errors / 0 warnings, standard and Market. Needs redeploy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:28:34 -04:00
//--- The shutdown half of that pair: persist, THEN clear the chart, and report both counts. See the
//--- definition for why the order is fixed and why the clear is conditional on the write.
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
void PersistAndClearChartSignals ( void ) { m_chartUI . PersistAndClearChartSignals ( ) ; }
fix: clear stale signal arrows when a fresh model starts at era 0
Arrow cleanup existed on two paths - the panel's reset-weights, and the
topology-mismatch discard - but both are gated on there being a saved .nnw to
delete. The third case had no cleanup at all: a fresh topology at era 0 with no
weights behind it, which is what a changed config produces. A new fingerprint
makes a new m_fileName, so the previous model's files are not "discarded", they
are simply not this model's files, and nothing ever cleared the chart.
That is not cosmetic. Arrows outlive the model that drew them twice over:
1. The chart objects live in the CHART, not the sidecar, so they survive a
remove/re-add, a recompile, a restart and a fresh deploy no matter what
happens to any file on disk.
2. SaveChartSignals() rebuilds the sidecar by SCANNING the chart for
SIG_ARROW_PREFIX objects. So the first save of the fresh run adopts the
dead model's calls and writes them out under the NEW model's filename -
laundering them into the new model's history where nothing can separate
them afterwards.
Extracted the duplicated cleanup into ClearPersistedChartSignals(reason) - it
cancels the deferred restore queue, deletes m_fileName + ".arrows", clears the
namespaced chart objects and logs why - and called it from all three paths.
The call sits at the BuildFreshTopology() call site, not inside it: the genetic
tuner rebuilds a throwaway topology per candidate (AutoTune.mqh) and must never
touch the chart. All three sites run after m_fileName has its config fingerprint
appended, so they target the right sidecar.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 10:13:01 -04:00
//--- Wipe this model's drawn arrows AND their .arrows sidecar, plus any deferred restore still in
//--- flight. Call from every path that discards or replaces the trained weights - see the definition
//--- for why leaving them behind resurrects a dead model's calls through SaveChartSignals.
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
void ClearPersistedChartSignals ( const string reason ) { m_chartUI . ClearPersistedChartSignals ( reason ) ; }
2026-08-22 00:24:45 -04:00
//--- Deferred ("async") half of LoadChartSignals: LoadChartSignals only PARSES the sidecar into
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
//--- the arrow-restore buffers (an ~80KB read - instant) and returns, so OnInit never blocks;
2026-08-22 00:24:45 -04:00
//--- this then creates the chart objects in ARROW_RESTORE_BUDGET_MS slices, driven by the same
//--- 500ms timer that already paces training.
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
void AdvanceChartSignalRestore ( void ) { m_chartUI . AdvanceChartSignalRestore ( ) ; }
2026-08-22 00:24:45 -04:00
//--- Deferred ("async") half of StartChartSignalRescan (public, defined inline further down):
//--- drains the per-bar inference loop in ARROW_RESTORE_BUDGET_MS slices off PollTraining's
//--- timer instead of blocking the button-click handler for however long a full lookback scan
//--- takes.
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
void AdvanceChartSignalRescan ( void ) { m_chartUI . AdvanceChartSignalRescan ( ) ; }
//--- The arrow-restore queue, the rescan queue/tally, m_lastArrowsSaved and the purge-mismatch
//--- latch all live on CChartUI now, next to the methods that own them - see Expert\Chart\ChartUI.mqh.
2026-07-14 22:36:27 -04:00
bool ResizeBuffers ( int barIndex ) ;
bool RefreshData ( ) ;
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
bool BufferTempData ( int idx ) { return m_featureBuilder . BufferTempData ( idx ) ; }
2026-08-22 00:24:45 -04:00
//--- Assembles the full m_historyBars-wide input window ending AT bar r into TempData, OLDEST
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- BAR FIRST. See the definition comment in Expert\Features\FeatureBuilder.mqh for the
//--- measurement behind that.
bool BuildFeatureWindow ( int r ) { return m_featureBuilder . BuildFeatureWindow ( r ) ; }
2026-07-14 22:36:27 -04:00
//--- shared by OnTickHandler() and the timer-driven PollTraining() - see definition
void ScheduleTrainingIfNeeded ( void ) ;
void Train ( datetime StartTrainBar = 0 ) ;
fix: a restart no longer loses the measured geometry or the training window
Terminal restart, 22:25: all four resumed models sat on empty windows with
enum 2:6 barriers. Three interlocking causes, all visible in one log excerpt:
1) THE PRE-SCAN WINDOW WAS SIZED BY THE SAVED WATERMARK. A resumed model's
dtStudied sits at its last studied bar, so Bars(dtStudied, now) ~ 0 and the
resumed-model MI pre-scan built a zero-bar "complete" label cache - logged as
"Buy: 0 | Sell: 0 | Neutral: 0". Train()'s own era start RESETS dtStudied to
the training-window rule before computing its window; the pre-scan did not.
The rule is now factored into TrainWindowStart() and both use it. The scan
also refuses to arm before SERIES_SYNCHRONIZED (it ran in the same second as
OnInit), and deployed models keep their watermark - for them it gates
inference recency, not a training window.
2) THE HORIZON LATCHED ON AN INDICATOR WARM-UP. ComputeBarrierHorizonBars ran
against a ZigZag with 0 calculated legs, fell back, and EnsureBarrierHorizon
latched fallback(32) x slMult x tpMult = 384 for the process lifetime. A
leg-starved horizon is now PROVISIONAL: re-resolved on the next rebuild, the
label cache wiped if it moved (labels from two horizons answer different
questions), and the geometry deriver refuses to run from it - a pair derived
over a warm-up window would get PINNED.
3) THE DERIVED GEOMETRY WAS NEVER PERSISTED. The .cfg is written at model
creation and at weights-reset - both BEFORE era 0 derives - so the measured
pair lived only in memory: every restart read back zeros, adopted nothing,
fell back to the enum barriers, and the era-0-only gate meant a resumed model
could NEVER re-derive. A full day of training on 3.33/1.62 resumed as 2:6.
Now: the settled pair is pinned to the .cfg the moment derivation completes
(one-shot, atomic write), and the derive gate accepts any model with no
pinned pair, not just era 0 - mid-run stability is carried by
m_geometryDerived itself, which never allows a second derivation.
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 22:40:43 -04:00
//--- the training window's start time - shared by Train()'s era start and the label-cache pre-scan
datetime TrainWindowStart ( datetime startTrainBar ) ;
2026-07-14 22:36:27 -04:00
//--- outer loop around Train(): when AutoTuneIndicators is on, tries randomized AD indicator
//--- input variations across m_indicatorTuneTrials calls to Train(), keeping the best-OOS one
void TuneIndicatorsAndTrain ( datetime StartTrainBar = 0 ) ;
fix: live inference queried the 1-tick forming bar - a window training never built
RefreshLatestSignal ran at the first tick after a bar opens and built its
window at r=0: series index 0 at that instant is a candle with one tick of
data - (close-open)/atr ~ 0, high ~ low, degenerate volume, indicators on a
1-tick bar. Training never produces such a window (every labeled bar is fully
closed, entry at that bar's CLOSE), so the deployed model's final timestep -
the one the LSTM/HYBRID output is keyed to - was out-of-distribution on every
live decision, and pass 3's deploy-gate OOS scores measured a different query
than live executed. The parity index is r=1: the newest CLOSED bar, whose
close IS the current price - the exact instant the label's hypothetical entry
happens. Single backtests shared the old skew (same r=0), which is why the
tester agreed with live while both disagreed with training.
Bookkeeping split that the index change forces: m_lastBarTime/dtStudied stay
anchored to the FORMING bar's open (they gate against SERIES_LASTBAR_DATE;
anchoring at bar 1 would re-fire the refresh every tick), while bt - the
arrow, its High/Low placement, and NMS declustering - anchors to the decision
bar, now matching the rescan path's convention.
Also: a failed refresh no longer trades the previous bar's signal for the
whole bar. RefreshLatestSignal returns success, zeroes dPrevSignal on failure
(no opinion beats a stale one), and RefreshConvergedSignal advances dtStudied
only on success so the next tick retries - the tester path (m_lastBarTime)
already worked this way; this is the live path catching up.
FORCES RE-VALIDATION of deployed models: the effective live query distribution
changes. Bundled with the backprop transpose fix's retrain.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:10:23 -04:00
//--- recomputes dPrevSignal/chart arrow for the newest CLOSED bar (bar 1 - see the definition's
//--- 2026-08-11 parity comment); used after restoring a checkpointed model at the end of Train()
2026-08-22 00:24:45 -04:00
//--- so the live signal matches the deployed weights.
fix: live inference queried the 1-tick forming bar - a window training never built
RefreshLatestSignal ran at the first tick after a bar opens and built its
window at r=0: series index 0 at that instant is a candle with one tick of
data - (close-open)/atr ~ 0, high ~ low, degenerate volume, indicators on a
1-tick bar. Training never produces such a window (every labeled bar is fully
closed, entry at that bar's CLOSE), so the deployed model's final timestep -
the one the LSTM/HYBRID output is keyed to - was out-of-distribution on every
live decision, and pass 3's deploy-gate OOS scores measured a different query
than live executed. The parity index is r=1: the newest CLOSED bar, whose
close IS the current price - the exact instant the label's hypothetical entry
happens. Single backtests shared the old skew (same r=0), which is why the
tester agreed with live while both disagreed with training.
Bookkeeping split that the index change forces: m_lastBarTime/dtStudied stay
anchored to the FORMING bar's open (they gate against SERIES_LASTBAR_DATE;
anchoring at bar 1 would re-fire the refresh every tick), while bt - the
arrow, its High/Low placement, and NMS declustering - anchors to the decision
bar, now matching the rescan path's convention.
Also: a failed refresh no longer trades the previous bar's signal for the
whole bar. RefreshLatestSignal returns success, zeroes dPrevSignal on failure
(no opinion beats a stale one), and RefreshConvergedSignal advances dtStudied
only on success so the next tick retries - the tester path (m_lastBarTime)
already worked this way; this is the live path catching up.
FORCES RE-VALIDATION of deployed models: the effective live query distribution
changes. Bundled with the backprop transpose fix's retrain.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:10:23 -04:00
bool RefreshLatestSignal ( ) ;
2026-07-15 21:47:37 -04:00
//--- inference-only "new bar" handler used once m_trainingComplete is true - see
//--- ScheduleTrainingIfNeeded()'s declaration comment for why this must NOT call Net.backProp()
void RefreshConvergedSignal ( void ) ;
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
//--- Online continual-learning step (live chart only), the shadow-net lifecycle, and the sample-
//--- weight rule - all now live on COnlineLearning (m_onlineLearning); one-line forwards at their
//--- original position.
perf(deinit): I/O-free chart cleanup, dead-panel purge, skip clean weight saves
The 18:23 terminal close (20260825.log) killed two of six charts inside
OnDeinit: they printed "shutting down" then nothing for 5.9 s until
"Abnormal termination", stranding ~700 objects each - including the one
family no prefix sweep can reach, the control panel (CAppDialog names
its 15 objects <numeric instance id><control>, and a re-attach mints a
new id, so a killed panel is a permanent ghost; XTIUSD carried one
across sessions). The stall sat in the two file writes that preceded
all visible cleanup while the four sibling charts flooded the same
2013-era disk - the ~4x18MB-per-chart shutdown weight saves.
Three changes:
1. OnDeinit touches no file until the chart is clean. CVoteArrowStore
splits Save() into Snapshot() (the chart scan, in memory) and
WriteSnapshot() (the disk half, consuming). New order: status label,
vote-arrow snapshot, prefix sweep, panel destroy - all object ops -
then member sidecars, final sweep, timings, and only then the
visibility file, the vote-arrow write and the weight saves.
2. PurgeOrphanedPanelObjects() at OnInit: deletes numeric-prefix
CAppDialog ghosts by name (6 chrome + 9 buttons), qualifying a
prefix only when >=4 of OUR button names carry it, so a foreign
dialog sharing stock chrome names is never touched.
3. m_netDirty: set by every net mutation (both backProp sites, both
RestoreWeights sites, online learning conservatively, panel reset),
cleared only on a successful Net.Save. Shutdown AND the per-bar
autosave now skip the ~18MB write when the net is provably unchanged
- for converged ensembles that is every save - which removes the
very flood that starved the sibling charts. .stats still writes
every time (small; carries the vote record and calibration). A
skipped save leaves the .nnw header dtStudied stale, which is the
already-handled attach-after-offline-gap case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 18:46:51 -04:00
//--- Conservative dirty-marking: the step self-gates and most calls learn nothing, but detecting
//--- "actually learned" here would couple this wrapper to the learner's internals. Over-marking
//--- costs one redundant autosave per bar for online-learning models - exactly the pre-flag
//--- behaviour - while models with the feature off keep a provably clean net.
void OnlineLearnStep ( void )
{
if ( m_onlineLearning . Enabled ( ) & & ! m_inferenceOnly & & ! m_trainRunActive )
m_netDirty = true ;
m_onlineLearning . OnlineLearnStep ( ) ;
}
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
double OnlineSampleWeight ( ENUM_SIGNAL trueSignal , double pBuy , double pSell , double pNeutral )
{ return m_onlineLearning . SampleWeight ( trueSignal , pBuy , pSell , pNeutral ) ; }
void EnsureShadowNet ( void ) { m_onlineLearning . EnsureShadowNet ( ) ; }
void SaveShadowNet ( const double & indicatorParams [ ] ) { m_onlineLearning . SaveShadowNet ( indicatorParams ) ; }
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- method of initialization of the indicators. InitOpen/InitClose/InitHigh/InitLow/InitTime/
2026-08-24 18:26:25 -04:00
//--- InitZigZag stay HERE (Expert\AIBase\Features.mqh) - they manage m_Open/m_Close/m_High/
//--- m_Low/m_Time/m_zigZag, which are genuinely shared with Labels.mqh/AutoTune.mqh/Training.mqh
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- (real .GetData() reads there, not just this file), so moving their Create/lifecycle would only
//--- relocate a hub behind dozens of pure-relay wrappers - same judgment as Topology's boot sequence.
2026-07-14 22:36:27 -04:00
bool InitOpen ( CIndicators * indicators ) ;
bool InitClose ( CIndicators * indicators ) ;
bool InitHigh ( CIndicators * indicators ) ;
bool InitLow ( CIndicators * indicators ) ;
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
bool InitVolumes ( CIndicators * indicators ) { return m_featureBuilder . InitVolumes ( indicators ) ; }
2026-07-14 22:36:27 -04:00
bool InitTime ( CIndicators * indicators ) ;
ditch(features): remove the eight dead feature groups from the input matrix
RSI, MACD, Ichimoku and the five AD/Wyckoff indicators (CumulativeDelta,
ShorteningOfThrust, WyckoffEventStream, WyckoffFailedStructure,
WyckoffSignificantBarInversion). All eight inputs shipped false and each carries a
closed verdict: the three oscillators are the same patterns that measured at chance
as entries, and the Wyckoff family returned zero out-of-sample on five independent
instruments - which is what closed the context score.
RETRAIN-NEUTRAL, and this one is worth stating precisely because the change looks
larger than it is. Every removed group contributed `flag ? N : 0` to the input
width, and every flag was false, so the width was ALREADY zero for all eight: no
.nnw's input layer changes. On the fingerprints, UseRSI and the five AD flags were
hashed unconditionally and become literal 0 legacy slots (the convention the
m_focalGamma slot above them already uses); UseMACD/UseIchimoku were appended only
when enabled, so their segments simply never appear - byte-identical to every
fingerprint ever produced, since neither ever shipped on.
CADIndicatorTuner IS DELIBERATELY NOT SHRUNK. Its flat parameter array is persisted
inside every .nnw, and Unflatten() rejects a size mismatch by falling back to
constructor defaults - so dropping the dead fields would silently revert the tuned
MA period of every model on disk while keeping its trained weights. That is the
feature/weight mismatch this project has already paid for twice, and it is not
worth 200 lines. AD_TUNE_PARAM_COUNT stays 42, the dead slots are still written and
read, and AutoTune's ParamOwner gate now matches only owner 5 (MA) so nothing
searches them. The class comment says all of this at the declaration.
Also renamed ReInitADIndicators -> ReInitTunableIndicators: it rebuilds exactly one
indicator now, and a name saying "AD" for the MA handle is the kind of stale label
that gets believed later. Its release-AFTER-recreate ordering is untouched - that
is a documented fix, not bookkeeping.
Compile-verified in the stage copy: 0 errors, 0 warnings, against the same 0/0
baseline.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 09:21:03 -04:00
//--- addToCollection=false is used by ReInitTunableIndicators() to rebuild an already-collected
2026-07-22 22:51:04 -04:00
//--- handle's params (here: a re-tuned period) without re-adding the (same) pointer into
//--- indicators a second time
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
bool InitMA ( CIndicators * indicators , bool addToCollection = true )
{ return m_featureBuilder . InitMA ( indicators , addToCollection ) ; }
2026-08-24 18:26:25 -04:00
bool InitZigZag ( CIndicators * indicators , bool addToCollection = true ) ;
2026-07-14 22:36:27 -04:00
//--- common=false targets a LOCAL (non-shared) file - used by the tester/optimizer per-agent
//--- weight cache so cross-pass reuse never touches the production FILE_COMMON config/weights.
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
bool SaveTopologyConfiguration ( string fileName , int initialNeuronsCount , int hiddenLayersCount , double neuronsReduction , int minNeuronsCount , int optimizationAlgo , int historyBars , int outputNeuronsCount , int neuronsCount , int studyPeriod , int minTrainYear , bool isInitialized , int stopTrainWR , int fractalPeriods , int convFilterCount , int lstmHiddenSize , bool common = true )
{ return m_modelPersistence . SaveTopologyConfiguration ( fileName , initialNeuronsCount , hiddenLayersCount , neuronsReduction , minNeuronsCount , optimizationAlgo , historyBars , outputNeuronsCount , neuronsCount , studyPeriod , minTrainYear , isInitialized , stopTrainWR , fractalPeriods , convFilterCount , lstmHiddenSize , common ) ; }
feat(nn): derive dense depth, train on all history, pin the shape in .cfg
Completes the derived-topology work. Three inputs removed.
AIType loses its depth suffix - AI_MLP/AI_CONV/AI_LSTM/AI_HYBRID, five
entries instead of eight. Depth is now derived from the two endpoints
the taper already has to connect (derived first-layer width, output-tied
final width) at a 2x per-layer compression target, clamped [2..5].
Asking a user to pick a layer count while the code derives the widths
those layers taper between was asking for half a decision: at 64 units
tapering to 12, four layers compress by 1.4x per step and five by 1.3x,
so the extra depth bought no abstraction. On the shipping H1/10y default
the derivation lands on 3 layers - the depth that actually won Run 2.
StudyPeriods removed. There is no case for training on less data than
the broker provides at a ~6% directional base rate; the honest
generalization read comes from the OOS holdout, not from withholding
history. Training now starts at the earliest available bar, floored by
MinTrainYear, which answers a different question (excluding dubious
pre-history) and stays.
That required closing the hazard the old code documented: the capacity
budget now MEASURES the symbol's real bar count, and a topology derived
from a measurement would widen as history downloads. Both ends are now
pinned. Every derived value left the weights-filename fingerprint -
keying a filename on a measured quantity means the EA looks for a file
that does not exist, starts from era 0 and orphans a trained model,
silently, because a missing cache is the normal first-run state. The
shape lives in the .cfg instead, where LoadAndCompare now ADOPTS the
four derived fields rather than diffing them; a mismatch there would
discard a fully-trained model over nothing the user did. Two fields
appended to the .cfg for the conv/LSTM stages, length-guarded on read
because FileReadInteger past EOF returns 0 with no error.
ForceHiddenLayers, a compile-time constant like DebuggingMode, pins
depth for diagnostic comparisons. It joins the fingerprint only when
non-zero, so forced depths get their own files - sequential comparisons
only, not simultaneous from one .ex5.
Derived shape, H1/10y defaults (21 features x 20 bars): first layer 64,
3 dense, 8 conv filters, 16 LSTM units. The LSTM block halves from
~58k to ~28k weights.
Both builds compile 0 errors, 0 warnings. Re-keys existing models.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 10:05:40 -04:00
//--- The four DERIVED shape fields are by REFERENCE and are ADOPTED from the .cfg, not compared
2026-08-22 00:24:45 -04:00
//--- against it.
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
bool LoadAndCompareTopologyConfiguration ( string fileName , int & initialNeuronsCount , int & hiddenLayersCount , double neuronsReduction , int minNeuronsCount , int optimizationAlgo , int & historyBars , int outputNeuronsCount , int neuronsCount , int minTrainYear , bool isInitialized , int stopTrainWR , int fractalPeriods , int & convFilterCount , int & lstmHiddenSize , bool common = true )
{ return m_modelPersistence . LoadAndCompareTopologyConfiguration ( fileName , initialNeuronsCount , hiddenLayersCount , neuronsReduction , minNeuronsCount , optimizationAlgo , historyBars , outputNeuronsCount , neuronsCount , minTrainYear , isInitialized , stopTrainWR , fractalPeriods , convFilterCount , lstmHiddenSize , common ) ; }
2026-08-23 21:18:32 -04:00
//--- CopyFileWithRetry()/CopySharedFile() moved to System\SharedFileCopy.mqh - pure functions,
//--- no member ever touched them, so they need no seam on this class at all. Retry helper for the
//--- tester/opt seed-copy race: a live chart's own atomic Save() (write .savetmp, then FileMove()
//--- over the real file) can hold the source or destination file for a moment, and a concurrent
//--- FileCopy/FileOpen from a Strategy Tester agent reading the SAME production file can hit a
//--- transient Windows sharing violation in that narrow window.
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
bool LoadNetWithRetry ( double & indicatorParams [ ] ) { return m_modelPersistence . LoadNetWithRetry ( indicatorParams ) ; }
2026-07-14 22:36:27 -04:00
//--- input data
bool m_useVolumes ;
bool m_useTime ;
bool m_useATR ;
2026-07-22 22:51:04 -04:00
//--- Uses its own period (m_indicatorTuner.maPeriod), fed as ATR-normalized OHLC distance-from-MA
//--- (4 values, same convention as the base close-open/high-open/low-open features) plus the MA's
//--- own bar-over-bar change (1 value, ATR-normalized like every other price-domain feature here -
//--- not volume's previous-bar-ratio scheme, since a moving average lives in price units and
//--- already has ATR as its natural scale reference). See BufferTempDataCompute()'s m_useMA block
//--- for the exact 5 values. maPeriod starts equal to the Classic Signals PeriodMA input (see
//--- CADIndicatorTuner's constructor) but may diverge from it once AutoTuneIndicators searches a
//--- trial - the Classic Signals MA vote itself is untouched by that search, since it needs no
//--- training/warm-up and there is nothing for a tuning trial to validate it against.
2026-07-22 17:17:23 -04:00
bool m_useMA ;
2026-08-22 00:24:45 -04:00
//--- Nine normalized swing-context features: 5 confirmed-pivot values plus 4 recent-price-action
//--- ones (Donchian position at 20/50 bars, 20-bar return, 20-bar SMA extension) giving fresh
2026-08-24 18:26:25 -04:00
//--- context the >=100-bar-stale pivot anchor cannot. Reads the same m_zigZag the labels come
2026-08-22 00:24:45 -04:00
//--- from, and is never tuned for the same reason the label side is not.
2026-07-19 11:04:38 -04:00
bool m_useSwingContext ;
feat: add configurable news event proximity/impact as an NN input feature
Price, time, volume, and volatility were already trained-model input
features; the real economic calendar (already used for the live
NewsFilter veto) is now an optional one too, reusing
System/NewsRelevance.mqh's symbol-relevance logic from the prior fix.
New EnableNews/NewsFeatureWindowMinutes inputs gate two features per
bar: minutes-since and minutes-until the nearest symbol-relevant
calendar event, impact-weighted. Deliberately limited to proximity +
impact, not actual-vs-forecast deviation - release schedules are
public knowledge ahead of time (not lookahead bias to use for a
historical training bar), but a release's actual outcome is not.
Wired identically to the existing EnableVolume/EnableTime/EnableATR
toggles: InitIndicators() accounts for the +2 neuron count,
BufferTempDataCompute() appends the two feature values, PAI/CONV/LSTM
all wired in Warrior_EA.mq5. Compiled clean (MetaEditor, 0 errors/0
warnings).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-18 17:26:04 -04:00
//--- see System\NewsRelevance.mqh's declaration comment for what this feature actually encodes
//--- (event proximity + impact, not actual-vs-forecast deviation) and why the forward-looking half
//--- of it isn't lookahead bias.
bool m_useNews ;
int m_newsFeatureWindowMinutes ;
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
//--- Cross-asset panel: the only feature block here whose inputs are NOT a transform of this
2026-08-22 00:24:45 -04:00
//--- symbol's own OHLCV series.
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
bool m_useCrossAsset ;
CCrossAssetPanel m_crossAsset ;
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
bool BuildCrossAssetPanel ( int bars ) { return m_featureBuilder . BuildCrossAssetPanel ( bars ) ; }
2026-08-11 21:29:14 -04:00
//--- Train->serve parity for the panel (2026-08-11): the pair set is a MEASURED property of the
//--- terminal, so like the derived barrier pair it is pinned in the .cfg, not the filename hash
2026-08-22 00:24:45 -04:00
//--- (see BuildConfigFingerprint's XA note).
2026-08-11 21:29:14 -04:00
string m_crossAssetPairsPinned ;
bool m_crossAssetCfgSaved ;
2026-08-16 13:39:00 -04:00
//--- Alternative-data panel (2026-08-16): the second feature block whose inputs are not a
//--- transform of this symbol's own series, and the first whose inputs are not derivable from
//--- the terminal at all - COT positioning, the VIX complex, macro series, collected and
2026-08-22 00:24:45 -04:00
//--- publication-stamped by research/altdata, served as plain CSVs.
2026-08-16 13:39:00 -04:00
bool m_useAltData ;
2026-08-22 00:24:45 -04:00
//--- EnableAltData input, distinct from m_useAltData: the input says the OPERATOR wants the
//--- block, m_useAltData says it is actually contributing features (input on AND file present
//--- AND >=1 column).
2026-08-16 15:12:54 -04:00
bool m_altDataEnabled ;
2026-08-16 20:04:13 -04:00
//--- One-shot guard for the "data landed after the model was pinned" warning - the upkeep tick
//--- runs every 30 minutes and this must not become a recurring line nobody reads.
bool m_altDataLateWarned ;
2026-08-16 13:39:00 -04:00
CAltDataPanel m_altData ;
string m_altDataNamesPinned ;
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
string ReadAltDataPinFromCfg ( void ) { return m_modelPersistence . ReadAltDataPinFromCfg ( ) ; }
2026-08-22 00:24:45 -04:00
//--- Spread as a feature. Measured as the strongest single feature in research/test_spread.py,
//--- though see the feature block for what it actually encodes and why that is less than it
//--- first appears.
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
bool m_useSpreadFeature ;
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- m_spreadSeries[]/m_spreadSeriesBars/m_spreadSeriesAnchor/m_crossAssetAnchor moved onto
//--- CFeatureBuilder as real members (exclusive, ctor-init-list only elsewhere).
bool EnsureSpreadSeries ( int bars ) { return m_featureBuilder . EnsureSpreadSeries ( bars ) ; }
2026-07-14 22:36:27 -04:00
public :
CExpertSignalAIBase ( void ) ;
~ CExpertSignalAIBase ( void ) ;
2026-08-22 00:24:45 -04:00
//--- Reload the alt-data panel after CAltDataFetch rebuilt the feature CSV (OnTimer path, live
//--- only). Safe against the per-bar feature cache because new alt rows only ever matter to a
//--- NEW D1 bar, which resets that cache anyway.
2026-08-16 13:59:03 -04:00
void AltDataReload ( void )
{
2026-08-16 20:04:13 -04:00
//--- Gated on the OPERATOR's switch, NOT on m_useAltData. m_useAltData latches false at init
//--- whenever the CSV was absent, so gating the reload on it made the EA structurally unable to
//--- consume data IT HAD JUST DOWNLOADED: on the first run after the alt-data folder is wiped -
//--- the normal pre-test routine here - the models are built ~30s BEFORE the fetch completes, the
//--- reload became a permanent no-op, and the entire run trained on price alone while a complete
//--- feature file sat on disk. Measured 2026-08-16 on SP500 H4: models pinned at fingerprint
//--- 6de8ba37 (0 alt features) at 19:36:40, SP500_D1.csv rebuilt with 13 features at 19:37:13,
//--- and every era after that trained without them - silently, because nothing looked again.
if ( ! m_altDataEnabled )
return ;
int before = m_altData . FeatureCount ( ) ;
m_altData . Load ( m_symbol . Name ( ) , ( ENUM_TIMEFRAMES ) m_period ) ;
int after = m_altData . FeatureCount ( ) ;
2026-08-22 00:24:45 -04:00
//--- Loading here is INERT while m_useAltData is false (both consumption sites gate on it),
//--- so this cannot widen the feature vector out from under a model whose width is already
//--- pinned.
2026-08-16 20:04:13 -04:00
if ( before = = 0 & & after > 0 & & ! m_altDataLateWarned )
{
m_altDataLateWarned = true ;
Print ( ID + " : ALT DATA ARRIVED AFTER THIS MODEL WAS BUILT - " + IntegerToString ( after ) +
" features are on disk now, but this model's input width was pinned WITHOUT them, so it "
" is training on price alone and will keep doing so for the rest of this run. "
" RE-ATTACH THE EA (or reload the chart) to build models that actually train on the "
" alt-data block. This is what happens when the alt-data folder is empty at attach time "
" and the EA downloads it moments later. " ) ;
}
2026-08-16 13:59:03 -04:00
}
2026-08-22 00:24:45 -04:00
//--- THE FRAMEWORK ENTRY POINT, and the only InitIndicators an AI signal needs: every subclass
//--- differs in TOPOLOGY (AddCustomLayers), never in how the net is brought up.
refactor(signals): AI signal files are identity + topology, nothing else
Every AI signal repeated the same five-line InitIndicators override that
did nothing but call InitNeuralNetwork. The cause was an access mismatch,
not a design: CExpertSignalCustom declares InitIndicators public, the AI
base redeclared it PROTECTED, and each subclass had to redeclare it
public to be reachable by CExpert. Worse, the base's own override does a
different job entirely - it creates the OHLC/ZigZag feature indicators -
and InitNeuralNetwork called it back scope-qualified to stop the virtual
dispatch landing in the subclass. Two jobs, one virtual name, and a
recursion trap held off by a scope qualifier.
The feature-indicator step is now InitFeatureIndicators() (protected,
non-virtual, named for what it does) and the AI base carries the single
public InitIndicators override. CONV/HYBRID/LSTM/PAI/META drop their
copies and are now purely identity plus topology, which is the classic
signal file's shape.
Comment pass on ExpertSignalAIBase.mqh, -100 lines with every constant
and every measured number kept. Three claims in the tier block were
stale and inverted - it named CalibratedConfidenceMagnitude() as the
tiering input where the code deliberately uses the RAW magnitude, and it
described the signal DB as re-ranking each tier when ApplyPatternWeight
declines the DB from the end of era 1. Also dropped a paragraph whose
subject was a previous version of the comment, and moved two notes down
onto the constants they document (CONV_COMPRESSION_DIVISOR was 16 lines
and three unrelated defines away from its own text).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 08:57:54 -04:00
virtual bool InitIndicators ( CIndicators * indicators ) override
{
return InitNeuralNetwork ( indicators ) ;
}
feat(panel): commands reach signals down the filter tree, not through a registry
The control panel drove training by looping g_aiSignals[] - a
hand-maintained, MAX_AI_SIGNALS-capped, AI-only registry that had
already dropped an ensemble member on the floor once (609be10). A model
missing from it still trains and still votes, it just cannot be paused,
stopped, deployed or reset, and every button label is computed from the
same short list, so the panel described one set of models while acting
on another. Classic signals could not respond to a panel action at all.
Commands now walk the signal tree CExpert already owns:
Expert.DispatchSignalCommand(cmd) -> root signal -> every filter,
recursively, returning how many actually acted.
CExpertSignalCustom carries the seam (OnSignalCommand / HasSignalTrait,
both no-ops by default), so a classic signal opts in by overriding two
methods and needs no registration and no cap. CExpertSignalAIBase
implements the training commands over its existing Pause/Stop/Deploy/
Reset methods - the behaviour is unchanged, only its reach reported.
Button labels ask the same tree via CountSignalTrait, with
SIGTRAIT_TRAINABLE as an explicit denominator: "all paused" is
meaningless without knowing how many could be paused. Pause/Stop resolve
their toggle direction ONCE in the EA and hand every model the same
plain command, instead of each re-deriving the direction from its own
local state - which is how a mixed set ends up half paused. The alerts
now report the count acted on rather than assuming it.
Two dispatch bugs found on the way, both from a database guard copied
onto event delivery: CExpertSignalCustom::OnTickHandler and
::OnChartEventHandler each skipped any filter whose GetFilterID() is
"NULL". That id is a DB folder name, and CSignalNewsFilter,
CSignalSessionFilter and CSignalRiskGuard never set one - so all three
were silently receiving neither ticks nor chart events. The guard stays
where it belongs, on the paths that write pattern tables.
ENUM_CP_ACTION moves to Enumerations\GlobalEnums.mqh (now include-
guarded) because the Expert bases have to name it and the panel is
included long after them.
The AI-only lifecycle loops - PollTraining, the weight autosave,
AltDataReload, OnDeinit's shutdown cascade - still use g_aiSignals[] and
are untouched here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 09:05:50 -04:00
//--- CONTROL PANEL. Every training action the panel offers arrives here, through the filter tree
//--- rather than through a registry - see CExpertSignalCustom::OnSignalCommand.
virtual bool OnSignalCommand ( const ENUM_SIGNAL_COMMAND cmd ) override ;
virtual bool HasSignalTrait ( const ENUM_SIGNAL_TRAIT trait ) override ;
2026-07-14 22:36:27 -04:00
//--- "voting" that price will grow/fall, common to every AI signal (single market model)
virtual int LongCondition ( void ) ;
virtual int ShortCondition ( void ) ;
2026-08-22 00:24:45 -04:00
//--- |dPrevSignal| is already a 0..1 confidence for classification output (softmax probability
//--- of the winning class) and typically bounded for regression output (tanh-activated network);
//--- OpenParams() clamps regardless.
2026-07-17 23:21:12 -04:00
double CalibratedConfidenceMagnitude ( void ) const
{
double mag = MathAbs ( dPrevSignal ) ;
2026-07-27 15:52:39 -04:00
if ( ! MathIsValidNumber ( mag ) )
return 0.0 ;
2026-07-17 23:21:12 -04:00
if ( m_outputNeuronsCount = = 3 )
mag = MathMin ( 1.0 , mag * m_confidenceCalScale ) ;
2026-07-27 15:52:39 -04:00
if ( ! MathIsValidNumber ( mag ) )
return 0.0 ;
2026-07-17 23:21:12 -04:00
return mag ;
}
refactor(trade-mgmt): remove all confidence-scaled trade management
Five modes went, all of them staking real risk on the model's confidence:
Intelligent entry (ENTRY_INTELLIGENT), stop (SL_INTELLIGENT), target
(TP_INTELLIGENT), trailing (CTrailingIntelligent) and lot size
(CMoneyIntelligent's quarter-Kelly). With them, the Confidence_Source
input and the CONFIDENCE_SOURCE enum, whose only job was choosing which
number those five read.
The reason is calibration, not correctness: the confidence magnitude is
known to be miscalibrated against the label prior, so every one of these
modes multiplied money by a quantity whose units were never established.
The DB arm had a second, independent defect - since the tester DB guard
(SignalDatabaseActive) it reads 0 in tester and optimizer but non-zero
live, so any backtest of CONF_DB/CONF_BLENDED could not reproduce live
trading. And what the DB produces is a filter-RANKING win rate, not a
per-trade win probability.
Both confidence numbers are still recorded per trade (aiConfidence /
dbConfidence) and still bucketed against outcome in TradeJournalReport.
Recording is what keeps the question answerable; acting on it was the
part with no evidence behind it. ConfidenceBridge.mqh now carries an
explicit telemetry-only rule at the top.
ENUM ORDINALS PINNED. Removing a member vacated a value in four enums at
once and MT5 does not validate an enum input replayed from a saved .set
or a stored optimization pass. TRAILING_STRATEGY and
MONEY_MANAGEMENT_STRATEGY now carry explicit values so the survivors keep
the numbers they were saved as, and ValidateBarrierInputs is widened into
ValidateTradeManagementInputs covering SL_Mode, TP_Mode,
Entry_Multiplier, TrailingStrategy and MM_STRATEGY. Without that gate a
chart saved with the Intelligent stop would feed SL_Mode = -1 into a
multiplier now used verbatim, placing the stop on the wrong side of entry.
RETRAIN-NEUTRAL: neither SL_Mode nor TP_Mode appears in
BuildModelFingerprint() or ComputeDbConfigFingerprint() since the
swing-pivot target replaced the barrier labels. No .nnw, .cfg or .db
re-keys. Also drops the now-dead g_TradeRewardRiskRatio bridge, the
CMoneyRiskBase::AdjustRiskAmount hook and the unsigned AIConfidence().
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 10:10:20 -04:00
// Signed for direction-aware use: sign matches dPrevSignal's convention (+ buy, - sell, 0
// neutral/no signal yet). dPrevSignal == -2 is the "not yet studied" sentinel, not a real sell
// signal - treat it as no confidence. (The unsigned AIConfidence() override that sat here was
// removed 2026-08-25: its only callers were the confidence-scaled trade-management modes.)
2026-07-17 23:21:12 -04:00
virtual double SignedAIConfidence ( void ) override
{
if ( dPrevSignal = = -2 )
return 0.0 ;
double sign = ( dPrevSignal > 0.0 ) ? 1.0 : ( dPrevSignal < 0.0 ) ? -1.0 : 0.0 ;
2026-07-27 15:52:39 -04:00
if ( sign = = 0.0 )
return 0.0 ;
2026-07-17 23:21:12 -04:00
return sign * CalibratedConfidenceMagnitude ( ) ;
}
2026-07-14 22:36:27 -04:00
//--- event handlers, common to every AI signal
virtual void OnTickHandler ( void ) ;
//--- drives the same training-scheduling check as OnTickHandler(), but callable from a timer so
//--- it isn't dependent on ticks (which don't arrive while the market is closed)
void PollTraining ( void ) ;
virtual void OnChartEventHandler ( const int id ,
const long & lparam ,
const double & dparam ,
const string & sparam ) ;
2026-07-23 08:21:41 -04:00
//--- methods of adjusting "weights" of the 4 confidence-tier market models - see m_pattern_0's
//--- declaration comment
2026-07-14 22:36:27 -04:00
void Pattern_0 ( int value ) { m_pattern_0 = value ; }
2026-07-23 08:21:41 -04:00
void Pattern_1 ( int value ) { m_pattern_1 = value ; }
void Pattern_2 ( int value ) { m_pattern_2 = value ; }
void Pattern_3 ( int value ) { m_pattern_3 = value ; }
2026-07-14 22:36:27 -04:00
virtual void ApplyPatternWeight ( int patternNumber , int weight ) ;
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
//--- Re-derives the four tier weights (and the module weight) from THIS era's held-out outcomes.
//--- Called once per era at the end of pass 3, when m_oosTierFired/Hits are complete.
void RankTiersFromOos ( void ) ;
refactor(baselines): the first real module - a class, not an #included partial
Baselines was 951 lines of CExpertSignalAIBase method bodies in a file
that only looked like a module. It is now CBaselineComparator: a class
the signal OWNS, which reads a CTrainingDataView and prints. It does
not name the signal anywhere in its code.
What the seam forced out into the open:
- Thirty-odd ArraySize() bounds tests, each carried by its caller, are
now one test per accessor next to the data. The two `hasValueN` and
one `arrowN` locals are gone with them.
- The -2.0 "never scored" sentinel on the arrow cache was tested at the
call site. It is now inside DataDirectionalCall, where it cannot be
read as a small confidence.
- DoubleToSignal needs m_outputNeuronsCount, so a raw double could not
be turned into a side by any reader. The view answers
DirectionalCall(bar, isBuy, magnitude) instead - the conversion
happens where the head width lives, and the module no longer needs
ENUM_SIGNAL at all.
- m_baselineDone was a latch on the signal for a decision only this
module makes. It is m_done, private, where it belongs.
Correction to my own earlier claim: I said Baselines had nine exclusive
members "polluting the signal class". It had none. m_x, m_f, m_ngrad,
m_AvgCE and the rest are FIELDS OF ALGLIB REPORT OBJECTS (state.m_x,
mrep.m_AvgCE) that my `\bm_\w+` scan matched after the dot. The module
needs no private state but its view pointer and that latch - which is
why it came out this cleanly.
The include sits below the g_ens* vote globals and the Alglib headers
it reads, because unlike the AIBase\*.mqh partials this is a real class
declaration compiled where it stands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 12:00:18 -04:00
//--- ONE FEATURE WINDOW as a plain double[] - the shape both Alglib predictors take, and the
//--- one thing the baseline module cannot get from a cache because building it is a live call.
//--- Reached through CTrainingDataView::RowFeatures, never named by the module itself.
feat(baselines): Alglib forest + linear on the NN's own matrix
Every direction verdict so far was measured through one architecture
family, so "flat" has two readings that no topology tuning can separate:
the net is the wrong learner, or the matrix carries no directional
information.
Two learners with completely different inductive biases - Alglib's
random decision forest and an ordinary least-squares fit - now train on
the SAME feature windows (BuildFeatureWindow, the net's own function, so
there is no second feature implementation to drift), the SAME labels,
the SAME IS/OOS split with both purges, and are scored through the SAME
precision-against-always-one-direction comparison and the same Sidak
family-wise arithmetic the deploy gate uses. If both also land at
chance, the matrix is the limit.
Deliberate choices, each of which could have made the comparison a
different question wearing this one's name:
- LRBuild, not LRBuildZ: the intercept absorbs the class imbalance, and
a baseline handicapped by a forced zero intercept would flatter the
net for the wrong reason.
- Raw call counts in the SE, matching the live gate's known-permissive
test rather than correcting it here - both sides must face the same
bar.
- No threshold sweep on the linear fit: a threshold fitted on the slice
being scored is the calibration leak the purged band exists to avoid.
- Uniform stride when a cap bites, not the newest N rows, so a score
difference cannot be a regime difference. What was dropped is logged.
Ships off (Run_Alglib_Baselines = false): it is a measurement, not a
trading feature, nothing trades on the answer and no model is saved.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 23:45:03 -04:00
bool BaselineRowFeatures ( const int bar , const int width , double & x [ ] ) ;
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
//--- Direct tier setter, deliberately NOT routed through ApplyPatternWeight(): that override
//--- declines writes once self-ranking is live, which is exactly what must not happen to the
//--- self-ranker's own writes. Two doors, because they serve opposite purposes.
void ApplyTierWeight ( const int tier , const int weight )
{
switch ( tier )
{
case 0 : Pattern_0 ( weight ) ; break ;
case 1 : Pattern_1 ( weight ) ; break ;
case 2 : Pattern_2 ( weight ) ; break ;
default : Pattern_3 ( weight ) ; break ;
}
}
//--- True once this model has measured its own tier win rates on held-out bars. While true the
//--- signal DB's ranking is declined for this filter - see ApplyPatternWeight's comment and
//--- CExpertSignalCustom::SelfRanked().
virtual bool SelfRanked ( void ) const override { return m_tiersSelfRanked ; }
feat(vote): edge-over-chance currency, no-skill exclusion, checkpoint burn-in
RETRAIN-FORCING and deliberately so. Two independent fixes for the same symptom -
charts that go quiet while others overtrade.
1. THE VOTE CURRENCY IS NOW EDGE OVER CHANCE, not an absolute win rate.
A tier weight is a raw win rate and a raw win rate means nothing without the
chance rate behind it: 30% is strong under a 14% base rate and catastrophic under
50%, yet both entered the mean as "30". That is why the threshold needed
re-tuning every time the label changed - 25 was permissive at ~70% win rates
under the old direction label and a near-unanimity rule at ~30% under the
pivot-event one - and why one chart's 25% was never the same statement as
another's. Subtracting the member's own chance rate makes the units percentage
points of demonstrated edge, comparable across charts, labels and regimes.
Clamped at zero: a below-chance tier is anti-informative, and contributing
negatively would act on a broken model as an inverted oracle rather than
discarding it.
2. A NO-SKILL MEMBER IS NOW ABSENT, NOT ABSTAINING.
Measured on XTIUSD: a Perceptron collapsed to B97/S6/N3, pooled win rate 11.5%
against a 14% chance rate - worse than guessing - and still voting. Three healthy
members voting Sell scored -21.06/0.77 = -27.4 and cleared; with the dead one
voting Buy it became (-21.06+1.44)/0.89 = -22.0 and was BLOCKED. It vetoed its
own ensemble on ~95% of bars, and that WAS the chart's 3.3% coverage. Neither
existing guard caught it: it IS self-ranked and its tier weights were 11-14.
The fix has to remove it from the DIVISOR, not just the sum - an abstainer
contributes weight by design, so zeroing only the contribution makes the dilution
worse. VoteCapableWeight() already means exactly "may this member's weight sit in
the denominator", so the skill test belongs there. ReconstructionWeight() and the
OOS scorer's divisor move with it or the scorer certifies a vote live does not
cast. The skill test reads the PREVIOUS era's measurement - gating this era's
vote on this era's own outcome would be circular.
3. CHECKPOINT BURN-IN (ENSEMBLE_CHECKPOINT_MIN_ERA 20).
XAUUSD deployed the checkpoint from ERA 2, XTIUSD from ERA 4, each after 69 and
65 further eras failed to beat it. Ensemble coverage measures AGREEMENT, and four
models that have barely moved off their initialisation agree almost by
construction - so coverage is inflated exactly when the models know least and
decays as they differentiate (XAUUSD 6.6% at era 8 -> 0.4% at era 75). Since
selectionScore is precision discounted by coverage, an early era outscores every
mature one and the ladder freezes on it.
INTENDED CONSEQUENCE: a chart whose MATURE coverage cannot clear the floor now
refuses to deploy rather than shipping era-2 weights. Fewer deploys, honest ones.
Burn-in eras are also kept out of g_ensCandidateEras (they could not have won, so
counting them inflates the family-wise N and raises the bar for nothing) and out
of g_ensErasSinceBest (or the run reaches "no better vote for N eras" with no
best to beat, exhausting the escalation ladder before the first era may compete).
Every pinned threshold and .stats record is in the OLD currency and is now
meaningless - this forces a fresh start on its own. Nothing needs re-tuning
because the threshold is DERIVED: the sweep re-picks the rung by itself.
Compiled clean; NOT yet run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 15:38:26 -04:00
//--- HAS THIS MEMBER DEMONSTRATED ANY SKILL AT ALL? Its own pooled holdout precision against its
//--- own chance rate - the same test the deploy gate applies to the ensemble, simply never applied
//--- to voting eligibility until now.
//---
//--- MEASURED, XTIUSD 2026-08-26: a Perceptron collapsed to B97/S6/N3, pooled win rate 11.5%
//--- against a 14% chance rate - worse than guessing - and it was still voting. Three healthy
//--- members voting Sell scored -21.06/0.77 = -27.4 and cleared the threshold; with the dead one
//--- voting Buy it became (-21.06+1.44)/0.89 = -22.0 and was BLOCKED. It vetoed its own ensemble on
//--- the ~95% of bars where it fired the wrong way, and that WAS the chart's 3.3% coverage.
//---
fix(vote): a resumed converged model passed the eligibility test and then abstained on every bar
Found by restarting the terminal against three charts that had just deployed -
the exact scenario d9092a2 was written for, run deliberately rather than
assumed. It failed, and the failure was mine.
SP500 swept 4999 bar(s), 4986 had a snapshot, 0 had a voter, drew 0
arrow(s). Strongest vote 0.0% against a 10.0% threshold.
The snapshot count proves d9092a2 worked: the certified edge was restored
(26.20% precision vs 13.70% chance, verified in the .stats bytes),
HasDemonstratedEdge returned true, ReconstructionWeight was non-zero and the
divisor was healthy on 4986 of 4999 bars.
But TWO readers need the member's chance rate, and I taught only one to fall
back. LiveVoteContribution still read m_eraStatChancePct DIRECTLY - era-only
state, -1 on a converged model that runs no eras - so it bailed out at
"no reference rate yet" and returned 0 for every call. The member was admitted
to the divisor and then contributed nothing to the sum: eligible, and silent.
Exactly the failure mode in feedback_rename_leaves_readers_behind, committed by
the person who wrote that note down.
Both quantities now come from one accessor each - MeasuredPrecPct() and
MeasuredChancePct() - so a third reader cannot repeat it.
ALSO: the "restored the certified edge" line was PrintVerbose. It marks a STATE
RESTORE, which by this codebase's own rule never sits behind the verbose gate,
and its absence from the log was briefly read as evidence the restore had not
happened. Promoted to Print.
Build tag -> voterestore-v1. Not a layout change: no retrain.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 00:52:59 -04:00
//--- THE MEMBER'S MEASURED PAIR, WITH ITS FALLBACK, IN ONE PLACE - AND THAT IS THE POINT.
//---
//--- Both the eligibility test (HasDemonstratedEdge) and the vote itself (LiveVoteContribution)
//--- need this member's chance rate, and both used to read m_eraStat* DIRECTLY. Those are
//--- ERA-ONLY state: a converged model runs no eras, so after a restart they are -1 and every
//--- reader silently degrades. Persisting the certified pair and teaching only ONE of the two
//--- readers to fall back to it produced a chart that passed the eligibility test - 4986 of 4999
//--- bars carried a snapshot, the divisor was healthy - and then voted 0.0% on every one of them,
//--- because the vote was still subtracting a -1 chance rate and bailing out. Measured on the
//--- 2026-08-27 00:41 restart of three freshly deployed charts.
//---
//--- One accessor per quantity, so a third reader cannot repeat the mistake.
double MeasuredPrecPct ( void ) const
{ return ( m_eraStatPrecPct > = 0.0 ) ? m_eraStatPrecPct : m_certifiedPrecPct ; }
double MeasuredChancePct ( void ) const
{ return ( m_eraStatChancePct > = 0.0 ) ? m_eraStatChancePct : m_certifiedChancePct ; }
feat(vote): edge-over-chance currency, no-skill exclusion, checkpoint burn-in
RETRAIN-FORCING and deliberately so. Two independent fixes for the same symptom -
charts that go quiet while others overtrade.
1. THE VOTE CURRENCY IS NOW EDGE OVER CHANCE, not an absolute win rate.
A tier weight is a raw win rate and a raw win rate means nothing without the
chance rate behind it: 30% is strong under a 14% base rate and catastrophic under
50%, yet both entered the mean as "30". That is why the threshold needed
re-tuning every time the label changed - 25 was permissive at ~70% win rates
under the old direction label and a near-unanimity rule at ~30% under the
pivot-event one - and why one chart's 25% was never the same statement as
another's. Subtracting the member's own chance rate makes the units percentage
points of demonstrated edge, comparable across charts, labels and regimes.
Clamped at zero: a below-chance tier is anti-informative, and contributing
negatively would act on a broken model as an inverted oracle rather than
discarding it.
2. A NO-SKILL MEMBER IS NOW ABSENT, NOT ABSTAINING.
Measured on XTIUSD: a Perceptron collapsed to B97/S6/N3, pooled win rate 11.5%
against a 14% chance rate - worse than guessing - and still voting. Three healthy
members voting Sell scored -21.06/0.77 = -27.4 and cleared; with the dead one
voting Buy it became (-21.06+1.44)/0.89 = -22.0 and was BLOCKED. It vetoed its
own ensemble on ~95% of bars, and that WAS the chart's 3.3% coverage. Neither
existing guard caught it: it IS self-ranked and its tier weights were 11-14.
The fix has to remove it from the DIVISOR, not just the sum - an abstainer
contributes weight by design, so zeroing only the contribution makes the dilution
worse. VoteCapableWeight() already means exactly "may this member's weight sit in
the denominator", so the skill test belongs there. ReconstructionWeight() and the
OOS scorer's divisor move with it or the scorer certifies a vote live does not
cast. The skill test reads the PREVIOUS era's measurement - gating this era's
vote on this era's own outcome would be circular.
3. CHECKPOINT BURN-IN (ENSEMBLE_CHECKPOINT_MIN_ERA 20).
XAUUSD deployed the checkpoint from ERA 2, XTIUSD from ERA 4, each after 69 and
65 further eras failed to beat it. Ensemble coverage measures AGREEMENT, and four
models that have barely moved off their initialisation agree almost by
construction - so coverage is inflated exactly when the models know least and
decays as they differentiate (XAUUSD 6.6% at era 8 -> 0.4% at era 75). Since
selectionScore is precision discounted by coverage, an early era outscores every
mature one and the ladder freezes on it.
INTENDED CONSEQUENCE: a chart whose MATURE coverage cannot clear the floor now
refuses to deploy rather than shipping era-2 weights. Fewer deploys, honest ones.
Burn-in eras are also kept out of g_ensCandidateEras (they could not have won, so
counting them inflates the family-wise N and raises the bar for nothing) and out
of g_ensErasSinceBest (or the run reaches "no better vote for N eras" with no
best to beat, exhausting the escalation ladder before the first era may compete).
Every pinned threshold and .stats record is in the OLD currency and is now
meaningless - this forces a fresh start on its own. Nothing needs re-tuning
because the threshold is DERIVED: the sweep re-picks the rung by itself.
Compiled clean; NOT yet run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 15:38:26 -04:00
//--- Neither existing guard caught it: it IS self-ranked, and its tier weights were 11-14, not 0.
bool HasDemonstratedEdge ( void ) const
{
if ( ! m_tiersSelfRanked )
return false ;
//--- No reference rate yet means "cannot judge", which must read as not-yet-eligible rather
//--- than as skill. A member with no measurement has demonstrated nothing.
fix(vote): a resumed converged model passed the eligibility test and then abstained on every bar
Found by restarting the terminal against three charts that had just deployed -
the exact scenario d9092a2 was written for, run deliberately rather than
assumed. It failed, and the failure was mine.
SP500 swept 4999 bar(s), 4986 had a snapshot, 0 had a voter, drew 0
arrow(s). Strongest vote 0.0% against a 10.0% threshold.
The snapshot count proves d9092a2 worked: the certified edge was restored
(26.20% precision vs 13.70% chance, verified in the .stats bytes),
HasDemonstratedEdge returned true, ReconstructionWeight was non-zero and the
divisor was healthy on 4986 of 4999 bars.
But TWO readers need the member's chance rate, and I taught only one to fall
back. LiveVoteContribution still read m_eraStatChancePct DIRECTLY - era-only
state, -1 on a converged model that runs no eras - so it bailed out at
"no reference rate yet" and returned 0 for every call. The member was admitted
to the divisor and then contributed nothing to the sum: eligible, and silent.
Exactly the failure mode in feedback_rename_leaves_readers_behind, committed by
the person who wrote that note down.
Both quantities now come from one accessor each - MeasuredPrecPct() and
MeasuredChancePct() - so a third reader cannot repeat it.
ALSO: the "restored the certified edge" line was PrintVerbose. It marks a STATE
RESTORE, which by this codebase's own rule never sits behind the verbose gate,
and its absence from the log was briefly read as evidence the restore had not
happened. Promoted to Print.
Build tag -> voterestore-v1. Not a layout change: no retrain.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 00:52:59 -04:00
double precPct = MeasuredPrecPct ( ) ;
double chancePct = MeasuredChancePct ( ) ;
2026-08-26 16:53:16 -04:00
if ( chancePct < 0.0 | | precPct < 0.0 )
feat(vote): edge-over-chance currency, no-skill exclusion, checkpoint burn-in
RETRAIN-FORCING and deliberately so. Two independent fixes for the same symptom -
charts that go quiet while others overtrade.
1. THE VOTE CURRENCY IS NOW EDGE OVER CHANCE, not an absolute win rate.
A tier weight is a raw win rate and a raw win rate means nothing without the
chance rate behind it: 30% is strong under a 14% base rate and catastrophic under
50%, yet both entered the mean as "30". That is why the threshold needed
re-tuning every time the label changed - 25 was permissive at ~70% win rates
under the old direction label and a near-unanimity rule at ~30% under the
pivot-event one - and why one chart's 25% was never the same statement as
another's. Subtracting the member's own chance rate makes the units percentage
points of demonstrated edge, comparable across charts, labels and regimes.
Clamped at zero: a below-chance tier is anti-informative, and contributing
negatively would act on a broken model as an inverted oracle rather than
discarding it.
2. A NO-SKILL MEMBER IS NOW ABSENT, NOT ABSTAINING.
Measured on XTIUSD: a Perceptron collapsed to B97/S6/N3, pooled win rate 11.5%
against a 14% chance rate - worse than guessing - and still voting. Three healthy
members voting Sell scored -21.06/0.77 = -27.4 and cleared; with the dead one
voting Buy it became (-21.06+1.44)/0.89 = -22.0 and was BLOCKED. It vetoed its
own ensemble on ~95% of bars, and that WAS the chart's 3.3% coverage. Neither
existing guard caught it: it IS self-ranked and its tier weights were 11-14.
The fix has to remove it from the DIVISOR, not just the sum - an abstainer
contributes weight by design, so zeroing only the contribution makes the dilution
worse. VoteCapableWeight() already means exactly "may this member's weight sit in
the denominator", so the skill test belongs there. ReconstructionWeight() and the
OOS scorer's divisor move with it or the scorer certifies a vote live does not
cast. The skill test reads the PREVIOUS era's measurement - gating this era's
vote on this era's own outcome would be circular.
3. CHECKPOINT BURN-IN (ENSEMBLE_CHECKPOINT_MIN_ERA 20).
XAUUSD deployed the checkpoint from ERA 2, XTIUSD from ERA 4, each after 69 and
65 further eras failed to beat it. Ensemble coverage measures AGREEMENT, and four
models that have barely moved off their initialisation agree almost by
construction - so coverage is inflated exactly when the models know least and
decays as they differentiate (XAUUSD 6.6% at era 8 -> 0.4% at era 75). Since
selectionScore is precision discounted by coverage, an early era outscores every
mature one and the ladder freezes on it.
INTENDED CONSEQUENCE: a chart whose MATURE coverage cannot clear the floor now
refuses to deploy rather than shipping era-2 weights. Fewer deploys, honest ones.
Burn-in eras are also kept out of g_ensCandidateEras (they could not have won, so
counting them inflates the family-wise N and raises the bar for nothing) and out
of g_ensErasSinceBest (or the run reaches "no better vote for N eras" with no
best to beat, exhausting the escalation ladder before the first era may compete).
Every pinned threshold and .stats record is in the OLD currency and is now
meaningless - this forces a fresh start on its own. Nothing needs re-tuning
because the threshold is DERIVED: the sweep re-picks the rung by itself.
Compiled clean; NOT yet run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 15:38:26 -04:00
return false ;
2026-08-26 16:53:16 -04:00
return ( precPct > chancePct ) ;
feat(vote): edge-over-chance currency, no-skill exclusion, checkpoint burn-in
RETRAIN-FORCING and deliberately so. Two independent fixes for the same symptom -
charts that go quiet while others overtrade.
1. THE VOTE CURRENCY IS NOW EDGE OVER CHANCE, not an absolute win rate.
A tier weight is a raw win rate and a raw win rate means nothing without the
chance rate behind it: 30% is strong under a 14% base rate and catastrophic under
50%, yet both entered the mean as "30". That is why the threshold needed
re-tuning every time the label changed - 25 was permissive at ~70% win rates
under the old direction label and a near-unanimity rule at ~30% under the
pivot-event one - and why one chart's 25% was never the same statement as
another's. Subtracting the member's own chance rate makes the units percentage
points of demonstrated edge, comparable across charts, labels and regimes.
Clamped at zero: a below-chance tier is anti-informative, and contributing
negatively would act on a broken model as an inverted oracle rather than
discarding it.
2. A NO-SKILL MEMBER IS NOW ABSENT, NOT ABSTAINING.
Measured on XTIUSD: a Perceptron collapsed to B97/S6/N3, pooled win rate 11.5%
against a 14% chance rate - worse than guessing - and still voting. Three healthy
members voting Sell scored -21.06/0.77 = -27.4 and cleared; with the dead one
voting Buy it became (-21.06+1.44)/0.89 = -22.0 and was BLOCKED. It vetoed its
own ensemble on ~95% of bars, and that WAS the chart's 3.3% coverage. Neither
existing guard caught it: it IS self-ranked and its tier weights were 11-14.
The fix has to remove it from the DIVISOR, not just the sum - an abstainer
contributes weight by design, so zeroing only the contribution makes the dilution
worse. VoteCapableWeight() already means exactly "may this member's weight sit in
the denominator", so the skill test belongs there. ReconstructionWeight() and the
OOS scorer's divisor move with it or the scorer certifies a vote live does not
cast. The skill test reads the PREVIOUS era's measurement - gating this era's
vote on this era's own outcome would be circular.
3. CHECKPOINT BURN-IN (ENSEMBLE_CHECKPOINT_MIN_ERA 20).
XAUUSD deployed the checkpoint from ERA 2, XTIUSD from ERA 4, each after 69 and
65 further eras failed to beat it. Ensemble coverage measures AGREEMENT, and four
models that have barely moved off their initialisation agree almost by
construction - so coverage is inflated exactly when the models know least and
decays as they differentiate (XAUUSD 6.6% at era 8 -> 0.4% at era 75). Since
selectionScore is precision discounted by coverage, an early era outscores every
mature one and the ladder freezes on it.
INTENDED CONSEQUENCE: a chart whose MATURE coverage cannot clear the floor now
refuses to deploy rather than shipping era-2 weights. Fewer deploys, honest ones.
Burn-in eras are also kept out of g_ensCandidateEras (they could not have won, so
counting them inflates the family-wise N and raises the bar for nothing) and out
of g_ensErasSinceBest (or the run reaches "no better vote for N eras" with no
best to beat, exhausting the escalation ladder before the first era may compete).
Every pinned threshold and .stats record is in the OLD currency and is now
meaningless - this forces a fresh start on its own. Nothing needs re-tuning
because the threshold is DERIVED: the sweep re-picks the rung by itself.
Compiled clean; NOT yet run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 15:38:26 -04:00
}
feat(vote): CONSENSUS arithmetic - agreement is now what the threshold dials
Era-680 report, all three observations one equation: "peak 29, no arrows at
threshold 30" / "at 20, arrows on EVERY bar" / "label at 12 while arrows
everywhere". Under the voters-only divisor, any bar with at least one
directional voter read the weighted mean of the firing tiers' weights - and
once the tiers self-ranked to each model's pooled win rate (~28-31), that
mean was NEAR-CONSTANT regardless of headcount. One member alone: ~29. Four
unanimous: ~29. Min_Vote_Open was a step function around that constant -
above it nothing ever fired, below it everything did - and the label's 12
was a 3v1 split netting through the same divisor. Not three display bugs:
one arithmetic that could not express agreement.
The divisor is now the CAPABLE weight - every filter that could vote,
whether it did or not:
* live (Direction): VoteCapableWeight() - classic pattern ladders always,
veto filters never, AI members once past the same readiness test
LongCondition gates on. A model still training must not dilute an
ensemble it cannot join: four trainees + one deployed model is a solo
chart wearing an ensemble label, and the solo vote reads full strength.
* gate (EnsembleEraVerdict): g_ensVoteWeightSum accumulates for every
member that EVALUATED the bar, Neutral included.
* overlay sweep + prospective readout: weight counts whenever the member
has data; a snapshotted Neutral dilutes.
One arithmetic, four sites, same numbers everywhere.
What the numbers become (four members, w~0.29, tiers~29): unanimous ~29 -
the CEILING, which is the pooled win rate and is what the peak displays;
3-of-4 ~22; 2-of-4 ~14.5; 3v1 ~14.5. Min_Vote_Open 20 now means "roughly
three-quarters of the ensemble's trust agrees, net". It MUST sit below the
ceiling to ever fire - the census/peak states the ceiling.
This is the ensemble the user specified in the original design discussion
("if the perceptron also votes, both together reach the threshold; if
another NN votes the other side, the threshold is not reached") - union
semantics was the pre-ensemble behaviour, kept until measurement showed its
vote magnitude was a constant.
Plus overlay DECLUSTERING, the other half of "arrows on every bar": the
same three NMS rules as the per-member arrows (same-direction runs collapse
to their first bar, cross-direction flicker keeps the stronger side), online
over the sweep's strictly oldest->newest walk. Suppression is a verdict and
deletes a standing arrow; the den==0 no-data skip still never does.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 06:58:03 -04:00
//--- An AI member's say in the consensus denominator: its module weight once it is ALLOWED to
2026-08-22 00:24:45 -04:00
//--- vote (the same readiness test LongCondition gates on), zero before that.
feat(vote): CONSENSUS arithmetic - agreement is now what the threshold dials
Era-680 report, all three observations one equation: "peak 29, no arrows at
threshold 30" / "at 20, arrows on EVERY bar" / "label at 12 while arrows
everywhere". Under the voters-only divisor, any bar with at least one
directional voter read the weighted mean of the firing tiers' weights - and
once the tiers self-ranked to each model's pooled win rate (~28-31), that
mean was NEAR-CONSTANT regardless of headcount. One member alone: ~29. Four
unanimous: ~29. Min_Vote_Open was a step function around that constant -
above it nothing ever fired, below it everything did - and the label's 12
was a 3v1 split netting through the same divisor. Not three display bugs:
one arithmetic that could not express agreement.
The divisor is now the CAPABLE weight - every filter that could vote,
whether it did or not:
* live (Direction): VoteCapableWeight() - classic pattern ladders always,
veto filters never, AI members once past the same readiness test
LongCondition gates on. A model still training must not dilute an
ensemble it cannot join: four trainees + one deployed model is a solo
chart wearing an ensemble label, and the solo vote reads full strength.
* gate (EnsembleEraVerdict): g_ensVoteWeightSum accumulates for every
member that EVALUATED the bar, Neutral included.
* overlay sweep + prospective readout: weight counts whenever the member
has data; a snapshotted Neutral dilutes.
One arithmetic, four sites, same numbers everywhere.
What the numbers become (four members, w~0.29, tiers~29): unanimous ~29 -
the CEILING, which is the pooled win rate and is what the peak displays;
3-of-4 ~22; 2-of-4 ~14.5; 3v1 ~14.5. Min_Vote_Open 20 now means "roughly
three-quarters of the ensemble's trust agrees, net". It MUST sit below the
ceiling to ever fire - the census/peak states the ceiling.
This is the ensemble the user specified in the original design discussion
("if the perceptron also votes, both together reach the threshold; if
another NN votes the other side, the threshold is not reached") - union
semantics was the pre-ensemble behaviour, kept until measurement showed its
vote magnitude was a constant.
Plus overlay DECLUSTERING, the other half of "arrows on every bar": the
same three NMS rules as the per-member arrows (same-direction runs collapse
to their first bar, cross-direction flicker keeps the stronger side), online
over the sweep's strictly oldest->newest walk. Suppression is a verdict and
deletes a standing arrow; the den==0 no-data skip still never does.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 06:58:03 -04:00
virtual double VoteCapableWeight ( void ) override
{
refactor(signal): only true signalers are filters - META becomes a gate
Operator's call: "META should be removed or implemented directly into
CExpertSignalBase. Only true signalers needs to be filters."
A meta head never votes - its Long/ShortCondition are structurally 0 and
its verdict reaches the pipeline through LiveMetaGate(), not through the
vote. Keeping it in m_filters meant every consumer of that list needed a
special case, and each one was a bug waiting: VoteCapableWeight() had to
return 0 for it or it would park a permanent abstainer in the consensus
divisor. The replay's divisor bug (d81ec15) had exactly this shape.
CExpertSignalCustom::IsVotingSignal() is the predicate, false for a meta
target. AddFilter() ROUTES on it into a second owned list, m_gates, so
the EA's init code stays one uniform AddFilterToSignal() call per signal
and the invariant is enforced in one place instead of re-checked by
every reader.
THE TRAP, and it is why this is not just a deletion: m_filters is not
only the voting list, it is also how a signal reaches its children for
INDICATORS, TICKS, PANEL COMMANDS, CHART EVENTS and TRAIT COUNTS.
OnTickHandler in particular is what drives each AI signal's training - a
gate dropped from it silently stops learning. So the tree is now split
by purpose:
m_filters (voting) Direction, HistoricalNetVote,
RefreshVoteReadout, vote rollback,
UpdateSignalsWeights (pattern/DB weights)
ChildSignalAt (whole tree) InitIndicators, OnTickHandler,
OnChartEventHandler, DispatchSignalCommand,
CountSignalTrait
and the IsMetaTarget() special case in VoteCapableWeight() is deleted -
the structure now guarantees what it was hand-checking.
META was already added last, so no filter's m_ignore/m_invert bit index
moves.
Not done here: removing META outright. It is default-off and has never
shown an operating point clearing break-even, so the case for deleting
it is real - but that is a feature decision, not a refactor, and it is
offered separately.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 10:03:48 -04:00
//--- No meta test here any more: a gate is not in the voting list at all (IsVotingSignal),
//--- so this is never asked of one. That is the point of the split - the special case was
//--- load-bearing precisely because a non-voter was in a voters' collection.
feat(vote): CONSENSUS arithmetic - agreement is now what the threshold dials
Era-680 report, all three observations one equation: "peak 29, no arrows at
threshold 30" / "at 20, arrows on EVERY bar" / "label at 12 while arrows
everywhere". Under the voters-only divisor, any bar with at least one
directional voter read the weighted mean of the firing tiers' weights - and
once the tiers self-ranked to each model's pooled win rate (~28-31), that
mean was NEAR-CONSTANT regardless of headcount. One member alone: ~29. Four
unanimous: ~29. Min_Vote_Open was a step function around that constant -
above it nothing ever fired, below it everything did - and the label's 12
was a 3v1 split netting through the same divisor. Not three display bugs:
one arithmetic that could not express agreement.
The divisor is now the CAPABLE weight - every filter that could vote,
whether it did or not:
* live (Direction): VoteCapableWeight() - classic pattern ladders always,
veto filters never, AI members once past the same readiness test
LongCondition gates on. A model still training must not dilute an
ensemble it cannot join: four trainees + one deployed model is a solo
chart wearing an ensemble label, and the solo vote reads full strength.
* gate (EnsembleEraVerdict): g_ensVoteWeightSum accumulates for every
member that EVALUATED the bar, Neutral included.
* overlay sweep + prospective readout: weight counts whenever the member
has data; a snapshotted Neutral dilutes.
One arithmetic, four sites, same numbers everywhere.
What the numbers become (four members, w~0.29, tiers~29): unanimous ~29 -
the CEILING, which is the pooled win rate and is what the peak displays;
3-of-4 ~22; 2-of-4 ~14.5; 3v1 ~14.5. Min_Vote_Open 20 now means "roughly
three-quarters of the ensemble's trust agrees, net". It MUST sit below the
ceiling to ever fire - the census/peak states the ceiling.
This is the ensemble the user specified in the original design discussion
("if the perceptron also votes, both together reach the threshold; if
another NN votes the other side, the threshold is not reached") - union
semantics was the pre-ensemble behaviour, kept until measurement showed its
vote magnitude was a constant.
Plus overlay DECLUSTERING, the other half of "arrows on every bar": the
same three NMS rules as the per-member arrows (same-direction runs collapse
to their first bar, cross-direction flicker keeps the stronger side), online
over the sweep's strictly oldest->newest walk. Suppression is a verdict and
deletes a standing arrow; the den==0 no-data skip still never does.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 06:58:03 -04:00
if ( ! m_trainingComplete & & ! ( m_inferenceOnly & & m_modelLoadedFromDisk ) )
return 0.0 ;
feat(vote): edge-over-chance currency, no-skill exclusion, checkpoint burn-in
RETRAIN-FORCING and deliberately so. Two independent fixes for the same symptom -
charts that go quiet while others overtrade.
1. THE VOTE CURRENCY IS NOW EDGE OVER CHANCE, not an absolute win rate.
A tier weight is a raw win rate and a raw win rate means nothing without the
chance rate behind it: 30% is strong under a 14% base rate and catastrophic under
50%, yet both entered the mean as "30". That is why the threshold needed
re-tuning every time the label changed - 25 was permissive at ~70% win rates
under the old direction label and a near-unanimity rule at ~30% under the
pivot-event one - and why one chart's 25% was never the same statement as
another's. Subtracting the member's own chance rate makes the units percentage
points of demonstrated edge, comparable across charts, labels and regimes.
Clamped at zero: a below-chance tier is anti-informative, and contributing
negatively would act on a broken model as an inverted oracle rather than
discarding it.
2. A NO-SKILL MEMBER IS NOW ABSENT, NOT ABSTAINING.
Measured on XTIUSD: a Perceptron collapsed to B97/S6/N3, pooled win rate 11.5%
against a 14% chance rate - worse than guessing - and still voting. Three healthy
members voting Sell scored -21.06/0.77 = -27.4 and cleared; with the dead one
voting Buy it became (-21.06+1.44)/0.89 = -22.0 and was BLOCKED. It vetoed its
own ensemble on ~95% of bars, and that WAS the chart's 3.3% coverage. Neither
existing guard caught it: it IS self-ranked and its tier weights were 11-14.
The fix has to remove it from the DIVISOR, not just the sum - an abstainer
contributes weight by design, so zeroing only the contribution makes the dilution
worse. VoteCapableWeight() already means exactly "may this member's weight sit in
the denominator", so the skill test belongs there. ReconstructionWeight() and the
OOS scorer's divisor move with it or the scorer certifies a vote live does not
cast. The skill test reads the PREVIOUS era's measurement - gating this era's
vote on this era's own outcome would be circular.
3. CHECKPOINT BURN-IN (ENSEMBLE_CHECKPOINT_MIN_ERA 20).
XAUUSD deployed the checkpoint from ERA 2, XTIUSD from ERA 4, each after 69 and
65 further eras failed to beat it. Ensemble coverage measures AGREEMENT, and four
models that have barely moved off their initialisation agree almost by
construction - so coverage is inflated exactly when the models know least and
decays as they differentiate (XAUUSD 6.6% at era 8 -> 0.4% at era 75). Since
selectionScore is precision discounted by coverage, an early era outscores every
mature one and the ladder freezes on it.
INTENDED CONSEQUENCE: a chart whose MATURE coverage cannot clear the floor now
refuses to deploy rather than shipping era-2 weights. Fewer deploys, honest ones.
Burn-in eras are also kept out of g_ensCandidateEras (they could not have won, so
counting them inflates the family-wise N and raises the bar for nothing) and out
of g_ensErasSinceBest (or the run reaches "no better vote for N eras" with no
best to beat, exhausting the escalation ladder before the first era may compete).
Every pinned threshold and .stats record is in the OLD currency and is now
meaningless - this forces a fresh start on its own. Nothing needs re-tuning
because the threshold is DERIVED: the sweep re-picks the rung by itself.
Compiled clean; NOT yet run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 15:38:26 -04:00
//--- NO DEMONSTRATED SKILL => NOT IN THE DIVISOR, i.e. ABSENT rather than abstaining. This is
//--- the distinction that makes the fix work: an abstainer still contributes its weight to the
//--- divisor BY DESIGN (it looked and said nothing, and diluting the consensus is what that
//--- should do), so zeroing only a no-skill member's CONTRIBUTION would make the dilution
//--- WORSE, not better. It has to leave the denominator too, which is exactly what this
//--- function already means for "a member that could not look at all".
if ( ! HasDemonstratedEdge ( ) )
return 0.0 ;
feat(vote): CONSENSUS arithmetic - agreement is now what the threshold dials
Era-680 report, all three observations one equation: "peak 29, no arrows at
threshold 30" / "at 20, arrows on EVERY bar" / "label at 12 while arrows
everywhere". Under the voters-only divisor, any bar with at least one
directional voter read the weighted mean of the firing tiers' weights - and
once the tiers self-ranked to each model's pooled win rate (~28-31), that
mean was NEAR-CONSTANT regardless of headcount. One member alone: ~29. Four
unanimous: ~29. Min_Vote_Open was a step function around that constant -
above it nothing ever fired, below it everything did - and the label's 12
was a 3v1 split netting through the same divisor. Not three display bugs:
one arithmetic that could not express agreement.
The divisor is now the CAPABLE weight - every filter that could vote,
whether it did or not:
* live (Direction): VoteCapableWeight() - classic pattern ladders always,
veto filters never, AI members once past the same readiness test
LongCondition gates on. A model still training must not dilute an
ensemble it cannot join: four trainees + one deployed model is a solo
chart wearing an ensemble label, and the solo vote reads full strength.
* gate (EnsembleEraVerdict): g_ensVoteWeightSum accumulates for every
member that EVALUATED the bar, Neutral included.
* overlay sweep + prospective readout: weight counts whenever the member
has data; a snapshotted Neutral dilutes.
One arithmetic, four sites, same numbers everywhere.
What the numbers become (four members, w~0.29, tiers~29): unanimous ~29 -
the CEILING, which is the pooled win rate and is what the peak displays;
3-of-4 ~22; 2-of-4 ~14.5; 3v1 ~14.5. Min_Vote_Open 20 now means "roughly
three-quarters of the ensemble's trust agrees, net". It MUST sit below the
ceiling to ever fire - the census/peak states the ceiling.
This is the ensemble the user specified in the original design discussion
("if the perceptron also votes, both together reach the threshold; if
another NN votes the other side, the threshold is not reached") - union
semantics was the pre-ensemble behaviour, kept until measurement showed its
vote magnitude was a constant.
Plus overlay DECLUSTERING, the other half of "arrows on every bar": the
same three NMS rules as the per-member arrows (same-direction runs collapse
to their first bar, cross-direction flicker keeps the stronger side), online
over the sweep's strictly oldest->newest walk. Suppression is a verdict and
deletes a standing arrow; the den==0 no-data skip still never does.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 06:58:03 -04:00
return ModuleWeight ( ) ;
}
feat(deploy): ship on positive EXPECTANCY, and let the chart draw before convergence
TWO CHANGES, both of which turn a permanent "nothing happens" into a decision.
1. THE DEPLOY GATE ASKS THE WRONG QUESTION. tradeable required the win rate to
clear chance by EDGE_MIN_SIGMAS - "can I PROVE an edge exists" from one OOS
window. On H4 that asks ~66% against a market supplying ~53%, so it is
unreachable by construction and no run has ever deployed through it.
SDeployVerdict now also carries the economics of the geometry actually being
traded - cost-adjusted break-even and reward:risk, both from the new
CostAdjustedGeometry() so a spread convention cannot be applied to one and
missed on the other - and derives
E[R] = (p - p*) * (1 + RR)
which is exactly zero at break-even by construction, so "profitable" and
"beats break-even" can never disagree. Under DeployOnExpectancy (new input,
default ON) tradeable becomes E[R] > 0 and selectionScore ranks eras by
expectancy instead of precision. Coverage and both-sides-live still gate
both: an expectancy over a handful of one-sided calls is not tradeable.
The struct also publishes scoreSE - the SE of selectionScore IN THE SCORE'S
OWN UNITS - because the score changes units with the objective (win-rate
points vs R). Both plateau bands now read it instead of precSE, which was
right for one objective and dimensionally wrong for the other.
Setting DeployOnExpectancy=false restores the previous behaviour exactly.
2. THE FILTERED VIEW COULD NOT DRAW WHILE ANY MODEL WAS TRAINING.
HistoricalNetVote built its divisor from VoteCapableWeight(), which answers
"may this member move real money" and returns 0.0 for an AI member until the
whole run converges. So the reconstruction's divisor was zero on EVERY bar,
every bar was skipped as "nobody looked", and the chart drew nothing at all -
for the entire training run, which before the plateau noise band was forever.
Reported as "no signals drawn since the refactor".
New ReconstructionWeight(): the same weight WITHOUT the converged-run
requirement, overridden on the AI member to ModuleWeight() gated on
SelfRanked() only. The overlay is a picture of what the vote WOULD have
shown, which a mid-training model can answer - the chart HUD already says so
with its "(trn)" marker. Live Direction() still uses VoteCapableWeight(), so
no untrained model gains a say in an order.
Compile-verified in the staging copy: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:15:06 -04:00
//--- Same weight, WITHOUT the converged-run requirement - see CExpertSignalCustom's declaration
//--- for why the chart reconstruction needs its own. Self-ranking IS still required: an unranked
//--- member's LiveVoteContribution is 0 by design (its tier ladder is the constructor's stock
//--- 25/50/75/100, which is the wrong unit, not a weak opinion), and putting a 0 contribution in
//--- the divisor would dilute the reconstruction with a member that never had an opinion to give.
feat(vote): edge-over-chance currency, no-skill exclusion, checkpoint burn-in
RETRAIN-FORCING and deliberately so. Two independent fixes for the same symptom -
charts that go quiet while others overtrade.
1. THE VOTE CURRENCY IS NOW EDGE OVER CHANCE, not an absolute win rate.
A tier weight is a raw win rate and a raw win rate means nothing without the
chance rate behind it: 30% is strong under a 14% base rate and catastrophic under
50%, yet both entered the mean as "30". That is why the threshold needed
re-tuning every time the label changed - 25 was permissive at ~70% win rates
under the old direction label and a near-unanimity rule at ~30% under the
pivot-event one - and why one chart's 25% was never the same statement as
another's. Subtracting the member's own chance rate makes the units percentage
points of demonstrated edge, comparable across charts, labels and regimes.
Clamped at zero: a below-chance tier is anti-informative, and contributing
negatively would act on a broken model as an inverted oracle rather than
discarding it.
2. A NO-SKILL MEMBER IS NOW ABSENT, NOT ABSTAINING.
Measured on XTIUSD: a Perceptron collapsed to B97/S6/N3, pooled win rate 11.5%
against a 14% chance rate - worse than guessing - and still voting. Three healthy
members voting Sell scored -21.06/0.77 = -27.4 and cleared; with the dead one
voting Buy it became (-21.06+1.44)/0.89 = -22.0 and was BLOCKED. It vetoed its
own ensemble on ~95% of bars, and that WAS the chart's 3.3% coverage. Neither
existing guard caught it: it IS self-ranked and its tier weights were 11-14.
The fix has to remove it from the DIVISOR, not just the sum - an abstainer
contributes weight by design, so zeroing only the contribution makes the dilution
worse. VoteCapableWeight() already means exactly "may this member's weight sit in
the denominator", so the skill test belongs there. ReconstructionWeight() and the
OOS scorer's divisor move with it or the scorer certifies a vote live does not
cast. The skill test reads the PREVIOUS era's measurement - gating this era's
vote on this era's own outcome would be circular.
3. CHECKPOINT BURN-IN (ENSEMBLE_CHECKPOINT_MIN_ERA 20).
XAUUSD deployed the checkpoint from ERA 2, XTIUSD from ERA 4, each after 69 and
65 further eras failed to beat it. Ensemble coverage measures AGREEMENT, and four
models that have barely moved off their initialisation agree almost by
construction - so coverage is inflated exactly when the models know least and
decays as they differentiate (XAUUSD 6.6% at era 8 -> 0.4% at era 75). Since
selectionScore is precision discounted by coverage, an early era outscores every
mature one and the ladder freezes on it.
INTENDED CONSEQUENCE: a chart whose MATURE coverage cannot clear the floor now
refuses to deploy rather than shipping era-2 weights. Fewer deploys, honest ones.
Burn-in eras are also kept out of g_ensCandidateEras (they could not have won, so
counting them inflates the family-wise N and raises the bar for nothing) and out
of g_ensErasSinceBest (or the run reaches "no better vote for N eras" with no
best to beat, exhausting the escalation ladder before the first era may compete).
Every pinned threshold and .stats record is in the OLD currency and is now
meaningless - this forces a fresh start on its own. Nothing needs re-tuning
because the threshold is DERIVED: the sweep re-picks the rung by itself.
Compiled clean; NOT yet run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 15:38:26 -04:00
//--- Same skill test as the live divisor, for the same reason: the overlay must picture the vote
//--- the EA would actually cast, and a no-skill member is absent from that vote.
feat(deploy): ship on positive EXPECTANCY, and let the chart draw before convergence
TWO CHANGES, both of which turn a permanent "nothing happens" into a decision.
1. THE DEPLOY GATE ASKS THE WRONG QUESTION. tradeable required the win rate to
clear chance by EDGE_MIN_SIGMAS - "can I PROVE an edge exists" from one OOS
window. On H4 that asks ~66% against a market supplying ~53%, so it is
unreachable by construction and no run has ever deployed through it.
SDeployVerdict now also carries the economics of the geometry actually being
traded - cost-adjusted break-even and reward:risk, both from the new
CostAdjustedGeometry() so a spread convention cannot be applied to one and
missed on the other - and derives
E[R] = (p - p*) * (1 + RR)
which is exactly zero at break-even by construction, so "profitable" and
"beats break-even" can never disagree. Under DeployOnExpectancy (new input,
default ON) tradeable becomes E[R] > 0 and selectionScore ranks eras by
expectancy instead of precision. Coverage and both-sides-live still gate
both: an expectancy over a handful of one-sided calls is not tradeable.
The struct also publishes scoreSE - the SE of selectionScore IN THE SCORE'S
OWN UNITS - because the score changes units with the objective (win-rate
points vs R). Both plateau bands now read it instead of precSE, which was
right for one objective and dimensionally wrong for the other.
Setting DeployOnExpectancy=false restores the previous behaviour exactly.
2. THE FILTERED VIEW COULD NOT DRAW WHILE ANY MODEL WAS TRAINING.
HistoricalNetVote built its divisor from VoteCapableWeight(), which answers
"may this member move real money" and returns 0.0 for an AI member until the
whole run converges. So the reconstruction's divisor was zero on EVERY bar,
every bar was skipped as "nobody looked", and the chart drew nothing at all -
for the entire training run, which before the plateau noise band was forever.
Reported as "no signals drawn since the refactor".
New ReconstructionWeight(): the same weight WITHOUT the converged-run
requirement, overridden on the AI member to ModuleWeight() gated on
SelfRanked() only. The overlay is a picture of what the vote WOULD have
shown, which a mid-training model can answer - the chart HUD already says so
with its "(trn)" marker. Live Direction() still uses VoteCapableWeight(), so
no untrained model gains a say in an order.
Compile-verified in the staging copy: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:15:06 -04:00
virtual double ReconstructionWeight ( void ) override
feat(vote): edge-over-chance currency, no-skill exclusion, checkpoint burn-in
RETRAIN-FORCING and deliberately so. Two independent fixes for the same symptom -
charts that go quiet while others overtrade.
1. THE VOTE CURRENCY IS NOW EDGE OVER CHANCE, not an absolute win rate.
A tier weight is a raw win rate and a raw win rate means nothing without the
chance rate behind it: 30% is strong under a 14% base rate and catastrophic under
50%, yet both entered the mean as "30". That is why the threshold needed
re-tuning every time the label changed - 25 was permissive at ~70% win rates
under the old direction label and a near-unanimity rule at ~30% under the
pivot-event one - and why one chart's 25% was never the same statement as
another's. Subtracting the member's own chance rate makes the units percentage
points of demonstrated edge, comparable across charts, labels and regimes.
Clamped at zero: a below-chance tier is anti-informative, and contributing
negatively would act on a broken model as an inverted oracle rather than
discarding it.
2. A NO-SKILL MEMBER IS NOW ABSENT, NOT ABSTAINING.
Measured on XTIUSD: a Perceptron collapsed to B97/S6/N3, pooled win rate 11.5%
against a 14% chance rate - worse than guessing - and still voting. Three healthy
members voting Sell scored -21.06/0.77 = -27.4 and cleared; with the dead one
voting Buy it became (-21.06+1.44)/0.89 = -22.0 and was BLOCKED. It vetoed its
own ensemble on ~95% of bars, and that WAS the chart's 3.3% coverage. Neither
existing guard caught it: it IS self-ranked and its tier weights were 11-14.
The fix has to remove it from the DIVISOR, not just the sum - an abstainer
contributes weight by design, so zeroing only the contribution makes the dilution
worse. VoteCapableWeight() already means exactly "may this member's weight sit in
the denominator", so the skill test belongs there. ReconstructionWeight() and the
OOS scorer's divisor move with it or the scorer certifies a vote live does not
cast. The skill test reads the PREVIOUS era's measurement - gating this era's
vote on this era's own outcome would be circular.
3. CHECKPOINT BURN-IN (ENSEMBLE_CHECKPOINT_MIN_ERA 20).
XAUUSD deployed the checkpoint from ERA 2, XTIUSD from ERA 4, each after 69 and
65 further eras failed to beat it. Ensemble coverage measures AGREEMENT, and four
models that have barely moved off their initialisation agree almost by
construction - so coverage is inflated exactly when the models know least and
decays as they differentiate (XAUUSD 6.6% at era 8 -> 0.4% at era 75). Since
selectionScore is precision discounted by coverage, an early era outscores every
mature one and the ladder freezes on it.
INTENDED CONSEQUENCE: a chart whose MATURE coverage cannot clear the floor now
refuses to deploy rather than shipping era-2 weights. Fewer deploys, honest ones.
Burn-in eras are also kept out of g_ensCandidateEras (they could not have won, so
counting them inflates the family-wise N and raises the bar for nothing) and out
of g_ensErasSinceBest (or the run reaches "no better vote for N eras" with no
best to beat, exhausting the escalation ladder before the first era may compete).
Every pinned threshold and .stats record is in the OLD currency and is now
meaningless - this forces a fresh start on its own. Nothing needs re-tuning
because the threshold is DERIVED: the sweep re-picks the rung by itself.
Compiled clean; NOT yet run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 15:38:26 -04:00
{ return HasDemonstratedEdge ( ) ? ModuleWeight ( ) : 0.0 ; }
2026-08-22 00:24:45 -04:00
//--- Gate for the settled PER-ERA diagnostics - see TRAIN_LOG_EVERY_ERAS. HOW OFTEN A DIRECTION
//--- ACTUALLY OCCURS, as a percentage of labelled bars. Prints that mark a state CHANGE (new
//--- best, stage transition, restore, deploy verdict, warning) must never be put behind this; it
//--- exists only for the lines that repeat with the era heartbeat.
feat(calibration): fit the operating point on the label rate instead of on edge
The margin threshold now sits where the model calls a direction as often as a
direction actually occurs. Nothing else.
WHY THE OLD OBJECTIVE HAD TO GO. It maximised `coverage x (precision -
breakEven)`, and this function's own comments were already the case against
it: over 98 consecutive fits of the shipped SP500 H4 model, correlation
between the chosen threshold and the win rate at it was -0.056, while the
era-to-era spread of that win rate (1.32pp) matched its own binomial SE
(1.25pp) to within 0.07pp. The margin does not rank trades. So the argmax
returned whichever of ~37 bins drew the luckiest sample, and the threshold
teleported 0.42 -> 0.04 -> 0.74 in three eras.
The response at the time was to build a null-of-the-maximum gate, an
effective-sample SE and a parsimony fallback to hold the noise down. All of
that is gone now, because fitting on calibration removes the problem instead
of bounding it: coverage is a ratio against a fixed denominator so it is well
determined at every bin, the target is a measured label rate rather than an
outcome, and nothing is maximised over a noisy curve so there is no best-of-N
to correct for. Net 174 lines out, 62 in.
It deliberately does not chase edge. It cannot - at ~0 measured edge no
operating point has more of it, and pretending otherwise is what produced a
threshold of 0.96 that still passed 60% of bars while the model called a
direction ~10x too often. The edge at the chosen point is still REPORTED,
just no longer what chooses it.
THREE READINGS OF ONE QUANTITY, AND THEY DISAGREE. "How often does a direction
occur" is measured in three places and gives ~7% (the scan's own tally), ~41%
(the era loop's counters, via this function's old coverage floor) and ~50%
(the ensemble gate's OOS base rate). They cannot all be right. Rather than
pick one silently, ScanDirectionalRatePct() and EraDirectionalRatePct() are
now named accessors, the fitter targets the SCAN - that is the tally the
operator reads, and the one "predict the labels as measured during the scan
phase" names - and the threshold line PRINTS BOTH every time it moves, so the
disagreement is on the record instead of buried in a derived floor.
The ensemble gate's own floor is deliberately NOT changed in this commit. If
the scan is right, a calibrated member covering ~7% of bars cannot clear a
12.4% floor and every model would fail the gate by construction; if the gate
is right, the scan tally is wrong. The CALIBRATION field added in 667f2bc
reports the OOS true class rates directly and settles it in one era - that
measurement comes first, and the floor follows it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 10:20:31 -04:00
double ScanDirectionalRatePct ( void ) const
{
long tot = ( long ) m_labelPrebuildBuyCount + m_labelPrebuildSellCount + m_labelPrebuildNeutralCount ;
if ( tot < = 0 )
return -1.0 ;
return 100.0 * ( double ) ( m_labelPrebuildBuyCount + m_labelPrebuildSellCount ) / tot ;
}
double EraDirectionalRatePct ( void ) const
{
long tot = ( long ) m_trueBuyCount + m_trueSellCount + m_trueNeutralCount ;
if ( tot < = 0 )
return -1.0 ;
return 100.0 * ( double ) ( m_trueBuyCount + m_trueSellCount ) / tot ;
}
revert(labels): drop the one-sided exit target; measure the calibration drift instead
Reverts a863796 on the operator's call - "unnecessary complexity". It was
right about the mechanism and wrong about the priority: it re-cut the classes
for a case the measured verdict never reaches (SP500 H4 reads "both sides" at
the derived geometry), while the drift that IS happening affects every chart
and every era. Recoverable from a863796 if a one-sided book ever becomes real.
Two pieces of it survive, both independent of the exit idea:
The drift verdict keeps reading m_winLongCache/m_winShortCache rather than the
collapsed label pair. That line reports always-long vs always-short win rates,
which is what the win caches hold - each side scored on its own barriers,
published before the collapse. The label pair carries only the side touched
first, so it undercounted long wins by the both-won-goes-to-short share. There
are zero both-won bars at any geometry with target >= stop, so this changes no
number today; it changes the wrong number to the right one.
And the .cfg gains nothing and loses nothing: the two appended ints go away
again, and they were the last fields, so a .cfg written by yesterday's build
still reads correctly - the loader simply stops before them.
WHAT THE REVERT MAKES ROOM FOR. The operator's actual requirement is that the
model reproduce the label distribution the scan measured, and nothing in the
pipeline ties it to that. The loss trains on a rebalanced sample and the
abstain rate is owned by a margin threshold fitted on EDGE, so the call rate
and the label prior can drift arbitrarily far apart - and did, invisibly:
at era 1350 the models call Buy on 20-28% and Sell on 22-32% of bars against
a scan-measured 2.1% and 4.8%. Roughly a 10x over-call, and not one line in
the journal said so.
The era line now carries it:
CALIBRATION calls vs true rate Buy 28% vs 2% (14.0x) Sell 32% vs 5% (6.4x)
Neutral 40% vs 93% (0.4x)
Reported as a ratio because that is the readable number - 1.0x is calibrated.
This is deliberately a measurement and not yet a correction: matching the
label rate would put coverage near 7%, below the ensemble gate's own 12.4%
coverage floor, so calibration and the gate are in direct conflict and which
one yields is the operator's call, not mine.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 10:14:26 -04:00
//--- "3.2x" / "0.4x" / "n/a" for a predicted-vs-true class rate pair. Both are already rounded
//--- percentages, so a true rate of 0 has no ratio to report rather than an infinite one.
string CalibrationRatio ( const int predPct , const int truePct ) const
{
if ( predPct < 0 | | truePct < = 0 )
return " n/a " ;
return StringFormat ( " %.1fx " , ( double ) predPct / ( double ) truePct ) ;
}
feat(logs): throttle the settled per-era diagnostics - measured 22MB/9.5h of confirmed-working systems
Measured from the journal (2026-08-19): the era deep-dive line (~2KB) plus
the excursion verdict, tier re-rank, calibration move, barrier hold and
selection-regressed note each printed EVERY era for EVERY member - ~940
eras/member/day - long after the systems they watch were confirmed
working. Yesterday's file was 1.3GB (70% of it the news-filter calendar
spam the sweep fix already removed).
VerboseMode returns as an INPUT (demoted 2026-08-01 for the marketplace;
that track is dead since the 2026-08-16 pivot) and gains a second job:
false throttles each settled per-era print to eras 0-3 plus every
TRAIN_LOG_EVERY_ERAS-th (25 ~= one deep-dive per ~15min per member);
true restores the per-era firehose, flippable live.
Never throttled: anything that marks a CHANGE - new bests, restores +
eta decays, plateau stage transitions, deploy approvals, warnings,
errors, the label-cache/adoption one-shots, and the combined-vote gate
line (the active system's primary telemetry, still every era).
Semantic fixes over blanket gating:
- barrier hold now ARMS silently and prints only when the hold outlasts
the 2-min report interval - a brief hold every era is the design, the
long hold is the watchdog case the line exists for;
- the ensemble deploy REFUSAL prints immediately when its reason
changes (that is a finding), on cadence when unchanged;
- the filtered-view census prints when its RESULT moves (drawn count,
or strongest vote by >=2pp) and at least every 10th sweep.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 09:36:15 -04:00
bool TrainLogDue ( void ) const
{
return VerboseMode | | m_eraCount < = 3 | | ( m_eraCount % TRAIN_LOG_EVERY_ERAS = = 0 ) ;
}
2026-08-22 00:24:45 -04:00
//--- What this model would vote on the CURRENT bar if it were deployed. The readiness gate in
//--- LongCondition() is what this bypasses, and ONLY for display: dPrevSignal is the decision,
//--- deployed or not.
feat(chart): show the PROSPECTIVE vote while the models are still training
The readout sat at "VOTE 0.0%, 0 voters" constantly. Correct, and useless.
LongCondition()/ShortCondition() return 0 behind the readiness gate for the
entire training run - a model that is not deployed does not vote - so the LIVE
vote is structurally zero for hours, which is exactly the period the readout
is being watched. Worse, it was the same display whether the models were
silent, undeployed, or the filter list was empty: three different situations,
one number.
When no filter casts a real vote, the readout now shows the PROSPECTIVE one -
what these models are saying right now, through the identical tier/weight
arithmetic, minus the readiness gate. That is the same quantity the historical
overlay reconstructs on cached bars, deliberately, so the live line and the
reconstructed arrows are the same measure and can be read against each other.
It can never be mistaken for a decision: labelled "-> training, not tradable
yet", drawn dimmer than "no trade", and `fires` is forced false regardless of
magnitude, because saying "-> TRADE" about a number that cannot place an order
is the precise overstatement this readout exists to prevent. m_direction is
untouched - display only, no trading path reads it.
Confirms the sweep fix from 155f56e is live and working:
"Filtered view: swept 4999 bar(s), 767 had a voter, drew 0 arrow(s).
Strongest vote 36.0% against a 40.0% threshold."
4,999 bars against the previous 0. The remaining emptiness is the models, not
the plumbing - see the reply for why lowering the threshold further is the
wrong response to it.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 18:29:27 -04:00
virtual bool ProspectiveVote ( double & signedVote , double & weight ) override
{
signedVote = 0.0 ;
weight = 0.0 ;
2026-08-22 00:24:45 -04:00
//--- FRESH FORWARD FIRST. DisplayInference() asks the CURRENT weights the live question on a
//--- ~4s throttle instead.
feat(hud): per-member neuron lines + a vote label that moves as the nets learn
Both 2026-08-19 reports were the same staleness: every source behind the
label was an ERA artifact (live cache refills at pass-3 completion, the
snapshot copies once per era, dPrevSignal is the frozen purge-band edge
bar) - so the readout stepped at era cadence at best, stayed glued to
one direction, and lagged the era counter.
DisplayInference(): throttled (4s, 1s across an era boundary),
SIDE-EFFECT-FREE forward of the current decision bar (window ending on
bar 1, same question the live path asks) through the LEARNER net.
Batch-norm running stats are bracketed frozen/RESTORED via the new
CNet::GetBatchNormFrozen() + CNeuronBatchNormOCL::StatsFrozen() - restore,
not unfreeze, because a display tick can land between pass-3 chunks whose
whole scan holds them frozen. Writes nothing a trading or training path
reads (dPrevSignal, NMS state, tallies, watermarks all untouched;
RefreshLatestSignal is not reusable here precisely because it writes all
of them). LSTM safe by construction: h/c zeroed per forward.
ProspectiveVote() reads the fresh forward as its FIRST source; the
era-artifact chain becomes the fallback (meta head, warm-up, window
holes).
DisplayHudLine(): the reference library's training label, per ensemble
member - name, output activations (softmax probs or raw scalar), the
decision, its weighted vote (the exact consensus numerator term), era,
recent average error, "(trn)" while not vote-capable. Rendered under the
vote line in RefreshVoteReadout BEFORE the live-vote defer (member lines
are telemetry, not tradable readings), coloured by the member's own
direction in muted tones - the vote line's strict
green-only-when-it-would-trade rule is untouched.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 08:48:34 -04:00
if ( DisplayInference ( ) )
{
signedVote = LiveVoteContribution ( m_dispSignal ) ;
weight = ModuleWeight ( ) ;
return true ;
}
//--- ERA-ARTIFACT FALLBACK CHAIN, newest first - reached only when the fresh forward above
2026-08-22 00:24:45 -04:00
//--- cannot run (meta head, warm-up, indicator hole).
fix(chart): the prospective vote was four models' opinion of ONE frozen bar
"Still glued to buy." Verified in the pass structure rather than guessed:
dPrevSignal is written ONLY by pass 1 (Training.mqh 1439/1442 - the sole
assignment sites), and pass 1 SKIPS the feedForward for any bar a later pass
will forward anyway (laterPassForwards) - which is the whole OOS window and
the calibration band. Pass 3 forwards the newest bars every era but never
writes dPrevSignal. Net effect: after every era, dPrevSignal holds the
model's opinion of the newest PURGE-BAND EDGE BAR pass 1 happened to forward
- one fixed mid-history bar, re-evaluated era after era. The readout was
therefore showing four models' verdict on the same frozen bar, and that bar
reads Buy. Glued to Buy, with flashes of Sell only while pass 1 was actively
walking (the one window where dPrevSignal moves).
ProspectiveVote() now reads the newest ARROW-CACHE entry first (walking back
from the decision bar, bounded at 16), falling back to dPrevSignal only when
the cache holds nothing. Pass 3 writes the adjusted decision for the newest
(OOS) bars each era, so the cache's first non-sentinel entry is the model's
most recent verdict on near-current data - and it is the same value the
filtered overlay draws from, so the label and the reconstruction stay one
quantity. A cached Neutral stops the walk: that is a real decision (vote 0,
abstain -> shows in the "flat" count), not a missing one. Early in an era
the cache is wiped to sentinel and everything falls through to dPrevSignal
exactly as before, until pass 3 refills the newest rows.
Expect the label to change per era now (as each pass 3 re-scores the newest
bars under that era's weights), with the vote/flat split moving as members
genuinely flip between direction and Neutral on recent data.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 21:47:51 -04:00
for ( int idx = 1 ; idx < = 16 ; idx + + )
{
if ( CachedVoteAt ( idx , signedVote ) )
{
weight = ModuleWeight ( ) ;
return true ;
}
}
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era
Full-pipeline analysis after "threshold 30, attained often, nothing drawn,
still glued to buy". The log falsified the premise before any code did:
21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0%
21:42:07 swept 4999, 0 voters, drew 0
21:51:30 swept 4999, 0 voters, drew 0
21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0%
The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root
cause, three symptoms: every display path read m_arrowSignalCache, which is
wiped to sentinel at each era start and only complete again when pass 3
finishes. With eras at ~30s and a sweep at ~17s:
* ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its
else-branch deleted the arrow on every voteless bar - erasing the previous
sweep's entire output. The chart cycled populated -> blank -> populated;
the user kept catching the blank phase.
* READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every
era and fell through to dPrevSignal - the frozen purge-band edge bar that
reads Buy. 659638e fixed which bar was frozen, not the freezing.
* VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a
different fraction of half-rebuilt caches.
THE FIX, structural rather than another patch:
1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one
moment the cache is complete - and now copies it (raw signals, newest
LOOKBACK+16 bars) into member-owned snapshot state, unconditionally,
BEFORE its early return: an all-Neutral era is a snapshot worth showing,
not an absence of one. Raw signals rather than votes, so a tier re-rank
between eras reprices them at read time via LiveVoteContribution for free.
2. The sweep (SnapshotVoteAt) and the prospective readout both read
snapshots; the readout's fallback chain is live-cache -> snapshot ->
dPrevSignal, and the snapshot leg is the one that fires most of the time.
3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual
sub-threshold vote takes an arrow down. This alone ends the wipe half of
the flicker even where snapshots are missing (before the first era).
4. Arming moved from an era-counter diff (which fires at era BOUNDARIES,
i.e. precisely when caches are about to be wiped) to
g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's
snapshot just got fresher", the only event a redraw can act on. 60s rate
limit collapses the four members' burst into one sweep. Classic-only
charts arm once at start.
5. Census now reports the direction split - "922 had a voter (610 buy / 312
sell)" - so "the vote leans buy" is checkable from the log instead of
inferred from arrow colours.
Also visible in the log and worth knowing: the threshold flip-flopped
30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51
ran at 40), so part of the observed blankness was configuration, not code.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 22:03:20 -04:00
//--- Era-end snapshot next - the fallback that actually fires for ~90% of every era, because
2026-08-22 00:24:45 -04:00
//--- the live cache above is wiped at era start and only refills when pass 3 completes.
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era
Full-pipeline analysis after "threshold 30, attained often, nothing drawn,
still glued to buy". The log falsified the premise before any code did:
21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0%
21:42:07 swept 4999, 0 voters, drew 0
21:51:30 swept 4999, 0 voters, drew 0
21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0%
The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root
cause, three symptoms: every display path read m_arrowSignalCache, which is
wiped to sentinel at each era start and only complete again when pass 3
finishes. With eras at ~30s and a sweep at ~17s:
* ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its
else-branch deleted the arrow on every voteless bar - erasing the previous
sweep's entire output. The chart cycled populated -> blank -> populated;
the user kept catching the blank phase.
* READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every
era and fell through to dPrevSignal - the frozen purge-band edge bar that
reads Buy. 659638e fixed which bar was frozen, not the freezing.
* VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a
different fraction of half-rebuilt caches.
THE FIX, structural rather than another patch:
1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one
moment the cache is complete - and now copies it (raw signals, newest
LOOKBACK+16 bars) into member-owned snapshot state, unconditionally,
BEFORE its early return: an all-Neutral era is a snapshot worth showing,
not an absence of one. Raw signals rather than votes, so a tier re-rank
between eras reprices them at read time via LiveVoteContribution for free.
2. The sweep (SnapshotVoteAt) and the prospective readout both read
snapshots; the readout's fallback chain is live-cache -> snapshot ->
dPrevSignal, and the snapshot leg is the one that fires most of the time.
3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual
sub-threshold vote takes an arrow down. This alone ends the wipe half of
the flicker even where snapshots are missing (before the first era).
4. Arming moved from an era-counter diff (which fires at era BOUNDARIES,
i.e. precisely when caches are about to be wiped) to
g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's
snapshot just got fresher", the only event a redraw can act on. 60s rate
limit collapses the four members' burst into one sweep. Classic-only
charts arm once at start.
5. Census now reports the direction split - "922 had a voter (610 buy / 312
sell)" - so "the vote leans buy" is checkable from the log instead of
inferred from arrow colours.
Also visible in the log and worth knowing: the threshold flip-flopped
30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51
ran at 40), so part of the observed blankness was configuration, not code.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 22:03:20 -04:00
if ( m_prospectiveSigSnap ! = -2.0 & & MathIsValidNumber ( m_prospectiveSigSnap ) )
{
signedVote = LiveVoteContribution ( m_prospectiveSigSnap ) ;
weight = ModuleWeight ( ) ;
return true ;
}
fix(chart): the prospective vote was four models' opinion of ONE frozen bar
"Still glued to buy." Verified in the pass structure rather than guessed:
dPrevSignal is written ONLY by pass 1 (Training.mqh 1439/1442 - the sole
assignment sites), and pass 1 SKIPS the feedForward for any bar a later pass
will forward anyway (laterPassForwards) - which is the whole OOS window and
the calibration band. Pass 3 forwards the newest bars every era but never
writes dPrevSignal. Net effect: after every era, dPrevSignal holds the
model's opinion of the newest PURGE-BAND EDGE BAR pass 1 happened to forward
- one fixed mid-history bar, re-evaluated era after era. The readout was
therefore showing four models' verdict on the same frozen bar, and that bar
reads Buy. Glued to Buy, with flashes of Sell only while pass 1 was actively
walking (the one window where dPrevSignal moves).
ProspectiveVote() now reads the newest ARROW-CACHE entry first (walking back
from the decision bar, bounded at 16), falling back to dPrevSignal only when
the cache holds nothing. Pass 3 writes the adjusted decision for the newest
(OOS) bars each era, so the cache's first non-sentinel entry is the model's
most recent verdict on near-current data - and it is the same value the
filtered overlay draws from, so the label and the reconstruction stay one
quantity. A cached Neutral stops the walk: that is a real decision (vote 0,
abstain -> shows in the "flat" count), not a missing one. Early in an era
the cache is wiped to sentinel and everything falls through to dPrevSignal
exactly as before, until pass 3 refills the newest rows.
Expect the label to change per era now (as each pass 3 re-scores the newest
bars under that era's weights), with the vote/flat split moving as members
genuinely flip between direction and Neutral on recent data.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 21:47:51 -04:00
signedVote = 0.0 ;
feat(chart): show the PROSPECTIVE vote while the models are still training
The readout sat at "VOTE 0.0%, 0 voters" constantly. Correct, and useless.
LongCondition()/ShortCondition() return 0 behind the readiness gate for the
entire training run - a model that is not deployed does not vote - so the LIVE
vote is structurally zero for hours, which is exactly the period the readout
is being watched. Worse, it was the same display whether the models were
silent, undeployed, or the filter list was empty: three different situations,
one number.
When no filter casts a real vote, the readout now shows the PROSPECTIVE one -
what these models are saying right now, through the identical tier/weight
arithmetic, minus the readiness gate. That is the same quantity the historical
overlay reconstructs on cached bars, deliberately, so the live line and the
reconstructed arrows are the same measure and can be read against each other.
It can never be mistaken for a decision: labelled "-> training, not tradable
yet", drawn dimmer than "no trade", and `fires` is forced false regardless of
magnitude, because saying "-> TRADE" about a number that cannot place an order
is the precise overstatement this readout exists to prevent. m_direction is
untouched - display only, no trading path reads it.
Confirms the sweep fix from 155f56e is live and working:
"Filtered view: swept 4999 bar(s), 767 had a voter, drew 0 arrow(s).
Strongest vote 36.0% against a 40.0% threshold."
4,999 bars against the previous 0. The remaining emptiness is the models, not
the plumbing - see the reply for why lowering the threshold further is the
wrong response to it.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 18:29:27 -04:00
if ( ! MathIsValidNumber ( dPrevSignal ) )
return false ;
signedVote = LiveVoteContribution ( dPrevSignal ) ;
weight = ModuleWeight ( ) ;
return true ;
}
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era
Full-pipeline analysis after "threshold 30, attained often, nothing drawn,
still glued to buy". The log falsified the premise before any code did:
21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0%
21:42:07 swept 4999, 0 voters, drew 0
21:51:30 swept 4999, 0 voters, drew 0
21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0%
The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root
cause, three symptoms: every display path read m_arrowSignalCache, which is
wiped to sentinel at each era start and only complete again when pass 3
finishes. With eras at ~30s and a sweep at ~17s:
* ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its
else-branch deleted the arrow on every voteless bar - erasing the previous
sweep's entire output. The chart cycled populated -> blank -> populated;
the user kept catching the blank phase.
* READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every
era and fell through to dPrevSignal - the frozen purge-band edge bar that
reads Buy. 659638e fixed which bar was frozen, not the freezing.
* VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a
different fraction of half-rebuilt caches.
THE FIX, structural rather than another patch:
1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one
moment the cache is complete - and now copies it (raw signals, newest
LOOKBACK+16 bars) into member-owned snapshot state, unconditionally,
BEFORE its early return: an all-Neutral era is a snapshot worth showing,
not an absence of one. Raw signals rather than votes, so a tier re-rank
between eras reprices them at read time via LiveVoteContribution for free.
2. The sweep (SnapshotVoteAt) and the prospective readout both read
snapshots; the readout's fallback chain is live-cache -> snapshot ->
dPrevSignal, and the snapshot leg is the one that fires most of the time.
3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual
sub-threshold vote takes an arrow down. This alone ends the wipe half of
the flicker even where snapshots are missing (before the first era).
4. Arming moved from an era-counter diff (which fires at era BOUNDARIES,
i.e. precisely when caches are about to be wiped) to
g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's
snapshot just got fresher", the only event a redraw can act on. 60s rate
limit collapses the four members' burst into one sweep. Classic-only
charts arm once at start.
5. Census now reports the direction split - "922 had a voter (610 buy / 312
sell)" - so "the vote leans buy" is checkable from the log instead of
inferred from arrow colours.
Also visible in the log and worth knowing: the threshold flip-flopped
30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51
ran at 40), so part of the observed blankness was configuration, not code.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 22:03:20 -04:00
//--- The sweep's data source - see CExpertSignalCustom::SnapshotVoteAt for why this is a
//--- snapshot and not the live cache.
virtual bool SnapshotVoteAt ( const int idx , double & signedVote ) override
{
signedVote = 0.0 ;
if ( idx < 0 | | idx > = m_overlaySnapBars )
return false ;
double sig = m_overlaySigSnap [ idx ] ;
if ( sig = = -2.0 | | ! MathIsValidNumber ( sig ) )
return false ;
signedVote = LiveVoteContribution ( sig ) ;
return true ;
}
feat(chart): reconstruct the filtered view behind the handover point
Completes the filtered view from 282b535, which only reached forward of
attach. On a multi-hour training run that is the entire time you are looking
at the chart, so the answer to "how would the whole bot have traded" was
blank exactly when it was wanted.
The sweep lives on the AGGREGATE signal, which is the only object holding
every filter. AI members contribute their CACHED per-bar decision from the
era scan - no inference re-runs, the cache already spans the chart - and the
classic ladders are replayed with EvalShift(i), the same mechanism
CSignalMETA's candidate sweep uses and exact because every classic pattern
condition anchors on StartIndex(). Combination is the live one: weighted mean
over voting filters, abstentions out of both sums, against Min_Vote_Open.
THE REPLAY CORRUPTS LIVE JOURNALING IF LEFT UNGUARDED, and this is the part
that is not obvious. Live journaling reads m_active_pattern_long/short from
the PREVIOUS Direction() call. Replaying hundreds of past bars between two
live bars leaves those slots holding whichever bar the sweep stopped on, so
the next live bar journals that pattern under the current timestamp - a
corrupted row in the very table pattern win rates are computed from, which is
now also where vote weights come from. Save/RestoreVoteState() brackets every
replayed call. CSignalMETA gets away without it only because its sweep runs
once, at the first era, before any of that state matters.
TWO SOURCES OF TRUTH, KEPT APART. A reconstruction cannot know the broker
rejected an order - it has no stops level, ATR warm-up or swing-history sync
as they were at that moment - so it is an upper bound: honest about the vote,
optimistic about placement. It therefore stops dead at the handover bar,
which is latched ONCE so later rebuilds cannot creep it forward and start
overwriting real decisions with guesses, and its arrows say "reconstructed
(vote only - order validation not replayed)" in the tooltip. Someone
comparing two arrows either side of that line has to be able to tell which is
a record and which is a replay, and the chart is the only place they look.
Re-armed on any era boundary (summed era counters), because that is when the
answer changes - RankTiersFromOos has just re-derived every tier's vote weight
- and only between sweeps, so a restart cannot leave the previous pass's tail
undrawn. Chunked at 150 bars per timer slice: each bar replays Direction() on
every classic filter, which is real indicator work on the chart thread, and an
unchunked sweep here is the 2026-07-26 arrow-restore freeze waiting to happen.
SIGNAL_RESCAN_LOOKBACK_BARS moves to ExpertSignalCustom.mqh alongside
SIG_ARROW_PREFIX - same include-order reason, and the two rebuilds should
reach the same distance or the raw and filtered views are not comparable.
Known gap: on a classic-only chart the reconstruction is built once and not
refreshed when the hourly DB ranking moves the classic weights.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:05:13 -04:00
virtual bool CachedVoteAt ( const int idx , double & signedVote ) override
{
signedVote = 0.0 ;
if ( idx < 0 | | idx > = ArraySize ( m_arrowSignalCache ) )
return false ;
double sig = m_arrowSignalCache [ idx ] ;
if ( sig = = -2.0 | | ! MathIsValidNumber ( sig ) )
return false ;
signedVote = LiveVoteContribution ( sig ) ;
return true ;
}
feat(vote): backfill the ensemble win-rate record from the overlay sweep
"Vote win rate: measuring..." never resolved on a deployed chart whose
.stats predate the WST7 ensemble record: g_ensCumOosTotal is fed only by
the era-end combined-vote scorer (Training.mqh), and a deployed ensemble
runs no further eras. The replay pass rebuilt every MEMBER's ladder
(64-71% each, per the 16:12 log) but nothing ever scored the COMBINED
vote, so the aggregate line sat on "measuring" while 300+ arrows drew.
The overlay sweep already reconstructs the vote per bar with the live
threshold and direction policy - so it now also tallies, BEFORE
declustering (NMS thins arrows, not calls), each threshold-clearing bar
against the inline swing-pivot label (same resolution ScoreReplayFromCache
uses, same window-mismatch reason). On sweep completion Warrior_EA.mq5
harvests the tally through a consuming one-shot read and adopts it ONLY
when the record is empty and the models are deployed - a training-time
sweep can never pre-empt the era scorer, and a restored record always
wins. The result is persisted immediately into every member's .stats.
Also verified against the same log: the sweep does NOT ignore
DrawUnfilteredSignals - 4986 voter bars -> ~300 arrows, all gated on the
25% open threshold. The arrow increase vs the restored set (41-312 saved)
is the replay-minted ladder reading stronger (partly in-sample), plus the
reconstruction deliberately not replaying order validation/session hours
(tooltip says so); the backfilled record carries the same caveat and is
labelled so in the log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 18:22:12 -04:00
//--- The overlay sweep's truth source - the same inline label resolution ScoreReplayFromCache
//--- uses, for the same reason (see its window-mismatch comment). Body in AIBase\Lifecycle.mqh.
virtual bool ReplayTruthAt ( const int idx , ENUM_SIGNAL & truth ) override ;
2026-07-23 08:21:41 -04:00
//--- buckets the live confidence magnitude into one of the 4 tiers above - see m_pattern_0's
//--- declaration comment. Public so PollTraining()/status-display code could surface which tier is
//--- currently active if ever useful, though LongCondition/ShortCondition are the only callers today.
int ConfidenceTier ( void ) ;
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
//--- The same bucketing asked of an ARBITRARY decision value rather than of dPrevSignal. Split out
//--- so the OOS scan can ask "what tier would this scanned bar have voted at" - it holds the bar's
//--- decision in a local, and dPrevSignal is the LIVE bar's, which is a different bar entirely.
int ConfidenceTierFor ( const double signal ) ;
2026-07-23 08:21:41 -04:00
int PatternWeightForTier ( int tier ) ;
2026-08-22 00:24:45 -04:00
//--- THE VOTE THIS MEMBER WOULD CAST, in the units CExpertSignalCustom::Direction() actually
//--- sums: m_weight (0..1, DB-ranked) x the tier's pattern weight (0..100, DB-ranked), signed +
//--- for Buy and - for Sell, and exactly 0.0 when the decision is Neutral (an abstention, which
//--- live drops from BOTH the sum and the divisor).
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
double LiveVoteContribution ( const double signal ) ;
2026-08-22 00:24:45 -04:00
//--- methods of setting adjustable parameters No public setter for m_initialNeuronsCount. An
//--- external setter could only ever be called after construction and would either be ignored
//--- (if before init) or silently re-key the model mid-run (if after).
2026-07-14 22:36:27 -04:00
void OutputNeuronsCount ( int value ) { m_outputNeuronsCount = value ; }
refactor(yagni): drop 13 accessors nothing called; unify the ATR trailing pair
Verified dead by grep across all first-party sources (references/, Scripts/,
research/ excluded): EraCount, HiddenLayersCount, LstmHiddenSize, ConvFilterCount,
HistoryBars and MinTrainYear setters, PendingBatchSamples, getPrevOutIndex,
BaseCurrency, QuoteCurrency, CurrencyCount, IsLoaded, LastFiredDirection,
DBConfidence, SpecIndex, and the conv Step/WindowOut shape accessors. Every
backing member stays - each is still read internally and several are pinned by
the positional .cfg layout - so this removes surface, not behaviour.
Two comments were asserting the opposite of the code and are now true: the
"No setter: the taper's endpoints are derived" note was directly above three
setters, and the conv shape block claimed EnforceTopologyContract reads all
three accessors when CNet::FirstConvWindow only ever calls Window().
CTrailingATR::CheckTrailingStopLong/Short were byte-identical but for Bid vs Ask
and the isLong flag; both now delegate to one CheckTrailingStop body.
Deliberately NOT removed: the fractal-target branch (TrainTargetFractal,
IsFractalTarget and their label machinery). It reads as dead because the
TrainingTarget input was withdrawn, but Warrior_EA.mq5:836 documents it as a
parked option with a three-line restore path - that is a product call, not a
refactor.
Not compiled - MetaEditor compile pending.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 18:55:36 -04:00
//--- No setters for m_hiddenLayersCount / m_lstmHiddenSize / m_convFilterCount: the taper's
2026-08-22 00:24:45 -04:00
//--- endpoints are derived, not configured. See BuildFreshTopology()'s taper block.
//--- MinSignalConfidence(double) removed with the AI entry floor - confidence now reaches the
//--- trade decision as vote weight (ConfidenceTier), gated by the one Min vote to open threshold
//--- that the classic votes already answer to.
2026-07-28 17:42:12 -04:00
void FreezePriorCalibration ( bool value ) { m_freezePriorCalibration = value ; }
2026-07-21 00:03:45 -04:00
void SignalClusterWindow ( int value ) { m_signalClusterWindow = value ; }
feat(signal): make the signal cooldown tunable, and add a hard any-direction gate
The declustering the charts needed already existed - NmsLiveAccept, per-direction
run-collapse plus cross-direction resolution plus strict alternation - and it was
already set to 10 bars. It could not be TUNED: SignalClusterWindow was a compile-
time const, so finding the right value needed a rebuild. That is the actual gap.
Now three inputs, as enum dropdowns:
Signal_CooldownScope per-direction, or a hard any-direction gate on top
Signal_CooldownBars SCB_OFF..SCB_50, default 10
Signal_CooldownMinutes SCM_OFF..SCM_1440, overrides bars when set
Minutes resolve against the CHART period and round UP, so a cooldown asked for in
wall-clock is never silently shorter than requested and survives a timeframe
change.
SCB_/SCM_ prefixes are deliberately unique. M15/M30/M60 are ALREADY members of
NF_LOOKBACK_PRESETS, and MQL5 binds a duplicated enum member to the first-declared
enum silently - the obvious names would have compiled straight into the news
filter's values.
THE ANY-DIRECTION GATE IS ADDITIVE, NOT A REPLACEMENT, and the first cut of this
had it backwards. Measured on the live log: the current rules draw 222 arrows over
4999 bars, while a BARE 10-bar cooldown permits up to 454 - because ALTERNATION is
what declutters today, not the window. Swapping the rules out would have roughly
doubled the clutter it was asked to remove. Layered, it can only ever suppress
more. Suppressed bars still advance the per-direction last-SEEN cursors, so a run
straddling the boundary does not restart as if it were fresh.
Applied at all THREE sites that must agree - live inference, OOS pass-3 scoring
and the chart renderer. Their own comments say why: an arrow set that does not
obey the same rule as the traded set shows calls the EA would never take.
Also corrects a stale comment that called this window "display only". It is not:
when it suppresses, the live path zeroes the signal outright - no arrow, no vote,
no position. Training never sees it, so these cost no retrain and are correctly
absent from the fingerprint.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 09:48:28 -04:00
void SignalCooldownScope ( SIGNAL_COOLDOWN_SCOPE v ) { m_signalCooldownScope = v ; }
SIGNAL_COOLDOWN_SCOPE SignalCooldownScope ( void ) const { return m_signalCooldownScope ; }
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
int SignalClusterWindow ( void ) const { return m_signalClusterWindow ; }
2026-07-14 22:36:27 -04:00
void SwingConfirmationBars ( int value ) { m_swingConfirmationBars = value ; }
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
int SwingConfirmationBars ( void ) const { return m_swingConfirmationBars ; }
2026-08-22 00:24:45 -04:00
//--- ENSEMBLE MEMBERSHIP (two or more direction NNs enabled on one chart).
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
void EnsembleMember ( bool value , double voteThreshold = -1.0 )
{
m_ensembleMember = value ;
if ( voteThreshold > = 0.0 )
g_ensembleVoteThreshold = voteThreshold ;
if ( value & & m_ensembleIndex < 0 )
{
int n = ArraySize ( g_warriorEnsemble ) ;
ArrayResize ( g_warriorEnsemble , n + 1 ) ;
g_warriorEnsemble [ n ] = GetPointer ( this ) ;
m_ensembleIndex = n ;
}
}
2026-08-22 00:24:45 -04:00
//--- Minimum era among the ensemble members still genuinely training. Falls back to this
//--- member's own era when nothing qualifies, which makes the barrier a no-op rather than a
//--- lock.
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
long EnsembleMinTrainingEra ( void )
{
long minEra = LONG_MAX ;
for ( int i = 0 ; i < ArraySize ( g_warriorEnsemble ) ; i + + )
{
CExpertSignalAIBase * mm = g_warriorEnsemble [ i ] ;
if ( CheckPointer ( mm ) = = POINTER_INVALID )
continue ;
if ( mm . m_trainingComplete | | mm . m_trainingStopRequested | | mm . m_trainingPaused | | ! mm . m_isInitialized )
continue ;
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
//--- THE LIVENESS EXEMPTION (see ENSEMBLE_BARRIER_STUCK_MS). The three flags above are all
2026-08-22 00:24:45 -04:00
//--- VOLUNTARY - a member that chose to stop participating.
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
if ( mm . m_barrierExcluded )
continue ;
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
if ( mm . m_eraCount < minEra )
minEra = mm . m_eraCount ;
}
return ( minEra = = LONG_MAX ) ? m_eraCount : minEra ;
}
fix(ensemble): the era barrier read healthy startup work as a dead member
Reported symptom: one member at era 17 while the rest sat at era 2, with
the combined vote never scoring. Two faults compound to produce exactly
that, and neither needs a broken model to trigger.
FIRST - BUSY WAS READ AS STUCK. BarrierEraHeartbeat() decides liveness
from one signal: has m_eraCount changed in the last 12 minutes. But
Train() returns early, before the era loop, for three ONE-TIME phases
that never touch m_eraCount - the label-cache prebuild, the pattern-DB
backfill and the OOS simulation walk - and those are precisely what a
slow topology spends its first many minutes doing. A member grinding
steadily through a prebuild therefore looked identical to a dead one and
was dropped from the barrier at startup, before it had trained a single
era. The constant's own comment states the flawed premise: "comfortably
past the slowest healthy ERA on the deepest chart" - true, and not the
question being asked. Those three branches now call NoteBarrierProgress()
and a chunk of phase work re-arms the watchdog exactly as an era does.
SECOND - EXCLUSION HAD NO BOUND. Once dropped, a member is skipped by
EnsembleMinTrainingEra(). Drop every OTHER member and that loop finds
nothing to take a minimum over, falls through to its `return m_eraCount`
fallback - the CALLER'S own era - and EnsembleEraBarrierHolds() evaluates
`era > era`, false, for everybody. The barrier silently becomes a no-op
and the fastest member runs away unbounded. EnsembleMinEraAnyMember()
now measures against every still-training member, excluded or not, and a
member may lead it by at most ENSEMBLE_MAX_ERA_LEAD eras.
The cap is deliberately a real stop rather than a warning. A
desynchronised ensemble is not a degraded one: the combined-vote score
and the joint checkpoint both require every member on the same era, so
weights trained past the cap can never be certified by any gate. The
hold reports which of the two it is, because the operator's next move
differs - an ordinary barrier hold resolves itself, a lead-cap hold names
a member that needs diagnosing and will not resolve on its own.
Not yet explained: "only one NN listened to the stop command". The panel
now dispatches down the filter tree and reports the count it reached
("training stopped (N model(s))"), so the next run answers that
definitively instead of leaving it to inference.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 09:35:25 -04:00
//--- Minimum era among still-training members, IGNORING the barrier exclusion. This is what the lead
//--- cap measures against, so an excluded member still BOUNDS the ensemble even though it no longer
//--- BLOCKS it - which is the difference between a liveness escape and an unbounded desync.
long EnsembleMinEraAnyMember ( void )
{
long minEra = LONG_MAX ;
for ( int i = 0 ; i < ArraySize ( g_warriorEnsemble ) ; i + + )
{
CExpertSignalAIBase * mm = g_warriorEnsemble [ i ] ;
if ( CheckPointer ( mm ) = = POINTER_INVALID )
continue ;
if ( mm . m_trainingComplete | | mm . m_trainingStopRequested | | mm . m_trainingPaused | | ! mm . m_isInitialized )
continue ;
if ( mm . m_eraCount < minEra )
minEra = mm . m_eraCount ;
}
return ( minEra = = LONG_MAX ) ? m_eraCount : minEra ;
}
2026-08-22 00:24:45 -04:00
//--- How many members are actively consuming training chunks right now: still training AND at
//--- the barrier's minimum era (a member held ABOVE the min declines its calls, so it costs
//--- nothing).
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
int EnsembleActiveTrainers ( void )
{
if ( ! m_ensembleMember )
return 1 ;
long minEra = EnsembleMinTrainingEra ( ) ;
int active = 0 ;
for ( int i = 0 ; i < ArraySize ( g_warriorEnsemble ) ; i + + )
{
CExpertSignalAIBase * mm = g_warriorEnsemble [ i ] ;
if ( CheckPointer ( mm ) = = POINTER_INVALID )
continue ;
if ( mm . m_trainingComplete | | mm . m_trainingStopRequested | | mm . m_trainingPaused | | ! mm . m_isInitialized )
continue ;
if ( mm . m_eraCount < = minEra )
active + + ;
}
return MathMax ( active , 1 ) ;
}
//--- True when this member has finished more eras than the slowest still-training member and must
//--- wait at the era barrier - checked at Train()'s entry (see the barrier note there).
bool EnsembleEraBarrierHolds ( void )
{
if ( ! m_ensembleMember | | m_trainingComplete )
return false ;
fix(ensemble): the era barrier read healthy startup work as a dead member
Reported symptom: one member at era 17 while the rest sat at era 2, with
the combined vote never scoring. Two faults compound to produce exactly
that, and neither needs a broken model to trigger.
FIRST - BUSY WAS READ AS STUCK. BarrierEraHeartbeat() decides liveness
from one signal: has m_eraCount changed in the last 12 minutes. But
Train() returns early, before the era loop, for three ONE-TIME phases
that never touch m_eraCount - the label-cache prebuild, the pattern-DB
backfill and the OOS simulation walk - and those are precisely what a
slow topology spends its first many minutes doing. A member grinding
steadily through a prebuild therefore looked identical to a dead one and
was dropped from the barrier at startup, before it had trained a single
era. The constant's own comment states the flawed premise: "comfortably
past the slowest healthy ERA on the deepest chart" - true, and not the
question being asked. Those three branches now call NoteBarrierProgress()
and a chunk of phase work re-arms the watchdog exactly as an era does.
SECOND - EXCLUSION HAD NO BOUND. Once dropped, a member is skipped by
EnsembleMinTrainingEra(). Drop every OTHER member and that loop finds
nothing to take a minimum over, falls through to its `return m_eraCount`
fallback - the CALLER'S own era - and EnsembleEraBarrierHolds() evaluates
`era > era`, false, for everybody. The barrier silently becomes a no-op
and the fastest member runs away unbounded. EnsembleMinEraAnyMember()
now measures against every still-training member, excluded or not, and a
member may lead it by at most ENSEMBLE_MAX_ERA_LEAD eras.
The cap is deliberately a real stop rather than a warning. A
desynchronised ensemble is not a degraded one: the combined-vote score
and the joint checkpoint both require every member on the same era, so
weights trained past the cap can never be certified by any gate. The
hold reports which of the two it is, because the operator's next move
differs - an ordinary barrier hold resolves itself, a lead-cap hold names
a member that needs diagnosing and will not resolve on its own.
Not yet explained: "only one NN listened to the stop command". The panel
now dispatches down the filter tree and reports the count it reached
("training stopped (N model(s))"), so the next run answers that
definitively instead of leaving it to inference.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 09:35:25 -04:00
//--- The ordinary barrier: ahead of the slowest member that is still in it.
if ( m_eraCount > EnsembleMinTrainingEra ( ) )
return true ;
//--- ...and the backstop for a member that has been EXCLUDED from that minimum. Without this the
//--- exclusion is a licence to run away without limit - see ENSEMBLE_MAX_ERA_LEAD.
return ( m_eraCount - EnsembleMinEraAnyMember ( ) > = ENSEMBLE_MAX_ERA_LEAD ) ;
}
//--- True when the hold above is the LEAD CAP rather than the ordinary barrier, i.e. we are waiting on
//--- a member the barrier has already given up on. Reported differently because the operator's next
//--- move differs: an ordinary hold resolves itself, this one needs the named member diagnosed.
bool EnsembleLeadCapHolds ( void )
{
return ( m_ensembleMember & & ! m_trainingComplete & &
m_eraCount < = EnsembleMinTrainingEra ( ) & &
m_eraCount - EnsembleMinEraAnyMember ( ) > = ENSEMBLE_MAX_ERA_LEAD ) ;
}
//--- A long one-time phase advanced a chunk. Called from the prebuild / backfill / simulation
2026-08-22 00:24:45 -04:00
//--- branches of Train(), which all return before the era loop and so leave m_eraCount untouched
//--- for as long as the phase lasts.
fix(ensemble): the era barrier read healthy startup work as a dead member
Reported symptom: one member at era 17 while the rest sat at era 2, with
the combined vote never scoring. Two faults compound to produce exactly
that, and neither needs a broken model to trigger.
FIRST - BUSY WAS READ AS STUCK. BarrierEraHeartbeat() decides liveness
from one signal: has m_eraCount changed in the last 12 minutes. But
Train() returns early, before the era loop, for three ONE-TIME phases
that never touch m_eraCount - the label-cache prebuild, the pattern-DB
backfill and the OOS simulation walk - and those are precisely what a
slow topology spends its first many minutes doing. A member grinding
steadily through a prebuild therefore looked identical to a dead one and
was dropped from the barrier at startup, before it had trained a single
era. The constant's own comment states the flawed premise: "comfortably
past the slowest healthy ERA on the deepest chart" - true, and not the
question being asked. Those three branches now call NoteBarrierProgress()
and a chunk of phase work re-arms the watchdog exactly as an era does.
SECOND - EXCLUSION HAD NO BOUND. Once dropped, a member is skipped by
EnsembleMinTrainingEra(). Drop every OTHER member and that loop finds
nothing to take a minimum over, falls through to its `return m_eraCount`
fallback - the CALLER'S own era - and EnsembleEraBarrierHolds() evaluates
`era > era`, false, for everybody. The barrier silently becomes a no-op
and the fastest member runs away unbounded. EnsembleMinEraAnyMember()
now measures against every still-training member, excluded or not, and a
member may lead it by at most ENSEMBLE_MAX_ERA_LEAD eras.
The cap is deliberately a real stop rather than a warning. A
desynchronised ensemble is not a degraded one: the combined-vote score
and the joint checkpoint both require every member on the same era, so
weights trained past the cap can never be certified by any gate. The
hold reports which of the two it is, because the operator's next move
differs - an ordinary barrier hold resolves itself, a lead-cap hold names
a member that needs diagnosing and will not resolve on its own.
Not yet explained: "only one NN listened to the stop command". The panel
now dispatches down the filter tree and reports the count it reached
("training stopped (N model(s))"), so the next run answers that
definitively instead of leaving it to inference.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 09:35:25 -04:00
void NoteBarrierProgress ( void )
{
m_barrierPhaseProgress = true ;
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
}
2026-08-22 00:24:45 -04:00
//--- Era-advance watchdog for the barrier, kept SEPARATE from m_lastEraCompleteTick on purpose.
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
void BarrierEraHeartbeat ( void )
{
if ( ! m_ensembleMember )
return ;
uint nowTick = GetTickCount ( ) ;
if ( m_barrierEraTick = = 0 | | m_barrierEraSeen ! = m_eraCount )
{
//--- Progress (or the first observation). Rejoining is unconditional and immediate: a member
//--- that just completed an era is by definition not stuck, whatever it was doing before.
if ( m_barrierExcluded )
Print ( ID + " : REJOINING THE ERA BARRIER at era " + IntegerToString ( ( int ) m_eraCount ) +
" - it completed an era, so it is training again. It is behind the rest of the "
" ensemble, which means it now sets the minimum and the others wait for it to catch "
" up. The combined-vote score resumes once every member reports the same era. " ) ;
m_barrierExcluded = false ;
m_barrierEraSeen = m_eraCount ;
m_barrierEraTick = nowTick ;
return ;
}
fix(ensemble): the era barrier read healthy startup work as a dead member
Reported symptom: one member at era 17 while the rest sat at era 2, with
the combined vote never scoring. Two faults compound to produce exactly
that, and neither needs a broken model to trigger.
FIRST - BUSY WAS READ AS STUCK. BarrierEraHeartbeat() decides liveness
from one signal: has m_eraCount changed in the last 12 minutes. But
Train() returns early, before the era loop, for three ONE-TIME phases
that never touch m_eraCount - the label-cache prebuild, the pattern-DB
backfill and the OOS simulation walk - and those are precisely what a
slow topology spends its first many minutes doing. A member grinding
steadily through a prebuild therefore looked identical to a dead one and
was dropped from the barrier at startup, before it had trained a single
era. The constant's own comment states the flawed premise: "comfortably
past the slowest healthy ERA on the deepest chart" - true, and not the
question being asked. Those three branches now call NoteBarrierProgress()
and a chunk of phase work re-arms the watchdog exactly as an era does.
SECOND - EXCLUSION HAD NO BOUND. Once dropped, a member is skipped by
EnsembleMinTrainingEra(). Drop every OTHER member and that loop finds
nothing to take a minimum over, falls through to its `return m_eraCount`
fallback - the CALLER'S own era - and EnsembleEraBarrierHolds() evaluates
`era > era`, false, for everybody. The barrier silently becomes a no-op
and the fastest member runs away unbounded. EnsembleMinEraAnyMember()
now measures against every still-training member, excluded or not, and a
member may lead it by at most ENSEMBLE_MAX_ERA_LEAD eras.
The cap is deliberately a real stop rather than a warning. A
desynchronised ensemble is not a degraded one: the combined-vote score
and the joint checkpoint both require every member on the same era, so
weights trained past the cap can never be certified by any gate. The
hold reports which of the two it is, because the operator's next move
differs - an ordinary barrier hold resolves itself, a lead-cap hold names
a member that needs diagnosing and will not resolve on its own.
Not yet explained: "only one NN listened to the stop command". The panel
now dispatches down the filter tree and reports the count it reached
("training stopped (N model(s))"), so the next run answers that
definitively instead of leaving it to inference.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 09:35:25 -04:00
//--- BUSY IS NOT STUCK. A member grinding through the label prebuild, the DB backfill or the OOS
//--- simulation walk never touches m_eraCount, so the era test above cannot see it working. Any
//--- chunk of those phases counts as progress and re-arms the clock, exactly as an era does.
if ( m_barrierPhaseProgress )
{
m_barrierPhaseProgress = false ;
if ( m_barrierExcluded )
Print ( ID + " : REJOINING THE ERA BARRIER - it is still on era " + IntegerToString ( ( int ) m_eraCount ) +
" but is making progress through a one-time preparation phase, not stuck. " ) ;
m_barrierExcluded = false ;
m_barrierEraTick = nowTick ;
return ;
}
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
//--- Same era as last look. Only a member that is AT the minimum can be the one blocking: a member
//--- ahead of it is not advancing because the barrier is holding it, which is correct behaviour and
//--- must never be mistaken for being stuck.
if ( m_barrierExcluded | | m_eraCount > EnsembleMinTrainingEra ( ) )
return ;
if ( nowTick - m_barrierEraTick < ENSEMBLE_BARRIER_STUCK_MS )
return ;
m_barrierExcluded = true ;
PrintFormat ( " %s: RELEASING THE ERA BARRIER - this member has not completed an era in %.0f minutes "
" (still at era %d) and every other member on this chart has been waiting on it for "
" that entire time. It is excluded from the barrier minimum so the rest can advance; "
" it keeps training and rejoins the moment it finishes an era. READ THE TRAIN STALL "
" LINE ABOVE for why it is not finishing - the barrier only reports that it is stuck, "
" never why. NOTE: while the ensemble is desynchronised the combined-vote OOS score "
" cannot be computed (it scores only bars EVERY member contributed at the same era), "
" so no ensemble verdict will be published until this member catches up. " ,
ID , ( nowTick - m_barrierEraTick ) / 60000.0 , ( int ) m_eraCount ) ;
}
feat(gate): measure what a call was WORTH, not only how often it was right
The ensemble deploy gate certifies PRECISION against a chance rate and has
never known whether a correct call pays for its own spread. Every verdict this
project has recorded - 33% precision against a 14% chance rate, an edge that
clears its exact-binomial bar comfortably - is silent on the one question that
decides whether any of it is tradeable, and the cost boundary is exactly where
several earlier edges died with their precision already believed.
Adds a per-row payoff measurement, taken once per ROW (a chart property, not a
member one) at the same time the label is written:
* forward close move over K = round(SwingLifespanEstimate()) bars,
* the up and down extreme excursions over the same window,
each divided by the bar's own ATR. K is deliberately the label lifespan the
effective-sample-size deflation already uses, so precision and payoff describe
the same window and can be read in one sentence.
POLICY-FREE: no stop, no target, no trailing rule. It measures the SIGNAL, not
a trade-management choice layered on top - exit shaping moves payoff around
without creating any, so mixing the two would hide which was responsible.
Stored unsigned by direction; the sign comes from the vote at verdict time, and
a short's excursions SWAP rather than negate - negating them would report a
short's worst case as a negative best case.
The newest K bars of the OOS slice have no forward window and are dropped from
the tally with their own denominator, never counted as a zero move: that is the
leading-edge trap that made the lag profile's first run a false positive.
The era verdict now prints mean R, MFE and MAE at the certified rung against
the spread in the same ATR units. It GATES NOTHING - wiring a policy to an
unvalidated payoff number is how a measurement becomes a decision before anyone
has checked it.
Build tag payoff-v1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 07:14:15 -04:00
//--- WHAT A CALL ON THIS BAR WAS WORTH, before any exit policy: the forward close move and the
//--- two extreme excursions over the next K bars, each divided by the bar's own ATR so they are
//--- comparable across instruments and across volatility regimes.
//---
2026-08-27 07:19:40 -04:00
//--- SERIES INDEXING: bar 0 is the NEWEST, so forward in time is a SMALLER index. The newest
//--- `horizon` bars of the OOS slice have no forward window at all and report unmeasurable.
//--- See the g_ensVoteFwdR declarations for why this is called at TWO horizons.
int PayoffHorizonShort ( void ) { return PIVOT_LABEL_TOLERANCE_BARS ; }
int PayoffHorizonHold ( void )
{
return PIVOT_LABEL_TOLERANCE_BARS + ( int ) MathRound ( m_topology . SwingLegMedianBars ( ) ) ;
}
void MeasureBarPayoff ( const int barIdx , const int horizonIn , double & fwdR , double & upR ,
double & dnR , bool & haveR )
feat(gate): measure what a call was WORTH, not only how often it was right
The ensemble deploy gate certifies PRECISION against a chance rate and has
never known whether a correct call pays for its own spread. Every verdict this
project has recorded - 33% precision against a 14% chance rate, an edge that
clears its exact-binomial bar comfortably - is silent on the one question that
decides whether any of it is tradeable, and the cost boundary is exactly where
several earlier edges died with their precision already believed.
Adds a per-row payoff measurement, taken once per ROW (a chart property, not a
member one) at the same time the label is written:
* forward close move over K = round(SwingLifespanEstimate()) bars,
* the up and down extreme excursions over the same window,
each divided by the bar's own ATR. K is deliberately the label lifespan the
effective-sample-size deflation already uses, so precision and payoff describe
the same window and can be read in one sentence.
POLICY-FREE: no stop, no target, no trailing rule. It measures the SIGNAL, not
a trade-management choice layered on top - exit shaping moves payoff around
without creating any, so mixing the two would hide which was responsible.
Stored unsigned by direction; the sign comes from the vote at verdict time, and
a short's excursions SWAP rather than negate - negating them would report a
short's worst case as a negative best case.
The newest K bars of the OOS slice have no forward window and are dropped from
the tally with their own denominator, never counted as a zero move: that is the
leading-edge trap that made the lag profile's first run a false positive.
The era verdict now prints mean R, MFE and MAE at the certified rung against
the spread in the same ATR units. It GATES NOTHING - wiring a policy to an
unvalidated payoff number is how a measurement becomes a decision before anyone
has checked it.
Build tag payoff-v1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 07:14:15 -04:00
{
fwdR = upR = dnR = 0.0 ;
haveR = false ;
2026-08-27 07:19:40 -04:00
int horizon = horizonIn ;
feat(gate): measure what a call was WORTH, not only how often it was right
The ensemble deploy gate certifies PRECISION against a chance rate and has
never known whether a correct call pays for its own spread. Every verdict this
project has recorded - 33% precision against a 14% chance rate, an edge that
clears its exact-binomial bar comfortably - is silent on the one question that
decides whether any of it is tradeable, and the cost boundary is exactly where
several earlier edges died with their precision already believed.
Adds a per-row payoff measurement, taken once per ROW (a chart property, not a
member one) at the same time the label is written:
* forward close move over K = round(SwingLifespanEstimate()) bars,
* the up and down extreme excursions over the same window,
each divided by the bar's own ATR. K is deliberately the label lifespan the
effective-sample-size deflation already uses, so precision and payoff describe
the same window and can be read in one sentence.
POLICY-FREE: no stop, no target, no trailing rule. It measures the SIGNAL, not
a trade-management choice layered on top - exit shaping moves payoff around
without creating any, so mixing the two would hide which was responsible.
Stored unsigned by direction; the sign comes from the vote at verdict time, and
a short's excursions SWAP rather than negate - negating them would report a
short's worst case as a negative best case.
The newest K bars of the OOS slice have no forward window and are dropped from
the tally with their own denominator, never counted as a zero move: that is the
leading-edge trap that made the lag profile's first run a false positive.
The era verdict now prints mean R, MFE and MAE at the certified rung against
the spread in the same ATR units. It GATES NOTHING - wiring a policy to an
unvalidated payoff number is how a measurement becomes a decision before anyone
has checked it.
Build tag payoff-v1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 07:14:15 -04:00
if ( horizon < 1 )
horizon = 1 ;
//--- No forward window: the leading edge of the series, not a zero move.
if ( barIdx < horizon )
return ;
double atr = m_ATR . Main ( barIdx ) ;
if ( ! MathIsValidNumber ( atr ) | | atr < = 0.0 )
return ;
double entry = m_Close . GetData ( barIdx ) ;
if ( ! MathIsValidNumber ( entry ) | | entry < = 0.0 )
return ;
double hi = entry , lo = entry ;
for ( int j = barIdx - 1 ; j > = barIdx - horizon ; j - - )
{
double h = m_High . GetData ( j ) , l = m_Low . GetData ( j ) ;
//--- A hole in the window makes the EXTREMES wrong, not merely noisy, so the whole row is
//--- abandoned rather than measured over a shortened span.
if ( ! MathIsValidNumber ( h ) | | ! MathIsValidNumber ( l ) | | h < = 0.0 | | l < = 0.0 )
return ;
if ( h > hi )
hi = h ;
if ( l < lo )
lo = l ;
}
double close = m_Close . GetData ( barIdx - horizon ) ;
if ( ! MathIsValidNumber ( close ) | | close < = 0.0 )
return ;
fwdR = ( close - entry ) / atr ;
upR = ( hi - entry ) / atr ;
dnR = ( entry - lo ) / atr ;
haveR = true ;
}
2026-08-22 00:24:45 -04:00
//--- Pass-3 hook: record the VOTE this member would have cast on this OOS bar into the combined-
//--- vote buffer. signedVote is in live vote units - m_weight x tier pattern weight, signed by
//--- direction (see LiveVoteContribution()) - NOT the raw confidence this used to carry.
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
void EnsembleOosContribute ( const int barIdx , const double signedVote ,
feat(vote): thresholds become confidence percentages, on ONE scale everywhere
User request: "the entry/exit thresholds are manual numbers, I would like
them to be confidence percentages, so the current 20 would be only 20%
confidence in a profitable trade."
WHY 20 WAS EVER SENSIBLE. Under UseDatabaseRanking both factors of a filter's
contribution are win rates: the pattern weight is that pattern's measured win
rate (UpdateSignalsWeights -> ApplyPatternWeight) and m_weight is the filter's
average win rate over its patterns, /100. Dividing the sum by the VOTER COUNT
therefore produced a mean of PRODUCTS of two win rates - a genuinely
60%-accurate filter firing a 60% pattern scored 0.60 x 60 = 36. The number was
never on a probability scale, so its magnitude meant nothing on its own.
Dividing by Sum(m_weight) instead makes it a weighted MEAN of win rates, which
is a win rate: result = Sum(w_i*p_i)/Sum(w_i). Every voter at 60% now reads 60;
MACD's double-divergence pattern (weight 100) voting alone reads 100. m_weight
stops being a discount on the probability and becomes how much a filter's
opinion COUNTS - which is what a module weight should always have been.
Default Min_Vote_Open 20 -> 50: not a tightening, the same bar re-expressed.
ONE SCALE, EVERYWHERE - the part that made this bigger than a rescale. Three
other places compared against a 0..1 softmax confidence and would each have
become a fresh currency mismatch the moment the input changed meaning:
* the AI early-exit route (LiveSignedConfidence vs m_ai_exit_threshold) now
reads m_lastAiVote - the AI filters' own weighted mean, undiluted by the
classic side, which is the only reason that route exists - against the
same m_threshold_close the averaged vote uses. m_ai_exit_threshold is
retired rather than left dangling.
* m_oosDecisionSeries now carries the vote, not the confidence, so the exit
SIMULATION stops modelling a close rule the EA does not run.
* ExitPolicy() clamped anything > 1.0 to zero. Passing the unscaled input
through that would have silently switched vote exits off in the
simulation while live went on running them - found before it shipped;
the bound now tracks the scale.
LiveSignedConfidence() is deliberately untouched and still 0..1: MM sizing,
SL/TP scaling and the intelligent trailing want a model confidence, not a win
rate.
CALIBRATION CAVEAT, stated in the code where the claim is made: this is only a
real probability to the extent the pattern weights are. A pattern with fewer
than MIN_TRADES_FOR_WIN_RATE journaled trades keeps its DEFAULT weight - a
designed prior (25/50/75/100 for the AI tiers), not a measurement. Until the
signal DB fills, "60" means "the designed conviction of the patterns that
fired". Closing that gap is the next commit.
Also corrects VOTE_CLOSE_PRESETS' comment, which documented the two scales
this removes.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:52:08 -04:00
const double voteWeight ,
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
const bool labelBuy , const bool labelSell , const bool dirLabel )
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
{
if ( ! m_ensembleMember | | m_ensembleIndex < 0 | | m_ensembleIndex > = 8 )
return ;
datetime t = m_Time . GetData ( barIdx ) ;
if ( t < = 0 )
return ;
if ( g_ensVoteEra ! = m_eraCount )
{
//--- first contribution of a new era resets the buffer (the era barrier keeps members aligned,
//--- so a mismatched stamp means "previous era's rows", never "a sibling's different era")
g_ensVoteEra = m_eraCount ;
g_ensVoteRows = 0 ;
g_ensVoteDoneMask = 0 ;
ArrayInitialize ( g_ensVoteCursor , 0 ) ;
}
int bit = ( 1 < < m_ensembleIndex ) ;
//--- monotonic cursor first (members scan bars oldest-to-newest, so the match is O(1) amortized),
//--- full wrap-around only when per-member window failures desynchronize the sequences
int row = -1 ;
int start = g_ensVoteCursor [ m_ensembleIndex ] ;
if ( start > g_ensVoteRows )
start = 0 ;
for ( int i = start ; i < g_ensVoteRows ; i + + )
if ( g_ensVoteTime [ i ] = = t ) { row = i ; break ; }
if ( row < 0 )
for ( int i = 0 ; i < start ; i + + )
if ( g_ensVoteTime [ i ] = = t ) { row = i ; break ; }
if ( row < 0 )
{
if ( g_ensVoteRows > = ArraySize ( g_ensVoteTime ) )
{
int cap = g_ensVoteRows + g_ensVoteRows / 2 + 512 ;
ArrayResize ( g_ensVoteTime , cap ) ;
ArrayResize ( g_ensVoteSum , cap ) ;
feat(baselines): combining-weight fit, MLP cross-validation, all-lags correlation
Three ALGLIB additions, all measurement-only and all under the existing
Run_Alglib_Baselines switch.
MinBLEIC COMBINING WEIGHTS. The live ensemble weights each member by its
own pooled holdout win rate - a defensible prior, but not a fit, and
nothing has ever asked what mixture minimises error on the bars the
members disagreed about. Two individually-mediocre members wrong in
different places can beat one individually better, and a per-member win
rate cannot express that because it never looks at them jointly.
Solved on the simplex (w >= 0, sum w = 1), which is exactly what
MinBLEIC is for. Non-negative because a negative weight asserts "trade
the opposite of this member", a claim ~60 effective observations cannot
support. Least squares on the signed outcome rather than precision:
precision is a STEP function of the threshold that no gradient method
can walk, and optimising a smooth proxy for a step decision is how
c3daded put every operating point 14pp underwater - so the result is
reported in BOTH currencies, the SSE it minimised and the directional
hit rate the mixture would actually have scored against the equal mix.
If the second does not improve, the first is noise.
This needed data that did not exist: g_ensVoteSum accumulates member
contributions and the sum destroys the decomposition, while
g_ensVoteVoterMask records only WHETHER a member voted, never what.
g_ensVoteMember[] keeps them unsummed. The live arithmetic is untouched.
ENS_MAX_MEMBERS is 8 and deliberately larger than MAX_AI_SIGNALS (5):
independent caps, over-allocating is free, and matching them would make
this array silently short the day the registry grows - a cap that has
already dropped a member once without saying so.
MLPKFoldCVLBFGS. Every baseline row carries a binomial SE, which is the
sampling error of SCORING a fixed model and says nothing about how much
the FIT moves. One LBFGS run from one random start can land anywhere,
and a baseline that cleared or missed the bar on luck of initialisation
reads exactly like one that did it on merit. 3 folds, because each is a
full retrain. LBFGS not LM - LM builds a Hessian over ~7,700 weights.
CCorr ALL-LAGS PROFILE. Added BESIDE the MI lag profile, not instead:
MI catches nonlinear dependence and is the stronger negative, which is
why it settled the verdict - but its per-lag permutation null limits it
to ~20 lags. FFT correlation gets every lag in one O(n log n) pass, so
linear structure parked at lag 300 would surface for free. Different
question, not a replacement. Walks CONTIGUOUS bars, unlike everything
else in this file, because a lag index is meaningless otherwise; both
series are mean-centred first since CorrR1D is a raw sum of products;
and the max over columns x lags is judged against a Sidak family of
exactly that size, not a bare 2-sigma line.
fasttransforms.mqh needed its own include - verified that none of
ap/optimization/statistics/solvers/linalg reaches it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 07:44:36 -04:00
ArrayResize ( g_ensVoteMember , cap * ENS_MAX_MEMBERS ) ;
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
ArrayResize ( g_ensVoteMask , cap ) ;
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
ArrayResize ( g_ensVoteVoterMask , cap ) ;
feat(vote): thresholds become confidence percentages, on ONE scale everywhere
User request: "the entry/exit thresholds are manual numbers, I would like
them to be confidence percentages, so the current 20 would be only 20%
confidence in a profitable trade."
WHY 20 WAS EVER SENSIBLE. Under UseDatabaseRanking both factors of a filter's
contribution are win rates: the pattern weight is that pattern's measured win
rate (UpdateSignalsWeights -> ApplyPatternWeight) and m_weight is the filter's
average win rate over its patterns, /100. Dividing the sum by the VOTER COUNT
therefore produced a mean of PRODUCTS of two win rates - a genuinely
60%-accurate filter firing a 60% pattern scored 0.60 x 60 = 36. The number was
never on a probability scale, so its magnitude meant nothing on its own.
Dividing by Sum(m_weight) instead makes it a weighted MEAN of win rates, which
is a win rate: result = Sum(w_i*p_i)/Sum(w_i). Every voter at 60% now reads 60;
MACD's double-divergence pattern (weight 100) voting alone reads 100. m_weight
stops being a discount on the probability and becomes how much a filter's
opinion COUNTS - which is what a module weight should always have been.
Default Min_Vote_Open 20 -> 50: not a tightening, the same bar re-expressed.
ONE SCALE, EVERYWHERE - the part that made this bigger than a rescale. Three
other places compared against a 0..1 softmax confidence and would each have
become a fresh currency mismatch the moment the input changed meaning:
* the AI early-exit route (LiveSignedConfidence vs m_ai_exit_threshold) now
reads m_lastAiVote - the AI filters' own weighted mean, undiluted by the
classic side, which is the only reason that route exists - against the
same m_threshold_close the averaged vote uses. m_ai_exit_threshold is
retired rather than left dangling.
* m_oosDecisionSeries now carries the vote, not the confidence, so the exit
SIMULATION stops modelling a close rule the EA does not run.
* ExitPolicy() clamped anything > 1.0 to zero. Passing the unscaled input
through that would have silently switched vote exits off in the
simulation while live went on running them - found before it shipped;
the bound now tracks the scale.
LiveSignedConfidence() is deliberately untouched and still 0..1: MM sizing,
SL/TP scaling and the intelligent trailing want a model confidence, not a win
rate.
CALIBRATION CAVEAT, stated in the code where the claim is made: this is only a
real probability to the extent the pattern weights are. A pattern with fewer
than MIN_TRADES_FOR_WIN_RATE journaled trades keeps its DEFAULT weight - a
designed prior (25/50/75/100 for the AI tiers), not a measurement. Until the
signal DB fills, "60" means "the designed conviction of the patterns that
fired". Closing that gap is the next commit.
Also corrects VOTE_CLOSE_PRESETS' comment, which documented the two scales
this removes.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:52:08 -04:00
ArrayResize ( g_ensVoteWeightSum , cap ) ;
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
ArrayResize ( g_ensVoteLabelBuy , cap ) ;
ArrayResize ( g_ensVoteLabelSell , cap ) ;
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
ArrayResize ( g_ensVoteDirLabel , cap ) ;
feat(gate): measure what a call was WORTH, not only how often it was right
The ensemble deploy gate certifies PRECISION against a chance rate and has
never known whether a correct call pays for its own spread. Every verdict this
project has recorded - 33% precision against a 14% chance rate, an edge that
clears its exact-binomial bar comfortably - is silent on the one question that
decides whether any of it is tradeable, and the cost boundary is exactly where
several earlier edges died with their precision already believed.
Adds a per-row payoff measurement, taken once per ROW (a chart property, not a
member one) at the same time the label is written:
* forward close move over K = round(SwingLifespanEstimate()) bars,
* the up and down extreme excursions over the same window,
each divided by the bar's own ATR. K is deliberately the label lifespan the
effective-sample-size deflation already uses, so precision and payoff describe
the same window and can be read in one sentence.
POLICY-FREE: no stop, no target, no trailing rule. It measures the SIGNAL, not
a trade-management choice layered on top - exit shaping moves payoff around
without creating any, so mixing the two would hide which was responsible.
Stored unsigned by direction; the sign comes from the vote at verdict time, and
a short's excursions SWAP rather than negate - negating them would report a
short's worst case as a negative best case.
The newest K bars of the OOS slice have no forward window and are dropped from
the tally with their own denominator, never counted as a zero move: that is the
leading-edge trap that made the lag profile's first run a false positive.
The era verdict now prints mean R, MFE and MAE at the certified rung against
the spread in the same ATR units. It GATES NOTHING - wiring a policy to an
unvalidated payoff number is how a measurement becomes a decision before anyone
has checked it.
Build tag payoff-v1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 07:14:15 -04:00
ArrayResize ( g_ensVoteFwdR , cap ) ;
ArrayResize ( g_ensVoteUpR , cap ) ;
ArrayResize ( g_ensVoteDnR , cap ) ;
ArrayResize ( g_ensVoteHasR , cap ) ;
2026-08-27 07:19:40 -04:00
ArrayResize ( g_ensVoteFwdR2 , cap ) ;
ArrayResize ( g_ensVoteUpR2 , cap ) ;
ArrayResize ( g_ensVoteDnR2 , cap ) ;
ArrayResize ( g_ensVoteHasR2 , cap ) ;
diag(label): expose bars-to-pivot - the confound the payoff test was missing
CORRECTION to what the payoff instrument was measuring. The 5-bar horizon looked
like the powered test and it is confounded.
SwingPivotDirectionLabel returns Buy when a swing LOW lands up to
PIVOT_LABEL_TOLERANCE_BARS bars AHEAD, and says the quiet part itself: gating on
where the pivot sits relative to entry "would drop exactly the bars where the
turn has not finished coming to us", and how much adverse move remains before
the turn "is a trade-management question".
So on a CORRECT Buy call price is often still falling for d more bars. A window
shorter than d measures the APPROACH, not the leg, and its negative contribution
is expected on the calls that are RIGHT. The tight null at 5 bars
(-0.012 +/- 0.074) is therefore not evidence of no payoff. Neither horizon is
both clean and powered: 5 bars is powered and confounded, 18-19 is clean and has
an SE of 0.277.
(idx - P1) was computed in the label and thrown away. Now cached beside
m_labelResolveAge under the same validity flag, and bucketed in the era verdict.
DELIBERATELY NOT USED AS A PER-CALL HORIZON, which is the trap sitting right
next to this: d exists only on bars the label found a pivot for, so a horizon
that varied with d would hand correct and incorrect calls different windows and
bias the comparison outright. The horizon stays fixed; d only buckets.
The bucket for "the label called no pivot here" is reported by name rather than
folded in, because it is the control the others are read against. Buckets 1..N
condition on the label, so they describe the MECHANISM, not what a book earns.
Reads: rising with d means the edge is in EARLY calls and the tolerance window
is spending it - fixable by reweighting the loss, not by a new label. Flat means
that hypothesis dies.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 08:16:47 -04:00
ArrayResize ( g_ensVoteD , cap ) ;
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
}
row = g_ensVoteRows + + ;
g_ensVoteTime [ row ] = t ;
g_ensVoteSum [ row ] = 0.0 ;
feat(baselines): combining-weight fit, MLP cross-validation, all-lags correlation
Three ALGLIB additions, all measurement-only and all under the existing
Run_Alglib_Baselines switch.
MinBLEIC COMBINING WEIGHTS. The live ensemble weights each member by its
own pooled holdout win rate - a defensible prior, but not a fit, and
nothing has ever asked what mixture minimises error on the bars the
members disagreed about. Two individually-mediocre members wrong in
different places can beat one individually better, and a per-member win
rate cannot express that because it never looks at them jointly.
Solved on the simplex (w >= 0, sum w = 1), which is exactly what
MinBLEIC is for. Non-negative because a negative weight asserts "trade
the opposite of this member", a claim ~60 effective observations cannot
support. Least squares on the signed outcome rather than precision:
precision is a STEP function of the threshold that no gradient method
can walk, and optimising a smooth proxy for a step decision is how
c3daded put every operating point 14pp underwater - so the result is
reported in BOTH currencies, the SSE it minimised and the directional
hit rate the mixture would actually have scored against the equal mix.
If the second does not improve, the first is noise.
This needed data that did not exist: g_ensVoteSum accumulates member
contributions and the sum destroys the decomposition, while
g_ensVoteVoterMask records only WHETHER a member voted, never what.
g_ensVoteMember[] keeps them unsummed. The live arithmetic is untouched.
ENS_MAX_MEMBERS is 8 and deliberately larger than MAX_AI_SIGNALS (5):
independent caps, over-allocating is free, and matching them would make
this array silently short the day the registry grows - a cap that has
already dropped a member once without saying so.
MLPKFoldCVLBFGS. Every baseline row carries a binomial SE, which is the
sampling error of SCORING a fixed model and says nothing about how much
the FIT moves. One LBFGS run from one random start can land anywhere,
and a baseline that cleared or missed the bar on luck of initialisation
reads exactly like one that did it on merit. 3 folds, because each is a
full retrain. LBFGS not LM - LM builds a Hessian over ~7,700 weights.
CCorr ALL-LAGS PROFILE. Added BESIDE the MI lag profile, not instead:
MI catches nonlinear dependence and is the stronger negative, which is
why it settled the verdict - but its per-lag permutation null limits it
to ~20 lags. FFT correlation gets every lag in one O(n log n) pass, so
linear structure parked at lag 300 would surface for free. Different
question, not a replacement. Walks CONTIGUOUS bars, unlike everything
else in this file, because a lag index is meaningless otherwise; both
series are mean-centred first since CorrR1D is a raw sum of products;
and the max over columns x lags is judged against a Sidak family of
exactly that size, not a bare 2-sigma line.
fasttransforms.mqh needed its own include - verified that none of
ap/optimization/statistics/solvers/linalg reaches it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 07:44:36 -04:00
for ( int mm = 0 ; mm < ENS_MAX_MEMBERS ; mm + + )
g_ensVoteMember [ row * ENS_MAX_MEMBERS + mm ] = 0.0 ;
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
g_ensVoteMask [ row ] = 0 ;
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
g_ensVoteVoterMask [ row ] = 0 ;
feat(vote): thresholds become confidence percentages, on ONE scale everywhere
User request: "the entry/exit thresholds are manual numbers, I would like
them to be confidence percentages, so the current 20 would be only 20%
confidence in a profitable trade."
WHY 20 WAS EVER SENSIBLE. Under UseDatabaseRanking both factors of a filter's
contribution are win rates: the pattern weight is that pattern's measured win
rate (UpdateSignalsWeights -> ApplyPatternWeight) and m_weight is the filter's
average win rate over its patterns, /100. Dividing the sum by the VOTER COUNT
therefore produced a mean of PRODUCTS of two win rates - a genuinely
60%-accurate filter firing a 60% pattern scored 0.60 x 60 = 36. The number was
never on a probability scale, so its magnitude meant nothing on its own.
Dividing by Sum(m_weight) instead makes it a weighted MEAN of win rates, which
is a win rate: result = Sum(w_i*p_i)/Sum(w_i). Every voter at 60% now reads 60;
MACD's double-divergence pattern (weight 100) voting alone reads 100. m_weight
stops being a discount on the probability and becomes how much a filter's
opinion COUNTS - which is what a module weight should always have been.
Default Min_Vote_Open 20 -> 50: not a tightening, the same bar re-expressed.
ONE SCALE, EVERYWHERE - the part that made this bigger than a rescale. Three
other places compared against a 0..1 softmax confidence and would each have
become a fresh currency mismatch the moment the input changed meaning:
* the AI early-exit route (LiveSignedConfidence vs m_ai_exit_threshold) now
reads m_lastAiVote - the AI filters' own weighted mean, undiluted by the
classic side, which is the only reason that route exists - against the
same m_threshold_close the averaged vote uses. m_ai_exit_threshold is
retired rather than left dangling.
* m_oosDecisionSeries now carries the vote, not the confidence, so the exit
SIMULATION stops modelling a close rule the EA does not run.
* ExitPolicy() clamped anything > 1.0 to zero. Passing the unscaled input
through that would have silently switched vote exits off in the
simulation while live went on running them - found before it shipped;
the bound now tracks the scale.
LiveSignedConfidence() is deliberately untouched and still 0..1: MM sizing,
SL/TP scaling and the intelligent trailing want a model confidence, not a win
rate.
CALIBRATION CAVEAT, stated in the code where the claim is made: this is only a
real probability to the extent the pattern weights are. A pattern with fewer
than MIN_TRADES_FOR_WIN_RATE journaled trades keeps its DEFAULT weight - a
designed prior (25/50/75/100 for the AI tiers), not a measurement. Until the
signal DB fills, "60" means "the designed conviction of the patterns that
fired". Closing that gap is the next commit.
Also corrects VOTE_CLOSE_PRESETS' comment, which documented the two scales
this removes.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:52:08 -04:00
g_ensVoteWeightSum [ row ] = 0.0 ;
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- the label comes from the shared label cache, so it is identical across members -
//--- whichever member reaches the bar first writes it
g_ensVoteLabelBuy [ row ] = labelBuy ;
g_ensVoteLabelSell [ row ] = labelSell ;
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
g_ensVoteDirLabel [ row ] = dirLabel ;
feat(gate): measure what a call was WORTH, not only how often it was right
The ensemble deploy gate certifies PRECISION against a chance rate and has
never known whether a correct call pays for its own spread. Every verdict this
project has recorded - 33% precision against a 14% chance rate, an edge that
clears its exact-binomial bar comfortably - is silent on the one question that
decides whether any of it is tradeable, and the cost boundary is exactly where
several earlier edges died with their precision already believed.
Adds a per-row payoff measurement, taken once per ROW (a chart property, not a
member one) at the same time the label is written:
* forward close move over K = round(SwingLifespanEstimate()) bars,
* the up and down extreme excursions over the same window,
each divided by the bar's own ATR. K is deliberately the label lifespan the
effective-sample-size deflation already uses, so precision and payoff describe
the same window and can be read in one sentence.
POLICY-FREE: no stop, no target, no trailing rule. It measures the SIGNAL, not
a trade-management choice layered on top - exit shaping moves payoff around
without creating any, so mixing the two would hide which was responsible.
Stored unsigned by direction; the sign comes from the vote at verdict time, and
a short's excursions SWAP rather than negate - negating them would report a
short's worst case as a negative best case.
The newest K bars of the OOS slice have no forward window and are dropped from
the tally with their own denominator, never counted as a zero move: that is the
leading-edge trap that made the lag profile's first run a false positive.
The era verdict now prints mean R, MFE and MAE at the certified rung against
the spread in the same ATR units. It GATES NOTHING - wiring a policy to an
unvalidated payoff number is how a measurement becomes a decision before anyone
has checked it.
Build tag payoff-v1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 07:14:15 -04:00
//--- Same "whichever member reaches the bar first writes it" rule as the label above: the
//--- forward excursion is a property of the CHART, identical for every member, so measuring
//--- it once per ROW rather than once per member per row keeps it off the per-member path.
diag(label): expose bars-to-pivot - the confound the payoff test was missing
CORRECTION to what the payoff instrument was measuring. The 5-bar horizon looked
like the powered test and it is confounded.
SwingPivotDirectionLabel returns Buy when a swing LOW lands up to
PIVOT_LABEL_TOLERANCE_BARS bars AHEAD, and says the quiet part itself: gating on
where the pivot sits relative to entry "would drop exactly the bars where the
turn has not finished coming to us", and how much adverse move remains before
the turn "is a trade-management question".
So on a CORRECT Buy call price is often still falling for d more bars. A window
shorter than d measures the APPROACH, not the leg, and its negative contribution
is expected on the calls that are RIGHT. The tight null at 5 bars
(-0.012 +/- 0.074) is therefore not evidence of no payoff. Neither horizon is
both clean and powered: 5 bars is powered and confounded, 18-19 is clean and has
an SE of 0.277.
(idx - P1) was computed in the label and thrown away. Now cached beside
m_labelResolveAge under the same validity flag, and bucketed in the era verdict.
DELIBERATELY NOT USED AS A PER-CALL HORIZON, which is the trap sitting right
next to this: d exists only on bars the label found a pivot for, so a horizon
that varied with d would hand correct and incorrect calls different windows and
bias the comparison outright. The horizon stays fixed; d only buckets.
The bucket for "the label called no pivot here" is reported by name rather than
folded in, because it is the control the others are read against. Buckets 1..N
condition on the label, so they describe the MECHANISM, not what a book earns.
Reads: rising with d means the edge is in EARLY calls and the tolerance window
is spending it - fixable by reweighting the loss, not by a new label. Flat means
that hypothesis dies.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 08:16:47 -04:00
g_ensVoteD [ row ] = LabelBarsToPivot ( barIdx ) ;
2026-08-27 07:19:40 -04:00
MeasureBarPayoff ( barIdx , PayoffHorizonShort ( ) ,
g_ensVoteFwdR [ row ] , g_ensVoteUpR [ row ] , g_ensVoteDnR [ row ] , g_ensVoteHasR [ row ] ) ;
MeasureBarPayoff ( barIdx , PayoffHorizonHold ( ) ,
g_ensVoteFwdR2 [ row ] , g_ensVoteUpR2 [ row ] , g_ensVoteDnR2 [ row ] , g_ensVoteHasR2 [ row ] ) ;
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
}
if ( ( g_ensVoteMask [ row ] & bit ) ! = 0 )
return ; // already contributed to this bar this era (defensive - a re-run must not double-count)
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
g_ensVoteSum [ row ] + = signedVote ;
feat(baselines): combining-weight fit, MLP cross-validation, all-lags correlation
Three ALGLIB additions, all measurement-only and all under the existing
Run_Alglib_Baselines switch.
MinBLEIC COMBINING WEIGHTS. The live ensemble weights each member by its
own pooled holdout win rate - a defensible prior, but not a fit, and
nothing has ever asked what mixture minimises error on the bars the
members disagreed about. Two individually-mediocre members wrong in
different places can beat one individually better, and a per-member win
rate cannot express that because it never looks at them jointly.
Solved on the simplex (w >= 0, sum w = 1), which is exactly what
MinBLEIC is for. Non-negative because a negative weight asserts "trade
the opposite of this member", a claim ~60 effective observations cannot
support. Least squares on the signed outcome rather than precision:
precision is a STEP function of the threshold that no gradient method
can walk, and optimising a smooth proxy for a step decision is how
c3daded put every operating point 14pp underwater - so the result is
reported in BOTH currencies, the SSE it minimised and the directional
hit rate the mixture would actually have scored against the equal mix.
If the second does not improve, the first is noise.
This needed data that did not exist: g_ensVoteSum accumulates member
contributions and the sum destroys the decomposition, while
g_ensVoteVoterMask records only WHETHER a member voted, never what.
g_ensVoteMember[] keeps them unsummed. The live arithmetic is untouched.
ENS_MAX_MEMBERS is 8 and deliberately larger than MAX_AI_SIGNALS (5):
independent caps, over-allocating is free, and matching them would make
this array silently short the day the registry grows - a cap that has
already dropped a member once without saying so.
MLPKFoldCVLBFGS. Every baseline row carries a binomial SE, which is the
sampling error of SCORING a fixed model and says nothing about how much
the FIT moves. One LBFGS run from one random start can land anywhere,
and a baseline that cleared or missed the bar on luck of initialisation
reads exactly like one that did it on merit. 3 folds, because each is a
full retrain. LBFGS not LM - LM builds a Hessian over ~7,700 weights.
CCorr ALL-LAGS PROFILE. Added BESIDE the MI lag profile, not instead:
MI catches nonlinear dependence and is the stronger negative, which is
why it settled the verdict - but its per-lag permutation null limits it
to ~20 lags. FFT correlation gets every lag in one O(n log n) pass, so
linear structure parked at lag 300 would surface for free. Different
question, not a replacement. Walks CONTIGUOUS bars, unlike everything
else in this file, because a lag index is meaningless otherwise; both
series are mean-centred first since CorrR1D is a raw sum of products;
and the max over columns x lags is judged against a Sidak family of
exactly that size, not a bare 2-sigma line.
fasttransforms.mqh needed its own include - verified that none of
ap/optimization/statistics/solvers/linalg reaches it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 07:44:36 -04:00
if ( m_ensembleIndex > = 0 & & m_ensembleIndex < ENS_MAX_MEMBERS )
g_ensVoteMember [ row * ENS_MAX_MEMBERS + m_ensembleIndex ] = signedVote ;
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
g_ensVoteMask [ row ] | = bit ;
2026-08-22 00:24:45 -04:00
//--- VOTER, not merely present. -0.0 compares equal to 0.0, so an abstention that arrived
//--- with a negative zero is still correctly excluded here.
feat(vote): CONSENSUS arithmetic - agreement is now what the threshold dials
Era-680 report, all three observations one equation: "peak 29, no arrows at
threshold 30" / "at 20, arrows on EVERY bar" / "label at 12 while arrows
everywhere". Under the voters-only divisor, any bar with at least one
directional voter read the weighted mean of the firing tiers' weights - and
once the tiers self-ranked to each model's pooled win rate (~28-31), that
mean was NEAR-CONSTANT regardless of headcount. One member alone: ~29. Four
unanimous: ~29. Min_Vote_Open was a step function around that constant -
above it nothing ever fired, below it everything did - and the label's 12
was a 3v1 split netting through the same divisor. Not three display bugs:
one arithmetic that could not express agreement.
The divisor is now the CAPABLE weight - every filter that could vote,
whether it did or not:
* live (Direction): VoteCapableWeight() - classic pattern ladders always,
veto filters never, AI members once past the same readiness test
LongCondition gates on. A model still training must not dilute an
ensemble it cannot join: four trainees + one deployed model is a solo
chart wearing an ensemble label, and the solo vote reads full strength.
* gate (EnsembleEraVerdict): g_ensVoteWeightSum accumulates for every
member that EVALUATED the bar, Neutral included.
* overlay sweep + prospective readout: weight counts whenever the member
has data; a snapshotted Neutral dilutes.
One arithmetic, four sites, same numbers everywhere.
What the numbers become (four members, w~0.29, tiers~29): unanimous ~29 -
the CEILING, which is the pooled win rate and is what the peak displays;
3-of-4 ~22; 2-of-4 ~14.5; 3v1 ~14.5. Min_Vote_Open 20 now means "roughly
three-quarters of the ensemble's trust agrees, net". It MUST sit below the
ceiling to ever fire - the census/peak states the ceiling.
This is the ensemble the user specified in the original design discussion
("if the perceptron also votes, both together reach the threshold; if
another NN votes the other side, the threshold is not reached") - union
semantics was the pre-ensemble behaviour, kept until measurement showed its
vote magnitude was a constant.
Plus overlay DECLUSTERING, the other half of "arrows on every bar": the
same three NMS rules as the per-member arrows (same-direction runs collapse
to their first bar, cross-direction flicker keeps the stronger side), online
over the sweep's strictly oldest->newest walk. Suppression is a verdict and
deletes a standing arrow; the den==0 no-data skip still never does.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 06:58:03 -04:00
g_ensVoteWeightSum [ row ] + = voteWeight ;
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
if ( signedVote ! = 0.0 )
g_ensVoteVoterMask [ row ] | = bit ;
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
g_ensVoteCursor [ m_ensembleIndex ] = row + 1 ;
}
2026-08-22 00:24:45 -04:00
//--- This member's just-finished era, held until the ensemble verdict can act on it. Same
//--- quantities the solo gate keeps in m_best*.
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
double m_eraStatPrecPct ;
double m_eraStatChancePct ;
2026-08-26 16:53:16 -04:00
//--- THE LADDER'S OWN VERDICT ON ITSELF, set by RankTiersFromOos from the SAME pooled holdout the
//--- tier weights come from. It exists because the era pair above is era-only state: a CONVERGED
//--- model runs no eras, so after a restart it had no measurement at all and HasDemonstratedEdge()
//--- ruled it no-skill. This one is written by every path that ranks a ladder - the era end AND the
//--- deployed replay - and is persisted (WST8), so a resumed model knows whether it may vote.
double m_certifiedPrecPct ;
double m_certifiedChancePct ;
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
int m_eraStatCalls ;
bool m_eraStatTradeable ;
bool m_eraStatTwoSided ;
double m_eraStatScore ;
double m_eraStatBlended ;
double m_eraStatThreshold ;
2026-08-22 00:24:45 -04:00
//--- Which era this member's in-memory snapshot belongs to (-1 = none).
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
long m_checkpointEra ;
void EnsembleStashEraStats ( const double precPct , const double chancePct , const int calls ,
const bool tradeable , const bool twoSided , const double score ,
const double blended )
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
{
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
m_eraStatPrecPct = precPct ;
m_eraStatChancePct = chancePct ;
m_eraStatCalls = calls ;
m_eraStatTradeable = tradeable ;
m_eraStatTwoSided = twoSided ;
m_eraStatScore = score ;
m_eraStatBlended = blended ;
//--- the operating point belongs with the weights it was fitted for - see m_bestDirConfThreshold
m_eraStatThreshold = m_dirConfThreshold ;
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
}
2026-08-22 00:24:45 -04:00
//--- Called once per era from the era-end block, after this member's statistics are final.
refactor(dry): one binomial arithmetic for every "is this edge real" test
The formula p(1-p)/n was transcribed nine times across six files - the two
deploy gates, the two edge floors, the collapse recall floor, the barrier
rung ladder, the inference bin SE, the pooled inverse-variance weights and
both detectability reports. System\BinomialStats.mqh now holds it once, as
free functions with no class dependency, so the god-class declaration does
not grow to host pure math.
BinomialVar(p, n) p(1-p)/n
BinomialSEPct(p, n) 100*sqrt(p(1-p)/n)
BinomialCallsForEdge(p, edge, sigmas) the same, solved for n
NormalUpperTailQ(z) Q(z), via Math\Stat\Normal.mqh
SidakFamilyP(z, N) 1-(1-Q(z))^N
Value-preserving by construction: rates go in as probabilities so no call
site gained a *100/100 round-trip, and BinomialSEPct is written through
BinomialVar so the multiply order is the one it replaced. Every degenerate
guard each site carried (p<=0, p>=1, n<=0) now lives in one place and
returns the 0 those sites already treated as "no bar to clear".
CExpertSignalAIBase::NormalUpperTail is gone; NormalUpperTailQ replaces it.
What consolidating SURFACED, and is deliberately NOT changed here: the two
Sidak selection gates compute their SE on the RAW call count, while every
other SE in the project deflates by EffectiveSampleSize() for triple-
barrier label overlap. That makes them the most permissive test in the
codebase, by ~sqrt(mean label lifespan). Correcting it tightens a live
deploy bar, which is a policy decision, not a refactor - flagged in the
code at both sites.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:50:24 -04:00
//--- Defined in Training.mqh - it needs the PLATEAU_* machinery.
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
void EnsembleOosPassComplete ( const long votedEra , double & etaLocal ) ;
//--- The verdict itself, and its pieces. needMask names the members whose reads the vote is built
//--- from (still-training members only - a paused or deployed member is not voting in training).
void EnsembleEraVerdict ( const int needMask , const long votedEra , double & etaLocal ) ;
void EnsembleCommitJointCheckpoint ( const long votedEra ) ;
//--- Does the best combined-vote era survive having been CHOSEN out of g_ensCandidateEras eras?
//--- Identical construction to BestCheckpointSurvivesSelection, applied to the vote.
bool EnsembleSurvivesSelection ( double & zObs , double & pFamily , int & nTried ) ;
2026-08-22 00:24:45 -04:00
//--- Single choke point for this signal's on-chart status text. Every AI-side SetStatusLabel
//--- call site routes through here so no mode can regress into four stacked panels.
2026-08-15 16:54:43 -04:00
void PublishStatus ( const string text , const bool force = false )
{
if ( ! m_ensembleMember )
{
feat(panel): one live vote line, no stale era count, no per-model HUD
Two chart-display fixes reported after watching a converged 4-model
ensemble: the ensemble panel's trailing "(era 69, 4 models,
DEPLOYING)" was frozen at whatever era the ensemble happened to
deploy on, and the separate top-right HUD (one line per model, raw
B/S/N + weight + era + error) was clutter once the vote itself is
what matters.
Root cause of the freeze: g_ensembleVoteLine is written once per era,
at pass-3 completion. A deployed/converged ensemble runs no further
eras (ScheduleTrainingIfNeeded's trainingComplete branch skips
Train() entirely), so that line could never update again - the era
count and "DEPLOYING" marker were permanent set-dressing from the
deploying era, not a live reading.
- EnsembleScoreCombinedVote() drops the era/DEPLOYING tail once
g_ensDeployApproved - nothing left there worth freezing.
- UpdateVoteReadout() (the aggregate "VOTE ..." line, previously its
own top-right chart object) now writes g_liveVoteLine instead of
drawing anything. Both status-label builders - PublishEnsembleStatus
for the ensemble panel, PublishStatus's choke point for the solo
panel - append it as one line, refreshed every tick/timer exactly
as the old HUD was, so the live vote replaces the frozen era tail
in the same visual slot.
- RefreshVoteReadout()'s per-member loop (DisplayHudLine, one
ObjectLabel per model) is deleted outright rather than folded in -
the operator asked for the aggregate only, "without telling me each
individual network".
Follow-on dead-code removal, since DisplayHudLine was the only
caller: the DispProb/DispSignal/MetaGateArmedNow/MetaHasScore/
MetaLastP/MetaLastBe/MetaApproved/MetaVetoed leg of IChartView (and
its AIBaseChartView/AIBaseChartViewImpl/ExpertSignalAIBase forwards)
had no other reader. The underlying data survives untouched -
m_metaTelemetry is still populated live by SignalMETA.mqh,
m_dispSignal still feeds ProspectiveVote - only the chart-view
forwarding that existed solely to reach the deleted HUD is gone.
Compile: 0 errors, 0 warnings (stage).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 22:27:41 -04:00
//--- THE LIVE AGGREGATED VOTE, appended here rather than by every caller of PublishStatus -
//--- see g_liveVoteLine's declaration (ExpertSignalCustom.mqh) for what replaced it.
2026-08-25 22:51:50 -04:00
string full = ( g_liveVoteLine ! = " " ) ? text + " \n " + g_liveVoteLine : text ;
//--- Same two guards PublishEnsembleStatus already applies: unchanged text needs no
//--- relayout, and changed text still respects a minimum redraw interval. SetStatusLabel()
//--- word-wraps, measures and re-sets several chart objects then calls ChartRedraw() - real
//--- work this was paying for on every tick regardless of whether anything visible changed.
uint now = GetTickCount ( ) ;
if ( ! force & & full = = m_soloStatusLastText )
return ;
if ( ! force & & m_soloStatusLastRender ! = 0 & & now - m_soloStatusLastRender < 300 )
return ;
m_soloStatusLastText = full ;
m_soloStatusLastRender = now ;
SetStatusLabel ( full ) ;
2026-08-15 16:54:43 -04:00
return ;
}
2026-08-22 00:24:45 -04:00
//--- Called EVERY publish, not just the first. The old "claim once, first publisher wins the
//--- next free row" form is what ordered the panel by who was busiest instead of by member
//--- index.
fix(indicators+panel): the dead handle is MEASURED now - recreate it; and order the ensemble panel by member, not by who published first
THE ANSWER, off the instrumentation added in be39674, first run:
ConvLSTM [HYB-2484]: TUNABLE INDICATOR REPORTS NO CALCULATED BARS - 1 tunable
indicator(s) enabled and the least-ready answers BarsCalculated()=-1 ...
Per-indicator depth: price=33982 MA=-1 ZigZag=33982 ATR=33982
MA=-1 with price, ZigZag and ATR all at full depth. **The handle is INVALID, not
short.** Same line on USDJPY (price=50179 MA=-1). Depth was never the problem;
the previous session's five theories were all answering the wrong question.
And it is per-member, not per-chart: LSTM-2484 ran the 34-candidate auto-tune on
that same chart at 15:24:18 and went on to train normally (feature health, 51
features, excursion head) reading the same indicator. Only ConvLSTM's handle -
the last member constructed - was dead. WHY is still not established. All four
members request ADMovingAverage with identical params, so MT5 hands them the SAME
refcounted handle, and the tuner's inner loop is Create-then-IndicatorRelease over
exactly that shared handle; that is the obvious suspect and it is NOT yet proven,
so this commit does not act on it.
1. IndicatorDepthReport() NOW PRINTS HANDLE NUMBERS, not just depths.
"MA=-1" says the handle is dead. "MA=-1(h12)" against another member's "MA=33982
(h12)" says it is the SAME handle and someone released it; "(h-1)" says it was
never created. That is the difference between a refcount bug and a creation
failure and it is one field. This is the measurement the shared-handle suspicion
needs before anyone acts on it.
2. RECREATE A DEAD HANDLE INSTEAD OF SWEEPING AGAINST IT.
A member that cannot read its own indicator must rebuild it. RepairDeadIndicatorHandles()
re-Creates only the ENABLED tunables reporting BarsCalculated() < 0 - a merely COLD
indicator (valid handle, 0 bars) is left alone to warm up the normal way. It does
NOT release first: -1 means the terminal no longer knows the handle, so there is
nothing to give back, and MT5 recycles handle VALUES so releasing a stale one could
decrement whatever now owns that number. 30s cooldown, because every ServableBars()
consumer reaches it including live inference on every tick. The feature cache is
dropped with it, and the log names before/after depths.
Cause-agnostic on purpose. Whatever is killing the handle, sweeping 50,163 bars
against a buffer that answers EMPTY_VALUE at every index - then discarding the era
and doing it again - is not a recovery.
3. THE SWEEP NOW HOLDS ON A DEAD HANDLE.
ServableBars() keeps answering `want` (its contract; live inference and online
learning have their own refusal paths and a 0 there reads as "no history at all").
SettledBars() - the training sweep's entry, the one caller that can afford to wait -
returns 0 instead, so Train() holds and reports rather than burning a full-history
pass it is guaranteed to throw away. A recreated handle is cold, so it primes
through the existing settle path on the next call. If the repair fails the member
holds indefinitely and says so every minute, and be39674's barrier liveness escape
releases the rest of the ensemble after 12 minutes - which is the correct
degradation and is exactly what the log shows happening.
4. THE PANEL ROWS WERE ORDERED BY WHO PUBLISHED FIRST.
Reported on XAUUSD: LSTM, ConvLSTM, Perceptron, Convolutional instead of
Perceptron, Convolutional, LSTM, ConvLSTM. ClaimEnsemblePanelSlot() handed out the
next free row on each member's FIRST PublishStatus() call, so the order was a race -
the members busy sweeping published before the ones sitting idle at the era barrier,
and be39674 sharpened it by (correctly) making a held member stop writing the terse
line. Rows are now keyed to m_ensembleIndex, the registration/construction order,
which is fixed for the life of the chart. Claimed on every publish rather than once,
so it is idempotent and refreshes the tag for a member whose ID was not final when it
first published (the config-tag suffix is appended during InitIndicators, after
EnsembleMember() registers). Unclaimed rows are skipped by the render and excluded
from the model count, so a member that has not published yet leaves no gap and shifts
nobody.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:40:00 -04:00
m_ensemblePanelSlot = ClaimEnsemblePanelSlot ( DisplayName ( ) , m_ensembleIndex ) ;
2026-08-15 16:54:43 -04:00
int nl = StringFind ( text , " \n " ) ;
PublishEnsembleStatus ( m_ensemblePanelSlot , ( nl > 0 ) ? StringSubstr ( text , 0 , nl ) : text , force ) ;
}
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
void EnableOnlineLearning ( bool value ) { m_onlineLearning . SetEnabled ( value ) ; }
2026-07-14 22:36:27 -04:00
void MaxErasPerRun ( int value ) { m_maxErasPerRun = value ; }
void OOSSplit ( int value ) { m_oosSplitPct = value ; }
refactor(yagni): drop 13 accessors nothing called; unify the ATR trailing pair
Verified dead by grep across all first-party sources (references/, Scripts/,
research/ excluded): EraCount, HiddenLayersCount, LstmHiddenSize, ConvFilterCount,
HistoryBars and MinTrainYear setters, PendingBatchSamples, getPrevOutIndex,
BaseCurrency, QuoteCurrency, CurrencyCount, IsLoaded, LastFiredDirection,
DBConfidence, SpecIndex, and the conv Step/WindowOut shape accessors. Every
backing member stays - each is still read internally and several are pinned by
the positional .cfg layout - so this removes surface, not behaviour.
Two comments were asserting the opposite of the code and are now true: the
"No setter: the taper's endpoints are derived" note was directly above three
setters, and the conv shape block claimed EnforceTopologyContract reads all
three accessors when CNet::FirstConvWindow only ever calls Window().
CTrailingATR::CheckTrailingStopLong/Short were byte-identical but for Bid vs Ask
and the isLong flag; both now delegate to one CheckTrailingStop body.
Deliberately NOT removed: the fractal-target branch (TrainTargetFractal,
IsFractalTarget and their label machinery). It reads as dead because the
TrainingTarget input was withdrawn, but Warrior_EA.mq5:836 documents it as a
parked option with a three-line restore path - that is a product call, not a
refactor.
Not compiled - MetaEditor compile pending.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 18:55:36 -04:00
//--- No setters for m_historyBars / m_minTrainYear. The window is DERIVED at InitNeuralNetwork or
//--- ADOPTED from the .cfg (see DeriveHistoryBars); the year floor is a constructor constant. Both
//--- remain members only because the .cfg field layout is positional.
2026-07-14 22:36:27 -04:00
void UseVolumes ( bool value ) { m_useVolumes = value ; }
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
bool UseVolumes ( void ) const { return m_useVolumes ; }
2026-07-14 22:36:27 -04:00
void UseTime ( bool value ) { m_useTime = value ; }
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
bool UseTime ( void ) const { return m_useTime ; }
2026-07-14 22:36:27 -04:00
void UseATR ( bool value ) { m_useATR = value ; }
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
bool UseATR ( void ) const { return m_useATR ; }
2026-07-22 17:17:23 -04:00
void UseMA ( bool value ) { m_useMA = value ; }
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
bool UseMA ( void ) const { return m_useMA ; }
2026-07-19 11:04:38 -04:00
void UseSwingContext ( bool value ) { m_useSwingContext = value ; }
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
bool UseSwingContext ( void ) const { return m_useSwingContext ; }
feat: add configurable news event proximity/impact as an NN input feature
Price, time, volume, and volatility were already trained-model input
features; the real economic calendar (already used for the live
NewsFilter veto) is now an optional one too, reusing
System/NewsRelevance.mqh's symbol-relevance logic from the prior fix.
New EnableNews/NewsFeatureWindowMinutes inputs gate two features per
bar: minutes-since and minutes-until the nearest symbol-relevant
calendar event, impact-weighted. Deliberately limited to proximity +
impact, not actual-vs-forecast deviation - release schedules are
public knowledge ahead of time (not lookahead bias to use for a
historical training bar), but a release's actual outcome is not.
Wired identically to the existing EnableVolume/EnableTime/EnableATR
toggles: InitIndicators() accounts for the +2 neuron count,
BufferTempDataCompute() appends the two feature values, PAI/CONV/LSTM
all wired in Warrior_EA.mq5. Compiled clean (MetaEditor, 0 errors/0
warnings).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-18 17:26:04 -04:00
void UseNews ( bool value ) { m_useNews = value ; }
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
bool UseNews ( void ) const { return m_useNews ; }
feat: add configurable news event proximity/impact as an NN input feature
Price, time, volume, and volatility were already trained-model input
features; the real economic calendar (already used for the live
NewsFilter veto) is now an optional one too, reusing
System/NewsRelevance.mqh's symbol-relevance logic from the prior fix.
New EnableNews/NewsFeatureWindowMinutes inputs gate two features per
bar: minutes-since and minutes-until the nearest symbol-relevant
calendar event, impact-weighted. Deliberately limited to proximity +
impact, not actual-vs-forecast deviation - release schedules are
public knowledge ahead of time (not lookahead bias to use for a
historical training bar), but a release's actual outcome is not.
Wired identically to the existing EnableVolume/EnableTime/EnableATR
toggles: InitIndicators() accounts for the +2 neuron count,
BufferTempDataCompute() appends the two feature values, PAI/CONV/LSTM
all wired in Warrior_EA.mq5. Compiled clean (MetaEditor, 0 errors/0
warnings).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-18 17:26:04 -04:00
void NewsFeatureWindowMinutes ( int value ) { m_newsFeatureWindowMinutes = value ; }
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
int NewsFeatureWindowMinutes ( void ) const { return m_newsFeatureWindowMinutes ; }
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
void UseCrossAsset ( bool value ) { m_useCrossAsset = value ; }
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
bool UseCrossAsset ( void ) const { return m_useCrossAsset ; }
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
void UseSpreadFeature ( bool value ) { m_useSpreadFeature = value ; }
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
bool UseSpreadFeature ( void ) const { return m_useSpreadFeature ; }
2026-07-14 22:36:27 -04:00
void AutoTuneIndicators ( bool value ) { m_autoTuneIndicators = value ; }
2026-08-16 15:12:54 -04:00
void UseAltData ( bool value ) { m_altDataEnabled = value ; }
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- TOPOLOGY VIEW published read/write API - see Expert\Topology\ITopologyView.mqh for the
//--- contract these serve. This one is the DERIVED alt-data flag InitFeatureIndicators sets once
//--- the feature width is known - distinct from UseAltData(bool) above, which is the operator's
//--- opt-in switch (m_altDataEnabled).
bool TopologyUseAltData ( void ) const { return m_useAltData ; }
2026-07-14 22:36:27 -04:00
//--- control-panel API (Warrior_EA.mq5): current-config-only training/weights control.
//--- "current config" == this signal instance's own m_fileName (symbol+period+id+topology),
//--- never touches another signal type's or another symbol/timeframe's saved files.
void PauseTraining ( void ) { m_trainingPaused = true ; PrintVerbose ( ID + " : training paused by user (era " + IntegerToString ( m_eraCount ) + " ) " ) ; }
void ResumeTraining ( void ) { m_trainingPaused = false ; PrintVerbose ( ID + " : training resumed by user (era " + IntegerToString ( m_eraCount ) + " ) " ) ; }
bool IsTrainingPaused ( void ) const { return m_trainingPaused ; }
bool IsTrainingStopped ( void ) const { return m_trainingStopRequested ; }
bool TrainingComplete ( void ) const { return m_trainingComplete ; }
fix(deinit): a full model write was running ahead of the cheap cleanup
"Abnormal termination" is back, and this time it is not the arrows. The
timing names the culprit exactly:
16:02:31.547 OnDeinit: shutting down
16:02:36.003 Abnormal termination <- 4.46 s, MetaTrader gave up
16:02:36.226 chart signals - persisted <- cleanup finished 0.2 s LATE
OnDeinit called StopTraining() BEFORE the chart cleanup. StopTraining()
finalises an in-flight run, and FinalizeTrainRun() restores the best
checkpoint and then persists it - a full ~1MB model write per signal. So
the expensive step ran ahead of the cheap bounded one, which is precisely
the inversion the shutdown ordering exists to prevent. The previous fix
put PersistWeightsOnShutdown last and missed that StopTraining smuggles a
second save in at the front.
Two changes:
Cleanup now runs FIRST, then StopTraining, then the weight save. The
visible teardown is cheap and bounded, so it always completes even when
everything after it is killed.
And the deploy-persist inside FinalizeTrainRun is suppressed during
shutdown. RestoreWeights() is an in-MEMORY swap, so the best checkpoint
is already the live net by that line, and PersistWeightsOnShutdown writes
exactly those weights moments later. The old path wrote the same model
twice per signal - eight full writes across four charts - for no benefit.
A user-pressed Stop still persists immediately, because nothing else
would.
Compiles 0 errors / 0 warnings. Build tag deinit-order-v2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 16:06:40 -04:00
//--- Set by OnDeinit before it calls StopTraining(), so FinalizeTrainRun() can tell a user-pressed Stop
//--- (persist the deployed model now - nothing else will) from a shutdown (PersistWeightsOnShutdown is
//--- moments away and writes the same bytes). See the guard in FinalizeTrainRun.
void MarkShutdown ( void ) { m_shutdownInProgress = true ; }
2026-08-22 00:24:45 -04:00
//--- THE ONE QUESTION every long loop in this class must ask: has this program been asked to
//--- stop? These two are terminal - once either is true the program is going away.
fix(shutdown): make ExitPolicy public, and stop every long loop the moment MT5 asks
Two things, one of which was a compile error.
1. ExitPolicy() was declared in the protected block but is pushed in from
Warrior_EA.mq5:770. Moved to public beside the other EA-facing setters.
2. Chart objects surviving OnDeinit. The 4,500 ms teardown budget is measured
from the STOP REQUEST, not from OnDeinit's first line, and OnDeinit cannot
begin until whatever is in flight returns - so a scan still running after
_StopFlag is raised does not delay the cleanup, it SPENDS it, and the purge
never gets its turn.
New CExpertSignalAIBase::ShutdownRequested() = IsStopped() || m_shutdownInProgress.
Deliberately NOT m_trainingStopRequested: that latches, and a latched flag
would permanently disable scans that must run again on the next Start.
Guarded, longest first:
- TuneIndicatorsByFilter - per candidate, restoring the OPERATOR's settings
on the way out (best[] is mutated in place; the tuner otherwise keeps the
last trial's parameters, which nothing chose).
- ReportBarrierGeometryScan - per pairing, breaking to ONE restore point so
m_barrierScanLiveLabels can never be left true (that makes ComputeLabelForBar
read the last candidate's multiples as the configured geometry).
- ReportFeatureLabelInformation / ReportExcursionInformation / lag profile -
nulls ABANDON rather than truncate: fewer draws is not a smaller null, it
is a wrong one, and p shifts toward significance. m_dirEvidence staying
false is the safe direction.
- SimulateExitPolicyOutcomes - zeroes its accumulators so the divergence line
is dropped instead of latching a partial expectancy as the run's only report.
- ReportGeometryExpectancyScan - per ladder rung.
- HttpGet - one choke point for up to a dozen blocking WebRequests per
first-pass Update(). An in-flight request cannot be cancelled; refusing to
start another is the whole remedy.
- PollTraining, OnChartEventHandler's study event, TuneIndicatorsAndTrain -
entry points, so a queued event cannot open an era during teardown.
TuneIndicatorsAndTrain's guard is the first statement, ahead of the
m_tuneFilterDone / g_ensembleChartTuneDone latches.
- OnTick / OnTimer / OnChartEvent.
Training's own bar loops already honoured this (pass 1 per bar, passes 2/2.5/3
yield on a 120 ms budget); the warm-up scans did not, and they are the longest
uninterruptible stretches the EA has.
StopTraining() is unchanged: the operator's Stop still finalises synchronously.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 17:03:11 -04:00
bool ShutdownRequested ( void ) const { return ( IsStopped ( ) | | m_shutdownInProgress ) ; }
refactor(arch): a read-only training-data view, so modules stop being #included code
The AIBase\*.mqh files are not modules. They are method bodies of one
3,400-line class, textually #included after its declaration. Every one
of them can touch every member of every other, which is why "move this
out" has so far meant "move the whole class".
Introduce the seam that ends that:
CTrainingDataView abstract - the ONLY thing a training-side
collaborator may see: a feature row, a label, an
outcome, an excursion, the shape they share, and
the identity to log under.
CAIBaseTrainingData the adapter. MQL5 gives a class exactly one base
and CExpertSignalAIBase is already a
CExpertSignalCustom, so it cannot implement the
view itself. It owns one of these instead.
Data*() on the the published read API the adapter forwards to.
signal MQL5 has no `friend`, so reaching in from outside
was never an option - and making it explicit is
the point rather than a workaround.
Every row accessor OWNS ITS BOUNDS TEST and answers false for a bar it
has nothing for. Thirty-odd call sites currently carry their own
ArraySize() guard; one that forgets reads past a cache that is shorter
than the bar count for the whole warm-up. The -2.0 "never scored"
sentinel on the arrow cache is folded in the same way, so it can no
longer be mistaken for a small confidence.
Nothing uses it yet - this is the seam only, kept as its own commit so
the pattern compiles before 951 lines of Baselines move onto it. The
pattern is the stdlib's own: abstract base with =0 (Canvas\DX\DXObject),
concrete override, forward-declared owner pointer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:53:00 -04:00
//+------------------------------------------------------------------+
//| PUBLISHED READ API for training-side collaborators. |
//| |
//| Everything a baseline, a geometry scan or a redundancy report |
//| needs to see, and nothing else. Before this, such code lived |
//| inside the class purely so it could reach these caches - which is |
//| why a 951-line diagnostic could not be moved, replaced or tested |
//| on its own. Collaborators reach these through CTrainingDataView |
//| and never name this class. |
//| |
//| Each row accessor OWNS ITS BOUNDS TEST and answers false for a |
//| bar it has nothing for. That is deliberate: the callers used to |
//| carry their own ArraySize() guards, and a caller that forgot one |
//| read past the end of a cache that is shorter than the bar count |
//| for the whole warm-up. |
//+------------------------------------------------------------------+
int DataHistoryBars ( void ) const { return ( int ) m_historyBars ; }
int DataFeaturesPerBar ( void ) const { return m_neuronsCount ; }
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
int DataLabelResolutionBars ( void ) const { return LabelResolutionBars ( ) ; }
int DataPurgeBars ( void ) const { return CalibPurgeBars ( ) ; }
refactor(arch): a read-only training-data view, so modules stop being #included code
The AIBase\*.mqh files are not modules. They are method bodies of one
3,400-line class, textually #included after its declaration. Every one
of them can touch every member of every other, which is why "move this
out" has so far meant "move the whole class".
Introduce the seam that ends that:
CTrainingDataView abstract - the ONLY thing a training-side
collaborator may see: a feature row, a label, an
outcome, an excursion, the shape they share, and
the identity to log under.
CAIBaseTrainingData the adapter. MQL5 gives a class exactly one base
and CExpertSignalAIBase is already a
CExpertSignalCustom, so it cannot implement the
view itself. It owns one of these instead.
Data*() on the the published read API the adapter forwards to.
signal MQL5 has no `friend`, so reaching in from outside
was never an option - and making it explicit is
the point rather than a workaround.
Every row accessor OWNS ITS BOUNDS TEST and answers false for a bar it
has nothing for. Thirty-odd call sites currently carry their own
ArraySize() guard; one that forgets reads past a cache that is shorter
than the bar count for the whole warm-up. The -2.0 "never scored"
sentinel on the arrow cache is folded in the same way, so it can no
longer be mistaken for a small confidence.
Nothing uses it yet - this is the seam only, kept as its own commit so
the pattern compiles before 951 lines of Baselines move onto it. The
pattern is the stdlib's own: abstract base with =0 (Canvas\DX\DXObject),
concrete override, forward-declared owner pointer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:53:00 -04:00
int DataCalibrationHiIndex ( const int totalIter , const int oosCutoff )
{ return CalibHiIndex ( totalIter , oosCutoff ) ; }
bool DataHasLabel ( const int bar ) const
{ return ( bar > = 0 & & bar < ArraySize ( m_labelCacheHasValue ) & & m_labelCacheHasValue [ bar ] ) ; }
bool DataIsBuyLabel ( const int bar ) const
{ return ( DataHasLabel ( bar ) & & bar < ArraySize ( m_labelCacheBuy ) & & m_labelCacheBuy [ bar ] ) ; }
bool DataIsSellLabel ( const int bar ) const
{ return ( DataHasLabel ( bar ) & & bar < ArraySize ( m_labelCacheSell ) & & m_labelCacheSell [ bar ] ) ; }
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- Bars-to-resolution of the bar's cached label; 0 for an unresolved bar.
int DataLabelResolveAge ( const int bar ) const
{ return ( DataHasLabel ( bar ) & & bar < ArraySize ( m_labelResolveAge ) ) ? m_labelResolveAge [ bar ] : 0 ; }
diag(label): expose bars-to-pivot - the confound the payoff test was missing
CORRECTION to what the payoff instrument was measuring. The 5-bar horizon looked
like the powered test and it is confounded.
SwingPivotDirectionLabel returns Buy when a swing LOW lands up to
PIVOT_LABEL_TOLERANCE_BARS bars AHEAD, and says the quiet part itself: gating on
where the pivot sits relative to entry "would drop exactly the bars where the
turn has not finished coming to us", and how much adverse move remains before
the turn "is a trade-management question".
So on a CORRECT Buy call price is often still falling for d more bars. A window
shorter than d measures the APPROACH, not the leg, and its negative contribution
is expected on the calls that are RIGHT. The tight null at 5 bars
(-0.012 +/- 0.074) is therefore not evidence of no payoff. Neither horizon is
both clean and powered: 5 bars is powered and confounded, 18-19 is clean and has
an SE of 0.277.
(idx - P1) was computed in the label and thrown away. Now cached beside
m_labelResolveAge under the same validity flag, and bucketed in the era verdict.
DELIBERATELY NOT USED AS A PER-CALL HORIZON, which is the trap sitting right
next to this: d exists only on bars the label found a pivot for, so a horizon
that varied with d would hand correct and incorrect calls different windows and
bias the comparison outright. The horizon stays fixed; d only buckets.
The bucket for "the label called no pivot here" is reported by name rather than
folded in, because it is the control the others are read against. Buckets 1..N
condition on the label, so they describe the MECHANISM, not what a book earns.
Reads: rising with d means the edge is in EARLY calls and the tolerance window
is spending it - fixable by reweighting the loss, not by a new label. Flat means
that hypothesis dies.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 08:16:47 -04:00
//--- -1 means "this bar calls no pivot" - either Neutral, or not resolved. Never 0-as-unknown:
//--- 0 is a REAL value here (the pivot is on the very next bar) and the two must not collide.
int LabelBarsToPivot ( const int bar ) const
{
return ( DataHasLabel ( bar ) & & bar < ArraySize ( m_labelBarsToPivot ) ) ? m_labelBarsToPivot [ bar ] : -1 ;
}
refactor(arch): a read-only training-data view, so modules stop being #included code
The AIBase\*.mqh files are not modules. They are method bodies of one
3,400-line class, textually #included after its declaration. Every one
of them can touch every member of every other, which is why "move this
out" has so far meant "move the whole class".
Introduce the seam that ends that:
CTrainingDataView abstract - the ONLY thing a training-side
collaborator may see: a feature row, a label, an
outcome, an excursion, the shape they share, and
the identity to log under.
CAIBaseTrainingData the adapter. MQL5 gives a class exactly one base
and CExpertSignalAIBase is already a
CExpertSignalCustom, so it cannot implement the
view itself. It owns one of these instead.
Data*() on the the published read API the adapter forwards to.
signal MQL5 has no `friend`, so reaching in from outside
was never an option - and making it explicit is
the point rather than a workaround.
Every row accessor OWNS ITS BOUNDS TEST and answers false for a bar it
has nothing for. Thirty-odd call sites currently carry their own
ArraySize() guard; one that forgets reads past a cache that is shorter
than the bar count for the whole warm-up. The -2.0 "never scored"
sentinel on the arrow cache is folded in the same way, so it can no
longer be mistaken for a small confidence.
Nothing uses it yet - this is the seam only, kept as its own commit so
the pattern compiles before 951 lines of Baselines move onto it. The
pattern is the stdlib's own: abstract base with =0 (Canvas\DX\DXObject),
concrete override, forward-declared owner pointer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:53:00 -04:00
//--- -2.0 is the "never scored" SENTINEL, not a small confidence - see m_arrowSignalCache.
refactor(baselines): the first real module - a class, not an #included partial
Baselines was 951 lines of CExpertSignalAIBase method bodies in a file
that only looked like a module. It is now CBaselineComparator: a class
the signal OWNS, which reads a CTrainingDataView and prints. It does
not name the signal anywhere in its code.
What the seam forced out into the open:
- Thirty-odd ArraySize() bounds tests, each carried by its caller, are
now one test per accessor next to the data. The two `hasValueN` and
one `arrowN` locals are gone with them.
- The -2.0 "never scored" sentinel on the arrow cache was tested at the
call site. It is now inside DataDirectionalCall, where it cannot be
read as a small confidence.
- DoubleToSignal needs m_outputNeuronsCount, so a raw double could not
be turned into a side by any reader. The view answers
DirectionalCall(bar, isBuy, magnitude) instead - the conversion
happens where the head width lives, and the module no longer needs
ENUM_SIGNAL at all.
- m_baselineDone was a latch on the signal for a decision only this
module makes. It is m_done, private, where it belongs.
Correction to my own earlier claim: I said Baselines had nine exclusive
members "polluting the signal class". It had none. m_x, m_f, m_ngrad,
m_AvgCE and the rest are FIELDS OF ALGLIB REPORT OBJECTS (state.m_x,
mrep.m_AvgCE) that my `\bm_\w+` scan matched after the dot. The module
needs no private state but its view pointer and that latch - which is
why it came out this cleanly.
The include sits below the g_ens* vote globals and the Alglib headers
it reads, because unlike the AIBase\*.mqh partials this is a real class
declaration compiled where it stands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 12:00:18 -04:00
//--- The threshold that turns a raw value into a side depends on the head's output width, which
//--- is why this conversion belongs here and not in whatever is reading. A bar the model called
//--- Neutral answers false, exactly like a bar it never scored: neither is a directional call.
bool DataDirectionalCall ( const int bar , bool & isBuy , double & magnitude )
refactor(arch): a read-only training-data view, so modules stop being #included code
The AIBase\*.mqh files are not modules. They are method bodies of one
3,400-line class, textually #included after its declaration. Every one
of them can touch every member of every other, which is why "move this
out" has so far meant "move the whole class".
Introduce the seam that ends that:
CTrainingDataView abstract - the ONLY thing a training-side
collaborator may see: a feature row, a label, an
outcome, an excursion, the shape they share, and
the identity to log under.
CAIBaseTrainingData the adapter. MQL5 gives a class exactly one base
and CExpertSignalAIBase is already a
CExpertSignalCustom, so it cannot implement the
view itself. It owns one of these instead.
Data*() on the the published read API the adapter forwards to.
signal MQL5 has no `friend`, so reaching in from outside
was never an option - and making it explicit is
the point rather than a workaround.
Every row accessor OWNS ITS BOUNDS TEST and answers false for a bar it
has nothing for. Thirty-odd call sites currently carry their own
ArraySize() guard; one that forgets reads past a cache that is shorter
than the bar count for the whole warm-up. The -2.0 "never scored"
sentinel on the arrow cache is folded in the same way, so it can no
longer be mistaken for a small confidence.
Nothing uses it yet - this is the seam only, kept as its own commit so
the pattern compiles before 951 lines of Baselines move onto it. The
pattern is the stdlib's own: abstract base with =0 (Canvas\DX\DXObject),
concrete override, forward-declared owner pointer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:53:00 -04:00
{
refactor(baselines): the first real module - a class, not an #included partial
Baselines was 951 lines of CExpertSignalAIBase method bodies in a file
that only looked like a module. It is now CBaselineComparator: a class
the signal OWNS, which reads a CTrainingDataView and prints. It does
not name the signal anywhere in its code.
What the seam forced out into the open:
- Thirty-odd ArraySize() bounds tests, each carried by its caller, are
now one test per accessor next to the data. The two `hasValueN` and
one `arrowN` locals are gone with them.
- The -2.0 "never scored" sentinel on the arrow cache was tested at the
call site. It is now inside DataDirectionalCall, where it cannot be
read as a small confidence.
- DoubleToSignal needs m_outputNeuronsCount, so a raw double could not
be turned into a side by any reader. The view answers
DirectionalCall(bar, isBuy, magnitude) instead - the conversion
happens where the head width lives, and the module no longer needs
ENUM_SIGNAL at all.
- m_baselineDone was a latch on the signal for a decision only this
module makes. It is m_done, private, where it belongs.
Correction to my own earlier claim: I said Baselines had nine exclusive
members "polluting the signal class". It had none. m_x, m_f, m_ngrad,
m_AvgCE and the rest are FIELDS OF ALGLIB REPORT OBJECTS (state.m_x,
mrep.m_AvgCE) that my `\bm_\w+` scan matched after the dot. The module
needs no private state but its view pointer and that latch - which is
why it came out this cleanly.
The include sits below the g_ens* vote globals and the Alglib headers
it reads, because unlike the AIBase\*.mqh partials this is a real class
declaration compiled where it stands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 12:00:18 -04:00
isBuy = false ;
magnitude = 0.0 ;
refactor(arch): a read-only training-data view, so modules stop being #included code
The AIBase\*.mqh files are not modules. They are method bodies of one
3,400-line class, textually #included after its declaration. Every one
of them can touch every member of every other, which is why "move this
out" has so far meant "move the whole class".
Introduce the seam that ends that:
CTrainingDataView abstract - the ONLY thing a training-side
collaborator may see: a feature row, a label, an
outcome, an excursion, the shape they share, and
the identity to log under.
CAIBaseTrainingData the adapter. MQL5 gives a class exactly one base
and CExpertSignalAIBase is already a
CExpertSignalCustom, so it cannot implement the
view itself. It owns one of these instead.
Data*() on the the published read API the adapter forwards to.
signal MQL5 has no `friend`, so reaching in from outside
was never an option - and making it explicit is
the point rather than a workaround.
Every row accessor OWNS ITS BOUNDS TEST and answers false for a bar it
has nothing for. Thirty-odd call sites currently carry their own
ArraySize() guard; one that forgets reads past a cache that is shorter
than the bar count for the whole warm-up. The -2.0 "never scored"
sentinel on the arrow cache is folded in the same way, so it can no
longer be mistaken for a small confidence.
Nothing uses it yet - this is the seam only, kept as its own commit so
the pattern compiles before 951 lines of Baselines move onto it. The
pattern is the stdlib's own: abstract base with =0 (Canvas\DX\DXObject),
concrete override, forward-declared owner pointer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:53:00 -04:00
if ( bar < 0 | | bar > = ArraySize ( m_arrowSignalCache ) )
return false ;
double v = m_arrowSignalCache [ bar ] ;
if ( v = = -2.0 | | ! MathIsValidNumber ( v ) )
return false ;
refactor(baselines): the first real module - a class, not an #included partial
Baselines was 951 lines of CExpertSignalAIBase method bodies in a file
that only looked like a module. It is now CBaselineComparator: a class
the signal OWNS, which reads a CTrainingDataView and prints. It does
not name the signal anywhere in its code.
What the seam forced out into the open:
- Thirty-odd ArraySize() bounds tests, each carried by its caller, are
now one test per accessor next to the data. The two `hasValueN` and
one `arrowN` locals are gone with them.
- The -2.0 "never scored" sentinel on the arrow cache was tested at the
call site. It is now inside DataDirectionalCall, where it cannot be
read as a small confidence.
- DoubleToSignal needs m_outputNeuronsCount, so a raw double could not
be turned into a side by any reader. The view answers
DirectionalCall(bar, isBuy, magnitude) instead - the conversion
happens where the head width lives, and the module no longer needs
ENUM_SIGNAL at all.
- m_baselineDone was a latch on the signal for a decision only this
module makes. It is m_done, private, where it belongs.
Correction to my own earlier claim: I said Baselines had nine exclusive
members "polluting the signal class". It had none. m_x, m_f, m_ngrad,
m_AvgCE and the rest are FIELDS OF ALGLIB REPORT OBJECTS (state.m_x,
mrep.m_AvgCE) that my `\bm_\w+` scan matched after the dot. The module
needs no private state but its view pointer and that latch - which is
why it came out this cleanly.
The include sits below the g_ens* vote globals and the Alglib headers
it reads, because unlike the AIBase\*.mqh partials this is a real class
declaration compiled where it stands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 12:00:18 -04:00
ENUM_SIGNAL side = DoubleToSignal ( v ) ;
if ( side ! = Buy & & side ! = Sell )
return false ;
isBuy = ( side = = Buy ) ;
magnitude = MathAbs ( v ) ;
refactor(arch): a read-only training-data view, so modules stop being #included code
The AIBase\*.mqh files are not modules. They are method bodies of one
3,400-line class, textually #included after its declaration. Every one
of them can touch every member of every other, which is why "move this
out" has so far meant "move the whole class".
Introduce the seam that ends that:
CTrainingDataView abstract - the ONLY thing a training-side
collaborator may see: a feature row, a label, an
outcome, an excursion, the shape they share, and
the identity to log under.
CAIBaseTrainingData the adapter. MQL5 gives a class exactly one base
and CExpertSignalAIBase is already a
CExpertSignalCustom, so it cannot implement the
view itself. It owns one of these instead.
Data*() on the the published read API the adapter forwards to.
signal MQL5 has no `friend`, so reaching in from outside
was never an option - and making it explicit is
the point rather than a workaround.
Every row accessor OWNS ITS BOUNDS TEST and answers false for a bar it
has nothing for. Thirty-odd call sites currently carry their own
ArraySize() guard; one that forgets reads past a cache that is shorter
than the bar count for the whole warm-up. The -2.0 "never scored"
sentinel on the arrow cache is folded in the same way, so it can no
longer be mistaken for a small confidence.
Nothing uses it yet - this is the seam only, kept as its own commit so
the pattern compiles before 951 lines of Baselines move onto it. The
pattern is the stdlib's own: abstract base with =0 (Canvas\DX\DXObject),
concrete override, forward-declared owner pointer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:53:00 -04:00
return true ;
}
string DataId ( void ) const { return ID ; }
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- The shared scratch buffer, for the views that hand it to a collaborator.
CArrayDouble * DataTempData ( void ) { return TempData ; }
ENUM_ACTIVATION DataHiddenLayerActivation ( void ) { return HiddenLayerActivation ( ) ; }
2026-08-24 04:39:17 -04:00
//--- The short id (m_id, set by SetIdentity) - CConfigLock's WarriorAI_<id>_<hash> global-variable
//--- name is the only outside reader; no prior public accessor exposed it.
string ConfigLockShortId ( void ) const { return m_id ; }
refactor(arch): a read-only training-data view, so modules stop being #included code
The AIBase\*.mqh files are not modules. They are method bodies of one
3,400-line class, textually #included after its declaration. Every one
of them can touch every member of every other, which is why "move this
out" has so far meant "move the whole class".
Introduce the seam that ends that:
CTrainingDataView abstract - the ONLY thing a training-side
collaborator may see: a feature row, a label, an
outcome, an excursion, the shape they share, and
the identity to log under.
CAIBaseTrainingData the adapter. MQL5 gives a class exactly one base
and CExpertSignalAIBase is already a
CExpertSignalCustom, so it cannot implement the
view itself. It owns one of these instead.
Data*() on the the published read API the adapter forwards to.
signal MQL5 has no `friend`, so reaching in from outside
was never an option - and making it explicit is
the point rather than a workaround.
Every row accessor OWNS ITS BOUNDS TEST and answers false for a bar it
has nothing for. Thirty-odd call sites currently carry their own
ArraySize() guard; one that forgets reads past a cache that is shorter
than the bar count for the whole warm-up. The -2.0 "never scored"
sentinel on the arrow cache is folded in the same way, so it can no
longer be mistaken for a small confidence.
Nothing uses it yet - this is the seam only, kept as its own commit so
the pattern compiles before 951 lines of Baselines move onto it. The
pattern is the stdlib's own: abstract base with =0 (Canvas\DX\DXObject),
concrete override, forward-declared owner pointer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:53:00 -04:00
bool DataIsEnsembleMember ( void ) const { return m_ensembleMember ; }
int DataEnsembleIndex ( void ) const { return m_ensembleIndex ; }
//--- calls == 0 means the gate has NOT scored yet, which is not a gate that scored zero - so
//--- this answers false there rather than handing back a 0% that reads as a measurement.
bool DataGateReference ( double & precPct , int & calls , double & chancePct ) const
{
precPct = m_bestDirPrecPct ;
calls = m_bestDirCalls ;
chancePct = m_bestChancePrecPct ;
return ( calls > 0 ) ;
}
double DataEffectiveSampleSize ( const double rawN ) const { return EffectiveSampleSize ( rawN ) ; }
//--- THE HANDLE COLLABORATORS ARE GIVEN. They take a CTrainingDataView* and so cannot reach
//--- anything above that is not on it - which is the point of handing them this and not `this`.
CTrainingDataView * TrainingData ( void ) { return GetPointer ( m_trainingData ) ; }
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
//--- Same doctrine, chart side: CChartUI takes a CChartView* and never `this`.
CChartView * ChartView ( void ) { return GetPointer ( m_chartView ) ; }
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
//--- Same doctrine, persistence side: CModelPersistence takes a CPersistenceView* and never `this`.
CPersistenceView * PersistenceView ( void ) { return GetPointer ( m_persistenceView ) ; }
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
//--- Same doctrine, online-learning side: COnlineLearning takes a COnlineLearningView* and never `this`.
COnlineLearningView * OnlineLearningView ( void ) { return GetPointer ( m_onlineLearningView ) ; }
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- Same doctrine, topology side: CTopology takes a CTopologyView* and never `this`.
CTopologyView * TopologyView ( void ) { return GetPointer ( m_topologyView ) ; }
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- Same doctrine, feature-building side: CFeatureBuilder takes a CFeaturesView* and never `this`.
CFeaturesView * FeaturesView ( void ) { return GetPointer ( m_featuresView ) ; }
2026-08-24 04:39:17 -04:00
//--- Same doctrine, config-lock side: CConfigLock takes a CConfigLockView* and never `this`.
CConfigLockView * ConfigLockView ( void ) { return GetPointer ( m_configLockView ) ; }
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
//--- CHART VIEW published read API - see Expert\Chart\IChartView.mqh for the contract these
//--- serve. Same doctrine as the Data*() block above: named and bounds-checked where the raw
//--- member would let a caller run past a cache, so CChartUI never pokes a member directly.
string ChartFileName ( void ) const { return m_fileName ; }
int ChartDigits ( void ) const { return m_symbol . Digits ( ) ; }
string ChartSymbolName ( void ) const { return m_symbol . Name ( ) ; }
2026-08-23 20:24:54 -04:00
//--- NOT "ChartPeriod" - MQL5's builtin global ChartPeriod(chart_id) exists, and a 0-arg member
//--- of the same name hides it for every unqualified caller in this class's own body (Lifecycle.mqh's
//--- ChartPeriod(owner) resolved here instead, "wrong parameters count, 1 passed but 0 requires").
ENUM_TIMEFRAMES ChartTimeframe ( void ) const { return ( ENUM_TIMEFRAMES ) m_period ; }
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
bool ChartModelLoadedFromDisk ( void ) const { return m_modelLoadedFromDisk ; }
bool ChartTrainingComplete ( void ) const { return m_trainingComplete ; }
int ChartOutputNeuronsCount ( void ) const { return m_outputNeuronsCount ; }
bool ChartNetReady ( void ) const { return ( CheckPointer ( Net ) ! = POINTER_INVALID & & m_isInitialized ) ; }
datetime ChartBarTime ( const int idx ) { return m_Time . GetData ( idx ) ; }
double ChartBarClose ( const int idx ) { return m_Close . GetData ( idx ) ; }
int ChartAvailableBars ( void ) { return Bars ( m_symbol . Name ( ) , PERIOD_CURRENT ) ; }
//--- ONE bar through the deployed (shadow-preferred) net for AdvanceChartSignalRescan: builds the
//--- feature window, forwards, and returns both the raw argmax-basis signal (pre logit-prior
//--- correction, for the raw tally) and the adjusted one (for the cache). False = the window
//--- could not be built (bar skipped, not scored) - same as the inline body this replaces.
bool ChartScoreBarForRescan ( const int idx , double & rawSignal , double & adjustedSignal )
{
rawSignal = 0.0 ;
adjustedSignal = -2.0 ;
if ( ! BuildFeatureWindow ( idx ) )
return false ;
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
CNet * deployNet = m_onlineLearning . DeployNet ( ) ;
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
deployNet . feedForward ( TempData ) ;
deployNet . getResults ( TempData ) ;
if ( m_outputNeuronsCount = = 1 )
{
adjustedSignal = TempData [ 0 ] ;
rawSignal = TempData [ 0 ] ;
}
else
{
rawSignal = ApplyClassificationSoftmax ( ) ;
adjustedSignal = AdjustedSignalFromSoftmax ( ) ;
}
return true ;
}
//--- PREDICTION CACHE (m_arrowSignalCache). Stays signal-owned - Training.mqh writes it directly
//--- every era and DataDirectionalCall already reads it for the baseline comparator - so this is
//--- a bounds-checked window onto shared state, not a copy.
int ChartPredictionCacheSize ( void ) const { return ArraySize ( m_arrowSignalCache ) ; }
double ChartPredictionAt ( const int idx ) const
{ return ( idx > = 0 & & idx < ArraySize ( m_arrowSignalCache ) ) ? m_arrowSignalCache [ idx ] : -2.0 ; }
void ChartSetPredictionAt ( const int idx , const double value )
{ if ( idx > = 0 & & idx < ArraySize ( m_arrowSignalCache ) ) m_arrowSignalCache [ idx ] = value ; }
void ChartResizePredictionCache ( const int size , const double fillValue )
{ ArrayResize ( m_arrowSignalCache , size ) ; ArrayInitialize ( m_arrowSignalCache , fillValue ) ; }
fix(chart): a deployed model rescans history to rebuild its vote arrows
The sidecar added in 484a9d8 restores the vote arrows from the previous
session - but there was no previous session to restore from, and a
deployed ensemble could never produce one.
The overlay that draws the vote layer replays each member's
m_overlaySigSnap, published in exactly one place: RankTiersFromOos, at
pass-3 completion. A converged model runs no further eras. So after a
restart every member's snapshot was empty, would never fill, the sweep
had nothing to replay and the chart stayed blank permanently - no route
back by any path.
The chart rescan is the route: it runs the DEPLOYED net forward over
history and rebuilds the per-bar cache, which is the same quantity pass 3
produces, obtained without training. It already existed for the panel's
Show-Signals button; it just never handed its result to the overlay, so
on the default filtered view a rescan rebuilt only the RAW per-member
layer - the one that is hidden - and appeared to do nothing.
- PublishOverlaySnapshotFromCache() extracted from RankTiersFromOos, so
the era end and a completed rescan publish through one implementation.
- A completed rescan now calls it, which also arms the sweep.
- PollTraining auto-arms one rescan for a model that is converged, has no
snapshot, and is on the filtered view. One-shot: a model that
legitimately calls Neutral everywhere must not rescan forever chasing a
snapshot that is correctly empty. On the timer, not in OnInit - it is a
full feedForward per bar over up to 5000 bars and drains in the same
time-boxed slices as a manual rescan.
Together with the sidecar this closes both halves: the rescan covers the
first session and any chart whose file was lost or invalidated by a
threshold change; the sidecar covers every session after one is saved.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 13:11:36 -04:00
//--- THE OVERLAY'S DATA SOURCE, PUBLISHED. One implementation, two callers: the era end
//--- (RankTiersFromOos, at pass-3 completion) and a completed chart rescan. See its definition for
//--- why the second caller is what makes a DEPLOYED model's arrows rebuildable at all.
void PublishOverlaySnapshotFromCache ( void ) ;
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
//--- THE REPLAY PASS. Scores the rescan's per-bar signals against the label cache and hands the
//--- result to RankTiersFromOos(), so a converged model can mint its tier ladder without a
//--- training run. See the definition.
void ScoreReplayFromCache ( void ) ;
//--- Drives the deployed-model rebuild (labels -> rescan -> score -> rank -> save) one timer slice
//--- at a time. Returns true while it still owns the slice.
bool AdvanceDeployedRebuild ( void ) ;
//--- STAGE 3, and the completion hook for every rescan. A manual rescan (the panel's Show Signals)
//--- only needs the snapshot republished; a rebuild rescan additionally has to SCORE what it just
//--- computed, rank the ladder from it, and write the result down so the next restart inherits it.
//--- Both arrive here so there is one place that knows what a finished rescan means.
void OnChartRescanComplete ( void )
{
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
if ( m_deployedRebuildStage = = 1 )
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
{
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
m_deployedRebuildStage = 2 ;
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
//--- Ranks the ladder, which publishes the overlay snapshot and arms the sweep on its way
//--- through - see ScoreReplayFromCache().
ScoreReplayFromCache ( ) ;
//--- PERSIST IMMEDIATELY. The whole failure this repairs is state that existed in memory and
//--- was never written down; recomputing it and then not saving it would repeat that exactly,
//--- and the next restart would pay for the replay all over again.
if ( m_tiersSelfRanked & & ! SaveModelStats ( m_activeFileName , m_activeFileCommon ) )
Print ( ID + " : ERROR - rebuilt the tier ladder but could not persist it to .stats; "
" it will have to be replayed again on the next attach. " ) ;
return ;
}
PublishOverlaySnapshotFromCache ( ) ;
}
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
long ChartEraCount ( void ) const { return m_eraCount ; }
long ChartCumIsTotal ( void ) const { return m_cumIsTotal ; }
long ChartCumIsCorrect ( void ) const { return m_cumIsCorrect ; }
long ChartCumOosTotal ( void ) const { return m_cumOosTotal ; }
long ChartCumOosCorrect ( void ) const { return m_cumOosCorrect ; }
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
void ChartOosTally ( int & buyPredicted , int & sellPredicted , int & buyPredictedHits , int & sellPredictedHits ) const
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
{
buyPredicted = m_oos . buyPredicted ;
sellPredicted = m_oos . sellPredicted ;
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
buyPredictedHits = m_oos . buyPredictedHits ;
sellPredictedHits = m_oos . sellPredictedHits ;
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
}
string ChartPassLabel ( void ) const { return m_passLabel ; }
int ChartPassProgressPct ( void ) const { return m_passProgressPct ; }
int ChartOosSplitPct ( void ) const { return m_oosSplitPct ; }
int ChartOosSamples ( void ) const { return m_oosSamples ; }
void ChartClassCounts ( int & predBuy , int & predSell , int & predNeutral ,
int & trueBuy , int & trueSell , int & trueNeutral ) const
{
predBuy = m_countBuySignals ;
predSell = m_countSellSignals ;
predNeutral = m_countNeutralSignals ;
trueBuy = m_trueBuyCount ;
trueSell = m_trueSellCount ;
trueNeutral = m_trueNeutralCount ;
}
void ChartOosRecallPct ( int & buyRecallPct , int & sellRecallPct ) const
{ buyRecallPct = m_lastBuyRecallPct ; sellRecallPct = m_lastSellRecallPct ; }
void ChartOosLivePrecision ( int & buyPrecPct , int & buyFired , int & sellPrecPct , int & sellFired ) const
{
buyPrecPct = m_lastBuyFiredPrecPct ;
buyFired = m_lastBuyFired ;
sellPrecPct = m_lastSellFiredPrecPct ;
sellFired = m_lastSellFired ;
}
double ChartForecast ( void ) const { return dForecast ; }
double ChartErrorPct ( void ) const { return dError ; }
double ChartOosForecast ( void ) const { return dOosForecast ; }
double ChartOosErrorPct ( void ) const { return dOosError ; }
double ChartNetRecentAverageError ( void ) const
{ return ( CheckPointer ( Net ) ! = POINTER_INVALID ) ? Net . getRecentAverageError ( ) : 0.0 ; }
2026-08-23 20:24:54 -04:00
//--- Forwards to PROTECTED members the adapter cannot reach directly - CAIBaseChartView is not a
//--- derived class (MQL5 has no `friend`), so every protected call the view needs is re-published
//--- here, same doctrine as the rest of this block.
string ChartArrowPrefix ( void ) const { return ArrowPrefix ( ) ; }
string ChartDisplayName ( void ) const { return DisplayName ( ) ; }
bool ChartBothDirectionsTradeable ( void ) const { return BothDirectionsTradeable ( ) ; }
bool ChartResizeBuffers ( const int barIndex ) { return ResizeBuffers ( barIndex ) ; }
bool ChartRefreshData ( void ) { return RefreshData ( ) ; }
int ChartServableBars ( const int want , const string context ) { return ServableBars ( want , context ) ; }
void ChartEnsureShadowNet ( void ) { EnsureShadowNet ( ) ; }
ENUM_SIGNAL ChartDoubleToSignal ( const double value ) { return DoubleToSignal ( value ) ; }
bool ChartDisplayInference ( void ) { return DisplayInference ( ) ; }
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
//--- PERSISTENCE VIEW published read/write API - see Expert\Persistence\IPersistenceView.mqh for
//--- the contract these serve. Read+write, unlike the Chart*() block above: Persistence's whole
//--- job is loading saved state BACK into the signal.
bool PersistNetLoaded ( void ) const { return CheckPointer ( Net ) ! = POINTER_INVALID ; }
bool PersistUsesConvStage ( void ) const { return UsesConvStage ( ) ; }
uint PersistNetFirstConvWindow ( void ) const { return ( CheckPointer ( Net ) ! = POINTER_INVALID ) ? Net . FirstConvWindow ( ) : 0 ; }
int PersistConvReceptiveFieldBars ( void ) const { return ConvReceptiveFieldBars ( ) ; }
ENUM_ACTIVATION PersistOutputLayerActivation ( void ) const { return OutputLayerActivation ( ) ; }
bool PersistNetEnforceOutputActivation ( const ENUM_ACTIVATION intended , ENUM_ACTIVATION & stale )
{ return ( CheckPointer ( Net ) ! = POINTER_INVALID ) ? Net . EnforceOutputActivation ( intended , stale ) : false ; }
void PersistSetTopologySuperseded ( const bool v ) { m_topologySuperseded = v ; }
//--- SaveModelStats()/LoadModelStats() fields - one scalar getter+setter per on-disk field
double PersistPriorBuy ( void ) const { return m_priorBuy ; }
void PersistSetPriorBuy ( const double v ) { m_priorBuy = v ; }
double PersistPriorSell ( void ) const { return m_priorSell ; }
void PersistSetPriorSell ( const double v ) { m_priorSell = v ; }
double PersistPriorNeutral ( void ) const { return m_priorNeutral ; }
void PersistSetPriorNeutral ( const double v ) { m_priorNeutral = v ; }
double PersistConfidenceCalScale ( void ) const { return m_confidenceCalScale ; }
void PersistSetConfidenceCalScale ( const double v ) { m_confidenceCalScale = v ; }
bool PersistMqlInferenceValidated ( void ) const { return m_mqlInferenceValidated ; }
void PersistSetMqlInferenceValidated ( const bool v ) { m_mqlInferenceValidated = v ; }
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
datetime PersistOnlineLearnedUpToTime ( void ) const { return m_onlineLearning . LearnedUpToTime ( ) ; }
void PersistSetOnlineLearnedUpToTime ( const datetime v ) { m_onlineLearning . SetLearnedUpToTime ( v ) ; }
double PersistOnlineRollingAcc ( void ) const { return m_onlineLearning . RollingAcc ( ) ; }
void PersistSetOnlineRollingAcc ( const double v ) { m_onlineLearning . SetRollingAcc ( v ) ; }
long PersistOnlineSamples ( void ) const { return m_onlineLearning . Samples ( ) ; }
void PersistSetOnlineSamples ( const long v ) { m_onlineLearning . SetSamples ( v ) ; }
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
int PersistLastBuyFiredPrecPct ( void ) const { return m_lastBuyFiredPrecPct ; }
void PersistSetLastBuyFiredPrecPct ( const int v ) { m_lastBuyFiredPrecPct = v ; }
int PersistLastSellFiredPrecPct ( void ) const { return m_lastSellFiredPrecPct ; }
void PersistSetLastSellFiredPrecPct ( const int v ) { m_lastSellFiredPrecPct = v ; }
int PersistLastBuyRecallPct ( void ) const { return m_lastBuyRecallPct ; }
void PersistSetLastBuyRecallPct ( const int v ) { m_lastBuyRecallPct = v ; }
int PersistLastSellRecallPct ( void ) const { return m_lastSellRecallPct ; }
void PersistSetLastSellRecallPct ( const int v ) { m_lastSellRecallPct = v ; }
int PersistLastBuyFired ( void ) const { return m_lastBuyFired ; }
void PersistSetLastBuyFired ( const int v ) { m_lastBuyFired = v ; }
int PersistLastSellFired ( void ) const { return m_lastSellFired ; }
void PersistSetLastSellFired ( const int v ) { m_lastSellFired = v ; }
void PersistSetCumIsCorrect ( const long v ) { m_cumIsCorrect = v ; }
void PersistSetCumIsTotal ( const long v ) { m_cumIsTotal = v ; }
void PersistSetCumOosCorrect ( const long v ) { m_cumOosCorrect = v ; }
void PersistSetCumOosTotal ( const long v ) { m_cumOosTotal = v ; }
fix(vote): persist the tier ladder - a converged model was mute after every restart
THIS IS NOT A DISPLAY BUG. A deployed model could not vote, or trade, at
any point after a terminal restart, and never would have.
LiveVoteContribution() returns 0 for every call until m_tiersSelfRanked
is set - deliberately, and correctly: before RankTiersFromOos() runs,
m_pattern_0..3 hold the constructor's stock 25/50/75/100, which since the
2026-08-18 currency change is the WRONG UNIT rather than a weak opinion,
and one unranked member would drag the whole ensemble over any threshold.
But that ladder is produced ONLY by a completed pass 3, and it was never
persisted - the code comment at LiveVoteContribution says so outright.
A converged model runs no further passes. So on every restart it lost its
entire vote permanently:
LiveVoteContribution -> 0 => no live vote ("0 vote/4 flat")
ReconstructionWeight -> 0 => overlay divisor 0 ("0 had a snapshot")
=> no arrows
=> no fired bars, so g_ensCumOosTotal stays 0
=> "measuring..." forever
Every symptom reported over the last three exchanges is that one cause.
The log is unambiguous: six H4 charts resumed at era 70/71, all 24
rescans completed with ~2700 Buy / ~2200 Sell per model, and the overlay
then swept 4999 bars finding "0 had a snapshot". The calls were there;
nothing was permitted to count them.
WST7 now stores the four tier weights, the module trust weight and the
self-ranked flag beside the model. Restored only when the stored flag
says the ladder was MEASURED - a .stats written before a model's first
pass 3 holds the stock ladder, and adopting that as if measured is the
exact error the flag exists to prevent.
A .stats predating WST7 has no ladder, so existing converged models stay
silent until their next scoring pass mints one. That case now prints a
warning naming all three of its symptoms, because each one independently
looks like a different bug.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 14:26:33 -04:00
//--- THE TIER LADDER, for .stats (WST7). See LiveVoteContribution(): until RankTiersFromOos() has
//--- run once, m_pattern_0..3 hold the constructor's stock 25/50/75/100 - the WRONG UNIT, not a
//--- weak opinion - and the member is deliberately silenced. That silencing is correct while
//--- training and catastrophic on a resume: a converged model runs no further passes, so without
//--- these fields it can never speak again.
int PersistTierWeight ( const int tier ) const
{
switch ( tier )
{
case 0 :
return m_pattern_0 ;
case 1 :
return m_pattern_1 ;
case 2 :
return m_pattern_2 ;
default :
return m_pattern_3 ;
}
}
void PersistSetTierWeight ( const int tier , const int v )
{
switch ( tier )
{
case 0 :
m_pattern_0 = v ;
break ;
case 1 :
m_pattern_1 = v ;
break ;
case 2 :
m_pattern_2 = v ;
break ;
default :
m_pattern_3 = v ;
break ;
}
}
void PersistSetModuleTrustWeight ( const double v ) { Weight ( v ) ; }
void PersistSetTiersSelfRanked ( const bool v ) { m_tiersSelfRanked = v ; }
2026-08-26 16:53:16 -04:00
//--- WST8: the pair HasDemonstratedEdge() falls back to. Deliberately NOT the era pair - that one
//--- belongs to an era that will not exist after a restart, and writing it down would claim a
//--- measurement for weights that may have moved since.
double PersistCertifiedPrecPct ( void ) const { return m_certifiedPrecPct ; }
double PersistCertifiedChancePct ( void ) const { return m_certifiedChancePct ; }
void PersistSetCertifiedEdge ( const double precPct , const double chancePct )
{
if ( precPct < 0.0 | | chancePct < 0.0 )
return ;
m_certifiedPrecPct = precPct ;
m_certifiedChancePct = chancePct ;
}
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
//--- ValidateCpuInference() - the whole Net-pointer/throwaway-clone core, consolidated: this is
//--- irreducible pointer/object work, not signal state, same doctrine as ChartScoreBarForRescan.
bool PersistRunCpuInferenceSelfCheck ( double & maxDiff )
{
maxDiff = DBL_MAX ;
if ( CheckPointer ( Net ) = = POINTER_INVALID | | CheckPointer ( TempData ) = = POINTER_INVALID )
return false ;
if ( ! BuildFeatureWindow ( 0 ) )
return false ;
Net . SetBatchNormFrozen ( true ) ;
bool refOk = Net . feedForward ( TempData ) ;
Net . SetBatchNormFrozen ( false ) ;
if ( ! refOk )
return false ;
CArrayDouble * refOut = new CArrayDouble ( ) ;
if ( CheckPointer ( refOut ) = = POINTER_INVALID )
return false ;
Net . getResults ( refOut ) ;
CNet * cpu = new CNet ( NULL ) ;
if ( CheckPointer ( cpu ) = = POINTER_INVALID )
{
delete refOut ;
return false ;
}
cpu . SetCpuInference ( true ) ;
double e , u , f ;
datetime tm ;
long era ;
bool complete ;
double ip [ ] ;
bool loaded = cpu . Load ( m_activeFileName + " .nnw " , e , u , f , tm , m_activeFileCommon , era , complete , ip ) ;
if ( loaded )
cpu . SetBatchNormFrozen ( true ) ;
bool pass = false ;
if ( loaded & & cpu . feedForward ( TempData ) )
{
CArrayDouble * cpuOut = new CArrayDouble ( ) ;
if ( CheckPointer ( cpuOut ) ! = POINTER_INVALID )
{
cpu . getResults ( cpuOut ) ;
if ( cpuOut . Total ( ) = = refOut . Total ( ) & & refOut . Total ( ) > 0 )
{
maxDiff = 0.0 ;
for ( int i = 0 ; i < refOut . Total ( ) ; i + + )
maxDiff = MathMax ( maxDiff , MathAbs ( refOut . At ( i ) - cpuOut . At ( i ) ) ) ;
pass = ( maxDiff < = CPU_INFERENCE_MAX_DIFF ) ;
}
delete cpuOut ;
}
}
delete cpu ;
delete refOut ;
return pass ;
}
//--- SaveTopologyConfiguration()/LoadAndCompareTopologyConfiguration() fields beyond the params
//--- every caller already passes
void PersistSetDirConfThreshold ( const double v ) { m_dirConfThreshold = v ; }
double PersistBestDirConfThreshold ( void ) const { return m_bestDirConfThreshold ; }
void PersistSetBestDirConfThreshold ( const double v ) { m_bestDirConfThreshold = v ; }
string PersistCrossAssetPairsPinned ( void ) const { return m_crossAssetPairsPinned ; }
void PersistSetCrossAssetPairsPinned ( const string v ) { m_crossAssetPairsPinned = v ; }
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
bool PersistCrossAssetCfgSaved ( void ) const { return m_crossAssetCfgSaved ; }
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
void PersistSetCrossAssetCfgSaved ( const bool v ) { m_crossAssetCfgSaved = v ; }
string PersistAltDataNamesPinned ( void ) const { return m_altDataNamesPinned ; }
void PersistSetAltDataNamesPinned ( const string v ) { m_altDataNamesPinned = v ; }
void PersistApplyAltDataPinnedNames ( const string v ) { m_altData . SetPinnedNames ( v ) ; }
//--- LoadNetWithRetry()
bool PersistLoadNetOnce ( double & indicatorParams [ ] )
{ return Net . Load ( m_activeFileName + " .nnw " , dError , dUndefine , dForecast , dtStudied , m_activeFileCommon , m_eraCount , m_trainingComplete , indicatorParams ) ; }
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
//--- ONLINE-LEARNING VIEW published read/write API - see Expert\OnlineLearning\IOnlineLearningView.mqh
//--- for the contract these serve. New wrappers for protected members/methods COnlineLearning has
//--- no other way to reach (MQL5 has no `friend`); everything already public elsewhere (DataId(),
//--- ChartEraCount(), PersistPriorBuy() etc.) is reused directly by the adapter instead of repeated here.
string OnlineActiveFileName ( void ) const { return m_activeFileName ; }
bool OnlineActiveFileCommon ( void ) const { return m_activeFileCommon ; }
double OnlineDUndefine ( void ) const { return dUndefine ; }
datetime OnlineDtStudied ( void ) const { return dtStudied ; }
CNet * OnlineNet ( void ) { return Net ; }
bool OnlineBuildFeatureWindow ( const int r ) { return BuildFeatureWindow ( r ) ; }
double OnlineApplyClassificationSoftmax ( void ) { return ApplyClassificationSoftmax ( ) ; }
double OnlineAdjustedSignalFromSoftmax ( void ) { return AdjustedSignalFromSoftmax ( ) ; }
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- The label the online step learns from: resolve-on-demand through the SAME finality-gated
//--- cache path training uses, then read the cache. Undefine = still unresolved, do not learn.
ENUM_SIGNAL OnlineBarLabel ( const int idx )
{
if ( idx > = 0 & & idx < ArraySize ( m_labelCacheHasValue ) & & ! m_labelCacheHasValue [ idx ] )
AdvanceSwingLabelState ( idx , ArraySize ( m_labelCacheHasValue ) ) ;
if ( idx < 0 | | idx > = ArraySize ( m_labelCacheHasValue ) | | ! m_labelCacheHasValue [ idx ] )
return Undefine ;
return m_labelCacheBuy [ idx ] ? Buy : ( m_labelCacheSell [ idx ] ? Sell : Neutral ) ;
}
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
int OnlineCalibLoIndex ( const int oosCutoff ) { return CalibLoIndex ( oosCutoff ) ; }
int OnlineCalibBandBars ( const int totalIter , const int oosCutoff ) { return CalibBandBars ( totalIter , oosCutoff ) ; }
bool OnlineInferenceOnlyFlag ( void ) const { return m_inferenceOnly ; }
bool OnlineTrainRunActiveFlag ( void ) const { return m_trainRunActive ; }
bool OnlineEnableLearningFlag ( void ) const { return m_onlineLearning . Enabled ( ) ; }
bool OnlineEnsureBarCachesCapacity ( const int bars ) { return EnsureBarCachesCapacity ( bars ) ; }
double OnlineAtrMain ( const int idx ) { return m_ATR . Main ( idx ) ; }
double OnlineSpreadPrice ( void ) { return ( double ) m_symbol . Spread ( ) * m_symbol . Point ( ) ; }
int OnlineConfidenceTier ( void ) { return ConfidenceTier ( ) ; }
double OnlinePatternWeightForTier ( const int tier ) { return PatternWeightForTier ( tier ) ; }
string OnlinePatternTableName ( const string filterID , const string pattern , const string direction )
{ return PatternTableName ( filterID , pattern , direction ) ; }
void OnlineRegisterSignal ( int year , int month , int day , int DOW , int hour , int minutes ,
string tableName , string pattern , string direction ,
double entryPrice , double exitPrice , string result , double netVote )
{ RegisterSignal ( year , month , day , DOW , hour , minutes , tableName , pattern , direction , entryPrice , exitPrice , result , netVote ) ; }
double OnlinePrevSignal ( void ) const { return dPrevSignal ; }
void OnlineSetPrevSignal ( const double v ) { dPrevSignal = v ; }
bool OnlineSaveModelStats ( void ) { return SaveModelStats ( m_activeFileName , m_activeFileCommon ) ; }
void OnlineFlattenIndicatorParams ( double & ip [ ] ) { m_indicatorTuner . Flatten ( ip ) ; }
double OnlineModelEta ( void ) const { return m_modelEta ; }
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- TOPOLOGY VIEW published read/write API (continued) - see Expert\Topology\ITopologyView.mqh.
//--- New wrappers for protected members/methods CTopology has no other way to reach (MQL5 has no
//--- `friend`); everything already public elsewhere (DataId(), ChartOutputNeuronsCount(),
//--- PersistUsesConvStage() etc.) is reused directly by the adapter instead of repeated here.
int TopologyOptimizationAlgo ( void ) const { return m_optimizationAlgo ; }
int TopologyMinTrainYear ( void ) const { return m_minTrainYear ; }
int TopologyFractalPeriods ( void ) const { return m_fractalPeriods ; }
int TopologyConvFilterCount ( void ) const { return m_convFilterCount ; }
int TopologyLstmHiddenSize ( void ) const { return m_lstmHiddenSize ; }
int TopologyInitialNeuronsCount ( void ) const { return m_initialNeuronsCount ; }
int TopologyHiddenLayersCount ( void ) const { return m_hiddenLayersCount ; }
bool TopologyUsesLstmStage ( void ) const { return UsesLstmStage ( ) ; }
bool TopologyHasConvBeforeLstm ( void ) const { return HasConvBeforeLstm ( ) ; }
int TopologyNetInputWidth ( void ) const { return NetInputWidth ( ) ; }
bool TopologyAddCustomLayers ( CArrayObj * topology ) { return AddCustomLayers ( topology ) ; }
fix(topology): size the network against observations, not bars
The capacity budget is stated in weights per INDEPENDENT observation
and divides by the mean label lifespan to get there. It never once
did: EstimatedInSampleBars() deflates via m_labelOverlap, but it is
only ever called from InitNeuralNetwork, where the label cache does
not exist yet (that same function sets m_labelCachePrebuilt = false
a few lines below), so MeanLifespan() returned its "nothing measured"
default of 1.0 at every call. Every fresh model was sized as though
its labels did not overlap - over-budgeting the first dense layer by
a factor of L, which is several rungs of a power-of-two ladder. The
"expect overfitting, reduce the feature set or pool instruments"
warning is the branch that should fire on H1 and structurally could
not.
Fixed at the source rather than by reordering the boot sequence (the
prebuild is chunked across Train() calls and cannot complete inside
init): MeasureSwingGeometry() walks the ZigZag ONCE at init and
answers both questions from it - the median leg gives the window,
and the leg series gives the mean label lifespan analytically.
SwingPivotDirectionLabel resolves bar i when the SECOND pivot after
it commits, so a bar d bars before pivot P waits d + (the leg
leaving P); summed over every bar of every leg that is exactly the
mean the label walk accumulates.
That also closes the coherence gap the swing target opened: the
window was measured with a private +/-12-bar fractal while the label
aimed at ZigZag(12,5,3) pivots, so it was sized against a leg
distribution the label never used. One pivot source now, the
label's.
Also:
- ResetWeights() re-derives the shape. It rebuilt from the members a
history-starved init had pinned and re-saved them - so the "let
history download, then reset from the panel" advice in both
fallback warnings did nothing at all.
- The CAPACITY line prints the measured lifespan beside the one the
topology was sized for, and warns when they differ by more than a
ladder rung. That is the check that makes the estimator falsifiable.
- Topology reads the view's symbol, not _Symbol (latent for pooling).
- Unmeasured geometry defaults to HISTORY_BARS_FALLBACK, never 1.0:
under-sizing is recoverable, over-sizing silently is not.
Compile: 0 errors, 0 warnings (stage).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 21:21:05 -04:00
//--- The LABEL'S pivot source. Handed over as a HANDLE, not as the CiCustom: CTopology reads it at
//--- init via CopyBuffer, before ResizeBuffers() has sized the wrapper's own buffers.
int TopologyZigZagHandle ( void ) { return m_zigZag . Handle ( ) ; }
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
//--- The whole Net-pointer swap, consolidated: not signal state, irreducible pointer/object work -
//--- same doctrine as PersistRunCpuInferenceSelfCheck/ChartScoreBarForRescan.
bool TopologyReplaceNetFromTopology ( CArrayObj * topology )
{
if ( CheckPointer ( Net ) ! = POINTER_INVALID )
delete Net ;
Net = new CNet ( topology ) ;
return ( CheckPointer ( Net ) ! = POINTER_INVALID ) ;
}
void TopologyResetOnlineLearningForFreshTopology ( void ) { m_onlineLearning . ResetForFreshTopology ( ) ; }
2026-08-25 22:51:50 -04:00
//--- CAPACITY SIZING WAS POOL-BLIND: EstimatedInSampleBars() budgeted the first hidden layer from
//--- THIS CHART's own bars only, while Use_Training_Pool feeds the trainer up to TRAINPOOL_MAX_ROWS
//--- peer rows (Training\TrainingPool.mqh) it never counted - every model was sized several times
//--- narrower than its actual training set, and the "cannot support N features" warning
//--- correspondingly overstated. header-only census (TrainPoolEstimateAvailableRows), never opens
//--- a row.
//---
//--- TWO CONSERVATIVE DISCOUNTS, deliberately not a 1:1 row count:
//--- 1. Divided by this chart's own SwingLifespanEstimate() - a peer row is not more independent
//--- of its OWN neighbours than this chart's bars are of theirs, and the label-overlap
//--- deflation this chart already applies to its own bars is the only overlap estimate
//--- available (peers do not publish their own).
//--- 2. Divided by the number of CONTRIBUTING peer files - pooled instruments are cross-
//--- sectionally correlated (EURUSD/USDCAD/USDJPY especially, all USD-legged), so N peers do
//--- not carry N independent peers' worth of evidence. Treating the whole pool as worth
//--- roughly one peer's independent contribution is a floor, not a measurement - there is no
//--- cross-instrument correlation structure measured anywhere in this codebase to do better.
//--- Only ever narrows the pool's contribution, never invents capacity beyond what
//--- TrainPoolEstimateAvailableRows() actually found on disk.
2026-08-25 23:16:05 -04:00
//--- DIAGNOSTIC, print-once: this number reached production 2026-08-25 reporting EXACTLY the
//--- pre-fix (0-contribution) figure on every chart's first log, which is ambiguous between
//--- "correctly found no compatible peer data yet" and "a bug in this function" from the log
//--- alone - see [[project_audit_20260825_mega_patch]]. Remove once a run confirms a nonzero
//--- census under known-good pool files.
bool m_poolCensusLogged ;
2026-08-25 22:51:50 -04:00
double TopologyPooledIndependentBars ( void )
{
if ( ! Use_Training_Pool )
return 0.0 ;
//--- NetInputWidth(), NOT DataFeaturesPerBar(): that is what CTrainPoolWriter/Reader actually
//--- key the pool's width field on (see Training.mqh's own m_trainPoolReader.Adopt() call) -
//--- the per-bar feature count alone would under-specify the row layout for any conv/LSTM
//--- front-end, where the vector reaching the dense stack differs from the raw feature count.
string fp = m_topology . BuildModelFingerprint ( ) ;
int peerFiles = 0 ;
int rows = TrainPoolEstimateAvailableRows ( fp , ChartSymbolName ( ) , ( int ) ChartTimeframe ( ) ,
NetInputWidth ( ) , peerFiles ) ;
double lifespan = m_topology . SwingLifespanEstimate ( ) ;
2026-08-25 23:16:05 -04:00
double result = ( rows < = 0 | | peerFiles < = 0 ) ? 0.0
: ( double ) rows / MathMax ( 1.0 , lifespan ) / ( double ) peerFiles ;
if ( ! m_poolCensusLogged )
{
m_poolCensusLogged = true ;
PrintFormat ( " %s: POOL CENSUS - fingerprint=%s width=%d symbol=%s period=%d -> %d compatible "
" row(s) from %d peer file(s), lifespan=%.1f -> +%.1f independent observations "
" added to this model's capacity budget. " ,
DataId ( ) , fp , NetInputWidth ( ) , ChartSymbolName ( ) , ( int ) ChartTimeframe ( ) , rows ,
peerFiles , lifespan , result ) ;
}
return result ;
2026-08-25 22:51:50 -04:00
}
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- FEATURES VIEW published read/write API - see Expert\Features\IFeaturesView.mqh for the
//--- contract these serve. New wrappers for protected members/methods CFeatureBuilder has no other
//--- way to reach (MQL5 has no `friend`); everything already public elsewhere (DataId(),
//--- ChartSymbolName(), UseMA() etc.) is reused directly by the adapter instead of repeated here.
2026-08-24 18:26:25 -04:00
//--- m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_zigZag stay signal-owned (real use in Labels.mqh/
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- AutoTune.mqh/Training.mqh too) - these are READ-ONLY windows onto them, never the Init/Resize/
//--- Refresh lifecycle (that stays in Expert\AIBase\Features.mqh, see InitOpen's comment).
double FeatureOpenAt ( const int idx ) const { return m_Open . GetData ( idx ) ; }
double FeatureHighAt ( const int idx ) const { return m_High . GetData ( idx ) ; }
double FeatureLowAt ( const int idx ) const { return m_Low . GetData ( idx ) ; }
int FeatureAtrBarsCalculated ( void ) const { return m_ATR . BarsCalculated ( ) ; }
int FeatureAtrHandle ( void ) const { return m_ATR . Handle ( ) ; }
2026-08-24 18:26:25 -04:00
int FeatureZigZagBarsCalculated ( void ) const { return m_zigZag . BarsCalculated ( ) ; }
int FeatureZigZagHandle ( void ) const { return m_zigZag . Handle ( ) ; }
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
double FeatureSymbolPoint ( void ) const { return m_symbol . Point ( ) ; }
CIndicators * FeatureIndicatorsPtr ( void ) const { return m_indicatorsPtr ; }
//--- Whole-object pointer, same doctrine as TrainingData()/ChartView() etc: CADIndicatorTuner
//--- already declares its fields public on ITS OWN class, so CFeatureBuilder reads
//--- .maPeriod/.ichiKijun/.adWES.lookback/etc and calls .Flatten()/.Unflatten() straight through
//--- this pointer instead of one accessor per nested field.
CADIndicatorTuner * FeatureIndicatorTuner ( void ) { return GetPointer ( m_indicatorTuner ) ; }
//--- Same doctrine for the cross-asset panel - CCrossAssetPanel's build/query API is already
//--- public on its own class.
CCrossAssetPanel * FeatureCrossAsset ( void ) { return GetPointer ( m_crossAsset ) ; }
int FeatureAltDataFeatureCount ( void ) const { return m_altData . FeatureCount ( ) ; }
void FeatureAltDataEnsureFresh ( const datetime asOf ) { m_altData . EnsureFresh ( asOf ) ; }
void FeatureAltDataFeatures ( const datetime t , double & out [ ] ) { m_altData . Features ( t , out ) ; }
//--- The SaveTopologyConfiguration() call BuildCrossAssetPanel makes when it first pins the
//--- cross-asset pair set to the .cfg - consolidated into one call, same doctrine as
//--- TopologyReplaceNetFromTopology (irreducible persistence side-effect, not signal state).
bool FeatureCommitCrossAssetPin ( void )
{
return SaveTopologyConfiguration ( m_activeFileName , m_initialNeuronsCount , m_hiddenLayersCount ,
m_neuronsReduction , m_minNeuronsCount , m_optimizationAlgo ,
m_historyBars , m_outputNeuronsCount , m_neuronsCount ,
LEGACY_STUDY_PERIOD_SLOT , m_minTrainYear , m_isInitialized ,
LEGACY_CONVERGE_WR_SLOT , m_fractalPeriods , m_convFilterCount ,
m_lstmHiddenSize , m_activeFileCommon ) ;
}
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- Zero-skill precision reference for the detectability report: the prebuild-measured base
//--- rate of the LARGER directional class. -1 until the prebuild has tallied.
double FeatureChanceRatePct ( void ) const
{
long tot = ( long ) m_labelPrebuildBuyCount + m_labelPrebuildSellCount + m_labelPrebuildNeutralCount ;
if ( tot < = 0 )
return -1.0 ;
return 100.0 * ( double ) MathMax ( m_labelPrebuildBuyCount , m_labelPrebuildSellCount ) / tot ;
}
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
int FeatureFirstLayerFanIn ( void ) const { return FirstLayerFanIn ( ) ; }
fix(topology): size the network against observations, not bars
The capacity budget is stated in weights per INDEPENDENT observation
and divides by the mean label lifespan to get there. It never once
did: EstimatedInSampleBars() deflates via m_labelOverlap, but it is
only ever called from InitNeuralNetwork, where the label cache does
not exist yet (that same function sets m_labelCachePrebuilt = false
a few lines below), so MeanLifespan() returned its "nothing measured"
default of 1.0 at every call. Every fresh model was sized as though
its labels did not overlap - over-budgeting the first dense layer by
a factor of L, which is several rungs of a power-of-two ladder. The
"expect overfitting, reduce the feature set or pool instruments"
warning is the branch that should fire on H1 and structurally could
not.
Fixed at the source rather than by reordering the boot sequence (the
prebuild is chunked across Train() calls and cannot complete inside
init): MeasureSwingGeometry() walks the ZigZag ONCE at init and
answers both questions from it - the median leg gives the window,
and the leg series gives the mean label lifespan analytically.
SwingPivotDirectionLabel resolves bar i when the SECOND pivot after
it commits, so a bar d bars before pivot P waits d + (the leg
leaving P); summed over every bar of every leg that is exactly the
mean the label walk accumulates.
That also closes the coherence gap the swing target opened: the
window was measured with a private +/-12-bar fractal while the label
aimed at ZigZag(12,5,3) pivots, so it was sized against a leg
distribution the label never used. One pivot source now, the
label's.
Also:
- ResetWeights() re-derives the shape. It rebuilt from the members a
history-starved init had pinned and re-saved them - so the "let
history download, then reset from the panel" advice in both
fallback warnings did nothing at all.
- The CAPACITY line prints the measured lifespan beside the one the
topology was sized for, and warns when they differ by more than a
ladder rung. That is the check that makes the estimator falsifiable.
- Topology reads the view's symbol, not _Symbol (latent for pooling).
- Unmeasured geometry defaults to HISTORY_BARS_FALLBACK, never 1.0:
under-sizing is recoverable, over-sizing silently is not.
Compile: 0 errors, 0 warnings (stage).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 21:21:05 -04:00
//--- The lifespan the TOPOLOGY was sized against, so the CAPACITY line can print it beside the one
//--- the labels actually measured. Those two agreeing is what makes the derived shape trustworthy.
double FeatureSwingLifespanEstimate ( void ) const { return m_topology . SwingLifespanEstimate ( ) ; }
double FeatureMeanLabelLifespan ( void ) const { return MeanLabelLifespan ( ) ; }
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
double FeatureEstimatedInSampleBarsRaw ( void ) const { return EstimatedInSampleBarsRaw ( ) ; }
bool FeatureFindConfirmedZigZagPivot ( const int fromIdx , int & pivotIdx , double & pivotPrice , bool & pivotIsLow )
{ return FindConfirmedZigZagPivot ( fromIdx , pivotIdx , pivotPrice , pivotIsLow ) ; }
//--- FEATURE-ROW CACHE (m_featureCache/m_featureCacheHasValue/m_featureCacheValid) - stays
//--- signal-owned (Labels.mqh ArrayResize()s it, Training.mqh ArrayInitialize()s it on a param
//--- change), reached element-by-element the same way BufferTempData() always indexed it.
int FeatureCacheSize ( void ) const { return ArraySize ( m_featureCacheHasValue ) ; }
bool FeatureCacheHasValue ( const int idx ) const { return m_featureCacheHasValue [ idx ] ; }
bool FeatureCacheIsValid ( const int idx ) const { return m_featureCacheValid [ idx ] ; }
double FeatureCacheAt ( const int flatIdx ) const { return m_featureCache [ flatIdx ] ; }
2026-08-25 22:51:50 -04:00
//--- Bulk cache-hit read - see IFeaturesView.mqh's declaration. One ArrayCopy against the same
//--- backing array FeatureCacheAt() indexes one element at a time.
void FeatureCacheBlock ( const int base , const int count , double & out [ ] ) const
{ ArrayCopy ( out , m_featureCache , 0 , base , count ) ; }
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
void FeatureCacheSetAt ( const int flatIdx , const double v ) { m_featureCache [ flatIdx ] = v ; }
2026-08-25 22:51:50 -04:00
//--- Bulk counterpart to FeatureCacheBlock() above, for the write side.
void FeatureCacheSetBlock ( const int base , const int count , const double & values [ ] )
{ ArrayCopy ( m_featureCache , values , base , 0 , MathMin ( count , ArraySize ( values ) ) ) ; }
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- Always set together at store time - see BufferTempData's original "ONLY SUCCESSES ARE
//--- CACHED" block.
void FeatureCacheMarkStored ( const int idx ) { m_featureCacheHasValue [ idx ] = true ; m_featureCacheValid [ idx ] = true ; }
ditch(features): remove the eight dead feature groups from the input matrix
RSI, MACD, Ichimoku and the five AD/Wyckoff indicators (CumulativeDelta,
ShorteningOfThrust, WyckoffEventStream, WyckoffFailedStructure,
WyckoffSignificantBarInversion). All eight inputs shipped false and each carries a
closed verdict: the three oscillators are the same patterns that measured at chance
as entries, and the Wyckoff family returned zero out-of-sample on five independent
instruments - which is what closed the context score.
RETRAIN-NEUTRAL, and this one is worth stating precisely because the change looks
larger than it is. Every removed group contributed `flag ? N : 0` to the input
width, and every flag was false, so the width was ALREADY zero for all eight: no
.nnw's input layer changes. On the fingerprints, UseRSI and the five AD flags were
hashed unconditionally and become literal 0 legacy slots (the convention the
m_focalGamma slot above them already uses); UseMACD/UseIchimoku were appended only
when enabled, so their segments simply never appear - byte-identical to every
fingerprint ever produced, since neither ever shipped on.
CADIndicatorTuner IS DELIBERATELY NOT SHRUNK. Its flat parameter array is persisted
inside every .nnw, and Unflatten() rejects a size mismatch by falling back to
constructor defaults - so dropping the dead fields would silently revert the tuned
MA period of every model on disk while keeping its trained weights. That is the
feature/weight mismatch this project has already paid for twice, and it is not
worth 200 lines. AD_TUNE_PARAM_COUNT stays 42, the dead slots are still written and
read, and AutoTune's ParamOwner gate now matches only owner 5 (MA) so nothing
searches them. The class comment says all of this at the declaration.
Also renamed ReInitADIndicators -> ReInitTunableIndicators: it rebuilds exactly one
indicator now, and a name saying "AD" for the MA handle is the kind of stale label
that gets believed later. Its release-AFTER-recreate ordering is untouched - that
is a documented fix, not bookkeeping.
Compile-verified in the stage copy: 0 errors, 0 warnings, against the same 0/0
baseline.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 09:21:03 -04:00
//--- Params just changed (ReInitTunableIndicators) or a dead handle was just recreated
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- (RepairDeadIndicatorHandles) - every cached row was computed against the OLD handle.
void FeatureCacheInvalidateAll ( void ) { ArrayInitialize ( m_featureCacheHasValue , false ) ; }
2026-08-25 22:51:50 -04:00
//--- SwingPivotDirectionLabel() (Labels.mqh) reads m_Close/m_ATR/m_zigZag directly - a repair
//--- that recreates any of those handles stales every label already resolved against the old
//--- one, the same way FeatureCacheInvalidateAll() stales the feature cache. Every reader gates
//--- on m_labelCacheHasValue (DataHasLabel()), so clearing it alone forces a full relabel.
void LabelCacheInvalidateAll ( void ) { ArrayInitialize ( m_labelCacheHasValue , false ) ; }
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- m_featureFailBlock/m_featureFailIdx/m_windowFailSlot/m_windowFailTotal stay signal-owned -
//--- Training.mqh reads all four directly for the pass-1 stall report.
void FeatureSetFailBlock ( const string block ) { m_featureFailBlock = block ; }
void FeatureSetFailIdx ( const int idx ) { m_featureFailIdx = idx ; }
void FeatureSetWindowFail ( const int slot , const int total ) { m_windowFailSlot = slot ; m_windowFailTotal = total ; }
//--- ResizeBuffers()/RefreshData() (Expert\AIBase\Features.mqh) size/refresh EVERY indicator
//--- together each bar, including the 10 CFeatureBuilder now owns - direct calls into the owned
//--- collaborator, no view needed (the signal always may call what it owns by value).
bool FeatureVolumesBufferResize ( const int n ) { return m_featureBuilder . VolumesBufferResize ( n ) ; }
void FeatureVolumesRefresh ( void ) { m_featureBuilder . VolumesRefresh ( ) ; }
bool FeatureMaBufferResize ( const int n ) { return m_featureBuilder . MaBufferResize ( n ) ; }
void FeatureMaRefresh ( void ) { m_featureBuilder . MaRefresh ( ) ; }
fix(reset): say what the reset actually did, per member and per file
The user reports "Delete & Reset Weights only wipes the first NN". I could not
find a code path that skips ensemble members, and I am not going to assert one:
the handler loops g_aiSignals[0..g_aiSignalCount), all four topologies register
unconditionally in OnInit, and SetIdentity gives each its own State\<id>\ folder
so the six deleted paths are genuinely distinct per member. What IS true is that
the whole success path was SILENT - six FileDelete calls per member printing only
on failure, and one chart-wide Alert - so a four-member reset and a one-member
reset produce byte-identical output. The symptom could be neither confirmed nor
refuted from a log. That is the defect I can fix today.
- COMPILED <timestamp> (__DATETIME__) beside the build tag. The hand-edited tag
had sat at scan-nofwd-v5 across a week of commits, so it could not answer the
question it exists for. The compile stamp cannot be forgotten. Tag bumped to
reset-census-v6.
- RegistryLine() (public): ID, active file path, common/local, era, deployed vs
training, ensemble index. The reset handler prints a numbered census of the
whole registry BEFORE the confirm dialog. If that says 1 on an AI_HYBRID chart
the fault is registration, not the reset - and RegisterAISignal already has a
loud MAX_AI_SIGNALS message for exactly that.
- The confirmation dialog now names the count, so a wrong registry is visible
before anything is deleted rather than after.
- ResetWeights prints one line per member: N deleted / N already absent / N
FAILED, plus a per-suffix breakdown. "absent" on a member that should have had
a .nnw is a completely different fault from "deleted"; they were identical.
- ResetWeights' return value was discarded. A member whose BuildFreshTopology
fails has had its files deleted and has no network - and the Alert still said
"weights reset". Counted now, with an INCOMPLETE alert when they disagree.
- Same for dbm.ResetDatabase(), whose bool was also dropped. The DB is one shared
file for every signal on the chart, so there is nothing per-member to loop -
the log now says that explicitly, since it is the question being asked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 17:57:26 -04:00
//--- ONE LINE OF IDENTITY, for the census Warrior_EA.mq5 prints before it acts on g_aiSignals[].
2026-08-22 00:24:45 -04:00
//--- This makes them different.
fix(reset): say what the reset actually did, per member and per file
The user reports "Delete & Reset Weights only wipes the first NN". I could not
find a code path that skips ensemble members, and I am not going to assert one:
the handler loops g_aiSignals[0..g_aiSignalCount), all four topologies register
unconditionally in OnInit, and SetIdentity gives each its own State\<id>\ folder
so the six deleted paths are genuinely distinct per member. What IS true is that
the whole success path was SILENT - six FileDelete calls per member printing only
on failure, and one chart-wide Alert - so a four-member reset and a one-member
reset produce byte-identical output. The symptom could be neither confirmed nor
refuted from a log. That is the defect I can fix today.
- COMPILED <timestamp> (__DATETIME__) beside the build tag. The hand-edited tag
had sat at scan-nofwd-v5 across a week of commits, so it could not answer the
question it exists for. The compile stamp cannot be forgotten. Tag bumped to
reset-census-v6.
- RegistryLine() (public): ID, active file path, common/local, era, deployed vs
training, ensemble index. The reset handler prints a numbered census of the
whole registry BEFORE the confirm dialog. If that says 1 on an AI_HYBRID chart
the fault is registration, not the reset - and RegisterAISignal already has a
loud MAX_AI_SIGNALS message for exactly that.
- The confirmation dialog now names the count, so a wrong registry is visible
before anything is deleted rather than after.
- ResetWeights prints one line per member: N deleted / N already absent / N
FAILED, plus a per-suffix breakdown. "absent" on a member that should have had
a .nnw is a completely different fault from "deleted"; they were identical.
- ResetWeights' return value was discarded. A member whose BuildFreshTopology
fails has had its files deleted and has no network - and the Alert still said
"weights reset". Counted now, with an INCOMPLETE alert when they disagree.
- Same for dbm.ResetDatabase(), whose bool was also dropped. The DB is one shared
file for every signal on the chart, so there is nothing per-member to loop -
the log now says that explicitly, since it is the question being asked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 17:57:26 -04:00
string RegistryLine ( void ) const
{
return StringFormat ( " %s | %s (%s) | era %d | %s%s " , ID , m_activeFileName ,
( m_activeFileCommon ? " common " : " local " ) , ( int ) m_eraCount ,
( m_trainingComplete ? " deployed " : " training " ) ,
( m_ensembleMember
? StringFormat ( " | ensemble member %d " , m_ensembleIndex ) : " | solo " ) ) ;
}
fix: flush the in-flight era on shutdown; sweep orphaned chart objects on attach
Chart objects live in the MT5 chart PROFILE, not in this EA's files.
They survive a terminal restart, a recompile, and deleting every
.nnw/.cfg/.stats/.arrows on disk. Only a deinit that RUNS TO COMPLETION
removes them - and MetaTrader force-terminates OnDeinit at roughly
4,500 ms, so a run killed mid-cleanup orphans them permanently with no
owner left to clean up after. That is the "deleted every file,
recompiled, restarted, old arrows and a stale panel still there"
report: nothing was wrong with the files and deleting them could not
have helped.
Both halves are fixed.
STOP OVERRUNNING THE BUDGET. OnDeinit used to finalise the in-flight
run (StopTraining -> FinalizeTrainRun: checkpoint restore, live-state
re-seed) and then write two full nets per chart. On four charts that is
the bulk of the budget, spent to preserve a PARTIAL era that was never
scored, never checkpointed and never deployable. FlushTrainRun()
discards it instead - drop the resumable bookkeeping, leave the net
neutral (unfreeze BN, flush the batch, batch size 1), skip the save -
and training resumes from the last completed era, which the era-end
save and the periodic autosave have already put on disk. What is
discarded is bounded by one era.
A CONVERGED model keeps the old finalise-and-save path: its weights can
carry online-learning updates made since the last era boundary, and for
a deployed model no further era boundary is coming to persist them.
MAKE CLEANUP SELF-HEALING. Every purge sat behind a branch - no model
loaded, sidecar missing - so the common paths returned leaving whatever
the previous instance stranded. LoadChartSignals now sweeps the arrow
namespace unconditionally before restoring, so the post-init chart
holds exactly what the sidecar holds whichever branch runs, and the
panel gets the same treatment before Create() (CAppDialog namespaces
its controls, so a killed Destroy strands the lot and the next attach
draws a second panel on the corpse).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 13:08:40 -04:00
//--- SHUTDOWN FLUSH: abandon an in-flight run instead of finishing it, and resume from the last
2026-08-22 00:24:45 -04:00
//--- COMPLETED, already-persisted era.
fix: flush the in-flight era on shutdown; sweep orphaned chart objects on attach
Chart objects live in the MT5 chart PROFILE, not in this EA's files.
They survive a terminal restart, a recompile, and deleting every
.nnw/.cfg/.stats/.arrows on disk. Only a deinit that RUNS TO COMPLETION
removes them - and MetaTrader force-terminates OnDeinit at roughly
4,500 ms, so a run killed mid-cleanup orphans them permanently with no
owner left to clean up after. That is the "deleted every file,
recompiled, restarted, old arrows and a stale panel still there"
report: nothing was wrong with the files and deleting them could not
have helped.
Both halves are fixed.
STOP OVERRUNNING THE BUDGET. OnDeinit used to finalise the in-flight
run (StopTraining -> FinalizeTrainRun: checkpoint restore, live-state
re-seed) and then write two full nets per chart. On four charts that is
the bulk of the budget, spent to preserve a PARTIAL era that was never
scored, never checkpointed and never deployable. FlushTrainRun()
discards it instead - drop the resumable bookkeeping, leave the net
neutral (unfreeze BN, flush the batch, batch size 1), skip the save -
and training resumes from the last completed era, which the era-end
save and the periodic autosave have already put on disk. What is
discarded is bounded by one era.
A CONVERGED model keeps the old finalise-and-save path: its weights can
carry online-learning updates made since the last era boundary, and for
a deployed model no further era boundary is coming to persist them.
MAKE CLEANUP SELF-HEALING. Every purge sat behind a branch - no model
loaded, sidecar missing - so the common paths returned leaving whatever
the previous instance stranded. LoadChartSignals now sweeps the arrow
namespace unconditionally before restoring, so the post-init chart
holds exactly what the sidecar holds whichever branch runs, and the
panel gets the same treatment before Create() (CAppDialog namespaces
its controls, so a killed Destroy strands the lot and the next attach
draws a second panel on the corpse).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 13:08:40 -04:00
bool FlushTrainRun ( void )
{
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
bool inFlight = ( m_trainRunActive | | m_eraResumePending | | m_labelPrebuildActive | | m_onlineLearning . SimRunActive ( ) ) ;
fix: flush the in-flight era on shutdown; sweep orphaned chart objects on attach
Chart objects live in the MT5 chart PROFILE, not in this EA's files.
They survive a terminal restart, a recompile, and deleting every
.nnw/.cfg/.stats/.arrows on disk. Only a deinit that RUNS TO COMPLETION
removes them - and MetaTrader force-terminates OnDeinit at roughly
4,500 ms, so a run killed mid-cleanup orphans them permanently with no
owner left to clean up after. That is the "deleted every file,
recompiled, restarted, old arrows and a stale panel still there"
report: nothing was wrong with the files and deleting them could not
have helped.
Both halves are fixed.
STOP OVERRUNNING THE BUDGET. OnDeinit used to finalise the in-flight
run (StopTraining -> FinalizeTrainRun: checkpoint restore, live-state
re-seed) and then write two full nets per chart. On four charts that is
the bulk of the budget, spent to preserve a PARTIAL era that was never
scored, never checkpointed and never deployable. FlushTrainRun()
discards it instead - drop the resumable bookkeeping, leave the net
neutral (unfreeze BN, flush the batch, batch size 1), skip the save -
and training resumes from the last completed era, which the era-end
save and the periodic autosave have already put on disk. What is
discarded is bounded by one era.
A CONVERGED model keeps the old finalise-and-save path: its weights can
carry online-learning updates made since the last era boundary, and for
a deployed model no further era boundary is coming to persist them.
MAKE CLEANUP SELF-HEALING. Every purge sat behind a branch - no model
loaded, sidecar missing - so the common paths returned leaving whatever
the previous instance stranded. LoadChartSignals now sweeps the arrow
namespace unconditionally before restoring, so the post-init chart
holds exactly what the sidecar holds whichever branch runs, and the
panel gets the same treatment before Create() (CAppDialog namespaces
its controls, so a killed Destroy strands the lot and the next attach
draws a second panel on the corpse).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 13:08:40 -04:00
m_trainingStopRequested = true ;
m_trainingPaused = false ;
//--- Drop the resumable bookkeeping WITHOUT calling FinalizeTrainRun: no checkpoint restore, no
//--- persist, no dtStudied advance. The next start re-derives all of it from the saved model.
m_trainRunActive = false ;
m_eraResumePending = false ;
m_haveOosCheckpoint = false ;
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
m_checkpointEra = -1 ; // the joint-checkpoint era stamp goes with the snapshot it describes
fix: flush the in-flight era on shutdown; sweep orphaned chart objects on attach
Chart objects live in the MT5 chart PROFILE, not in this EA's files.
They survive a terminal restart, a recompile, and deleting every
.nnw/.cfg/.stats/.arrows on disk. Only a deinit that RUNS TO COMPLETION
removes them - and MetaTrader force-terminates OnDeinit at roughly
4,500 ms, so a run killed mid-cleanup orphans them permanently with no
owner left to clean up after. That is the "deleted every file,
recompiled, restarted, old arrows and a stale panel still there"
report: nothing was wrong with the files and deleting them could not
have helped.
Both halves are fixed.
STOP OVERRUNNING THE BUDGET. OnDeinit used to finalise the in-flight
run (StopTraining -> FinalizeTrainRun: checkpoint restore, live-state
re-seed) and then write two full nets per chart. On four charts that is
the bulk of the budget, spent to preserve a PARTIAL era that was never
scored, never checkpointed and never deployable. FlushTrainRun()
discards it instead - drop the resumable bookkeeping, leave the net
neutral (unfreeze BN, flush the batch, batch size 1), skip the save -
and training resumes from the last completed era, which the era-end
save and the periodic autosave have already put on disk. What is
discarded is bounded by one era.
A CONVERGED model keeps the old finalise-and-save path: its weights can
carry online-learning updates made since the last era boundary, and for
a deployed model no further era boundary is coming to persist them.
MAKE CLEANUP SELF-HEALING. Every purge sat behind a branch - no model
loaded, sidecar missing - so the common paths returned leaving whatever
the previous instance stranded. LoadChartSignals now sweeps the arrow
namespace unconditionally before restoring, so the post-init chart
holds exactly what the sidecar holds whichever branch runs, and the
panel gets the same treatment before Create() (CAppDialog namespaces
its controls, so a killed Destroy strands the lot and the next attach
draws a second panel on the corpse).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 13:08:40 -04:00
m_labelPrebuildActive = false ;
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
m_onlineLearning . AbortSimIfActive ( ) ;
fix: flush the in-flight era on shutdown; sweep orphaned chart objects on attach
Chart objects live in the MT5 chart PROFILE, not in this EA's files.
They survive a terminal restart, a recompile, and deleting every
.nnw/.cfg/.stats/.arrows on disk. Only a deinit that RUNS TO COMPLETION
removes them - and MetaTrader force-terminates OnDeinit at roughly
4,500 ms, so a run killed mid-cleanup orphans them permanently with no
owner left to clean up after. That is the "deleted every file,
recompiled, restarted, old arrows and a stale panel still there"
report: nothing was wrong with the files and deleting them could not
have helped.
Both halves are fixed.
STOP OVERRUNNING THE BUDGET. OnDeinit used to finalise the in-flight
run (StopTraining -> FinalizeTrainRun: checkpoint restore, live-state
re-seed) and then write two full nets per chart. On four charts that is
the bulk of the budget, spent to preserve a PARTIAL era that was never
scored, never checkpointed and never deployable. FlushTrainRun()
discards it instead - drop the resumable bookkeeping, leave the net
neutral (unfreeze BN, flush the batch, batch size 1), skip the save -
and training resumes from the last completed era, which the era-end
save and the periodic autosave have already put on disk. What is
discarded is bounded by one era.
A CONVERGED model keeps the old finalise-and-save path: its weights can
carry online-learning updates made since the last era boundary, and for
a deployed model no further era boundary is coming to persist them.
MAKE CLEANUP SELF-HEALING. Every purge sat behind a branch - no model
loaded, sidecar missing - so the common paths returned leaving whatever
the previous instance stranded. LoadChartSignals now sweeps the arrow
namespace unconditionally before restoring, so the post-init chart
holds exactly what the sidecar holds whichever branch runs, and the
panel gets the same treatment before Create() (CAppDialog namespaces
its controls, so a killed Destroy strands the lot and the next attach
draws a second panel on the corpse).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 13:08:40 -04:00
//--- Leave the net in the same neutral state FinalizeTrainRun leaves it in - a frozen batch-norm or
//--- a half-filled mini-batch must not be what a later inference path finds. Cheap, unlike the save.
if ( CheckPointer ( Net ) ! = POINTER_INVALID )
{
Net . SetBatchNormFrozen ( false ) ;
Net . FlushBatch ( ) ;
Net . SetBatchSize ( 1 ) ;
}
return inFlight ;
}
2026-07-14 22:36:27 -04:00
void StopTraining ( void )
{
m_trainingStopRequested = true ;
m_trainingPaused = false ;
//--- ScheduleTrainingIfNeeded() refuses to schedule another "New Bar" event while
//--- m_trainingStopRequested is set, so a run interrupted mid-chunk would otherwise never get
2026-08-22 00:24:45 -04:00
//--- called again to finalize (restore the best checkpoint, persist state) - do it
//--- synchronously here instead.
2026-07-14 22:36:27 -04:00
if ( m_trainRunActive )
FinalizeTrainRun ( ) ;
Print ( ID + " : training stopped by user (era " + IntegerToString ( m_eraCount ) + " , weights as of last completed era retained) " ) ;
diag: inference-path census, to explain zero-trade backtests
A backtest of the CONVERGED CONV model produced "Final directional result:
0.00000000" on every one of 1744 bars and therefore zero trades. Nothing in
the log could separate the three candidate causes, and each needs a
different fix:
1. RefreshLatestSignal never called (new-bar gate never fires)
2. called, but bailing at one of its two early returns
3. running fine, and the model genuinely answers Neutral every bar
Counts all three plus the Buy/Sell/Neutral split, printed once at shutdown
via StopTraining (which the tester reaches through OnDeinit). Three
increments per bar against a full feedForward - not worth gating.
Ruled out while writing this, so the next session does not re-derive it:
- the alternation gate (m_lastNonNeutralSignal) is NOT the cause. It starts
at Neutral, so a first Buy would still fire and show up as one non-zero
direction. We saw zero. It IS still a live hazard for a one-sided model -
CONV currently calls Buy:17% Sell:0%, and after the first Buy every later
Buy is suppressed until a Sell that never comes - but it cannot explain
an all-zero run.
- shallow buffers do not hard-fail the feature builder: the swing-context
Donchian loop breaks gracefully when it runs off loaded history. It does
mean converged-path inference computes Donchian/return/SMA features over
a TRUNCATED window versus training, which is a real train/inference skew
worth its own fix, but it degrades features rather than zeroing them.
Both builds 0/0. Diagnostic only.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 18:24:32 -04:00
PrintInferenceTally ( ) ;
}
2026-08-22 00:24:45 -04:00
//--- Inference-path census, printed at shutdown. Each implies a completely different fix.
//--- Counting is the cheapest way to tell them apart and it costs nothing per bar.
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets
The 31:1 class imbalance was self-inflicted by the TARGET, not a property
of the market. Labelling only the exact bar where a ZigZag pivot confirms
gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism
this codebase accumulated sits downstream of that one choice: the
logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias
seed, balanced-accuracy-then-precision selection with its coverage floor,
the recall floor and its catch-22, the alternation gate, NMS, and the four
oversampling designs that collapsed before them.
The reference this engine is built on (references/neuronetworksbook.pdf
ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT
EXTREMUM on every bar - ~50/50 by construction, with no imbalance to
correct at all. It never had this problem because it never asked "is this
the pivot bar".
Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's
OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its
target before its stop, within a horizon. Buy = long resolves, Sell =
short resolves, Neutral = neither. Consequences:
- dir-precision in the era line stops being a proxy and becomes the win
rate of the strategy under its own exit rules.
- Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e.
~2:1 instead of 31:1. Measured and logged at the end of the prebuild.
- Spread is charged on both legs, so it is a NET win rate.
- Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches
inside one bar and the optimistic reading is how a backtested edge
becomes a live loss.
ZigZag stays as input features (EnableSwingContext) and now also supplies
the vertical barrier: the horizon is the median confirmed leg length,
snapped to a coarse ladder. Derived, not configured, and deliberately kept
out of the filename fingerprint - a filename keyed on a measured quantity
orphans a trained model the moment the measurement moves.
Removed, because the premise died with the old target:
- the alternation gate. Correct for pivot labels (a ZigZag cannot emit two
same-type pivots in a row, so a repeat was provably a false fire), and
wrong for barrier labels, which answer each bar independently. It also
took its worst consequence with it: a one-sided model previously got ONE
trade per backtest, a hard blocker on marketplace validation.
- SignalClusterWindow now defaults off - it de-duplicated repeats that are
now real trades. Kept as an opt-in display control.
- LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel.
- the era-0 output-bias seed now needs a genuinely dominant class (0.70)
rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a
correction.
Also fixed, both found while wiring the above:
1. RefreshConvergedSignal sized its buffers from a date delta
(Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training
watermark; in the tester it is loaded from a live-chart save AHEAD of
the simulated date, so the interval inverted, Bars() returned ~0, and
the buffer came out at exactly m_historyBars - deep enough for the OHLC
window and far too shallow for the Donchian-50 / 20-bar-return / SMA
extension behind it. Inference silently computed DIFFERENT features
from the ones training learned on, live as well as in the tester. Now
sized from what the feature builder actually needs.
2. The barrier horizon is resolved on the deployed path too. A deployed
model never enters Train(), so it never reached the prebuild, and
OnlineLearnStep reads the horizon as its confirmation delay - left at
the fallback it would have backpropped bars whose barriers had not
resolved. Silent lookahead in the one place that writes to a live model.
SL_Mode/TP_Mode join the weights fingerprint: they define the labels now,
so a model trained at 1:3 must never be silently reused at 1:1. This
re-keys every pre-existing model by design - none were trained on this task.
Inference census extended with the vote gate. LongCondition/ShortCondition
open with a readiness check the refresh counters never see; in the tester it
reduces to "the seeded _optcache.nnw must have LOADED", and if it did not,
every vote is hard-zeroed while the model still answers Buy. The old three
counters would have read that as "the model says Neutral" - false, and a
completely different fix. This is the leading candidate for the
zero-direction backtest and the census can now name it in one run.
Both builds compile 0 errors / 0 warnings. Forces a full retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 20:39:49 -04:00
void NoteVoteGate ( bool directional )
{
if ( ! directional )
return ;
bool open = m_trainingComplete | | ( m_inferenceOnly & & m_modelLoadedFromDisk ) ;
if ( m_voteGateCompleteAtFirst < 0 )
{
m_voteGateCompleteAtFirst = ( int ) m_trainingComplete ;
m_voteGateLoadedAtFirst = ( int ) m_modelLoadedFromDisk ;
}
if ( open )
m_voteGatePassed + + ;
else
m_voteGateBlocked + + ;
}
diag: inference-path census, to explain zero-trade backtests
A backtest of the CONVERGED CONV model produced "Final directional result:
0.00000000" on every one of 1744 bars and therefore zero trades. Nothing in
the log could separate the three candidate causes, and each needs a
different fix:
1. RefreshLatestSignal never called (new-bar gate never fires)
2. called, but bailing at one of its two early returns
3. running fine, and the model genuinely answers Neutral every bar
Counts all three plus the Buy/Sell/Neutral split, printed once at shutdown
via StopTraining (which the tester reaches through OnDeinit). Three
increments per bar against a full feedForward - not worth gating.
Ruled out while writing this, so the next session does not re-derive it:
- the alternation gate (m_lastNonNeutralSignal) is NOT the cause. It starts
at Neutral, so a first Buy would still fire and show up as one non-zero
direction. We saw zero. It IS still a live hazard for a one-sided model -
CONV currently calls Buy:17% Sell:0%, and after the first Buy every later
Buy is suppressed until a Sell that never comes - but it cannot explain
an all-zero run.
- shallow buffers do not hard-fail the feature builder: the swing-context
Donchian loop breaks gracefully when it runs off loaded history. It does
mean converged-path inference computes Donchian/return/SMA features over
a TRUNCATED window versus training, which is a real train/inference skew
worth its own fix, but it degrades features rather than zeroing them.
Both builds 0/0. Diagnostic only.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 18:24:32 -04:00
void PrintInferenceTally ( void )
{
long attempts = m_refreshOk + m_refreshFailFeatures + m_refreshFailShort ;
if ( attempts < = 0 )
{
Print ( ID + " : inference census - RefreshLatestSignal was NEVER CALLED (0 attempts). The new-bar gate never fired. " ) ;
return ;
}
Print ( ID + " : inference census - " , attempts , " refresh attempts: " , m_refreshOk , " completed, " ,
m_refreshFailFeatures , " bailed in BufferTempData, " , m_refreshFailShort , " bailed on a short feature window " ,
" | decisions Buy: " , m_refreshBuy , " Sell: " , m_refreshSell , " Neutral: " , m_refreshNeutral ) ;
2026-08-22 00:24:45 -04:00
//--- Second half of the census, and the half that separates "the model said nothing" from
//--- "the model spoke and was not allowed to vote" - see m_voteGateBlocked for why that
//--- distinction is the whole point.
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets
The 31:1 class imbalance was self-inflicted by the TARGET, not a property
of the market. Labelling only the exact bar where a ZigZag pivot confirms
gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism
this codebase accumulated sits downstream of that one choice: the
logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias
seed, balanced-accuracy-then-precision selection with its coverage floor,
the recall floor and its catch-22, the alternation gate, NMS, and the four
oversampling designs that collapsed before them.
The reference this engine is built on (references/neuronetworksbook.pdf
ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT
EXTREMUM on every bar - ~50/50 by construction, with no imbalance to
correct at all. It never had this problem because it never asked "is this
the pivot bar".
Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's
OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its
target before its stop, within a horizon. Buy = long resolves, Sell =
short resolves, Neutral = neither. Consequences:
- dir-precision in the era line stops being a proxy and becomes the win
rate of the strategy under its own exit rules.
- Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e.
~2:1 instead of 31:1. Measured and logged at the end of the prebuild.
- Spread is charged on both legs, so it is a NET win rate.
- Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches
inside one bar and the optimistic reading is how a backtested edge
becomes a live loss.
ZigZag stays as input features (EnableSwingContext) and now also supplies
the vertical barrier: the horizon is the median confirmed leg length,
snapped to a coarse ladder. Derived, not configured, and deliberately kept
out of the filename fingerprint - a filename keyed on a measured quantity
orphans a trained model the moment the measurement moves.
Removed, because the premise died with the old target:
- the alternation gate. Correct for pivot labels (a ZigZag cannot emit two
same-type pivots in a row, so a repeat was provably a false fire), and
wrong for barrier labels, which answer each bar independently. It also
took its worst consequence with it: a one-sided model previously got ONE
trade per backtest, a hard blocker on marketplace validation.
- SignalClusterWindow now defaults off - it de-duplicated repeats that are
now real trades. Kept as an opt-in display control.
- LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel.
- the era-0 output-bias seed now needs a genuinely dominant class (0.70)
rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a
correction.
Also fixed, both found while wiring the above:
1. RefreshConvergedSignal sized its buffers from a date delta
(Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training
watermark; in the tester it is loaded from a live-chart save AHEAD of
the simulated date, so the interval inverted, Bars() returned ~0, and
the buffer came out at exactly m_historyBars - deep enough for the OHLC
window and far too shallow for the Donchian-50 / 20-bar-return / SMA
extension behind it. Inference silently computed DIFFERENT features
from the ones training learned on, live as well as in the tester. Now
sized from what the feature builder actually needs.
2. The barrier horizon is resolved on the deployed path too. A deployed
model never enters Train(), so it never reached the prebuild, and
OnlineLearnStep reads the horizon as its confirmation delay - left at
the fallback it would have backpropped bars whose barriers had not
resolved. Silent lookahead in the one place that writes to a live model.
SL_Mode/TP_Mode join the weights fingerprint: they define the labels now,
so a model trained at 1:3 must never be silently reused at 1:1. This
re-keys every pre-existing model by design - none were trained on this task.
Inference census extended with the vote gate. LongCondition/ShortCondition
open with a readiness check the refresh counters never see; in the tester it
reduces to "the seeded _optcache.nnw must have LOADED", and if it did not,
every vote is hard-zeroed while the model still answers Buy. The old three
counters would have read that as "the model says Neutral" - false, and a
completely different fix. This is the leading candidate for the
zero-direction backtest and the census can now name it in one run.
Both builds compile 0 errors / 0 warnings. Forces a full retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 20:39:49 -04:00
if ( m_voteGateCompleteAtFirst < 0 )
Print ( ID + " : inference census - vote gate was NEVER REACHED (no directional decision ever hit "
" LongCondition/ShortCondition). Either every decision was Neutral, or this filter was never polled. " ) ;
else
Print ( ID + " : inference census - vote gate passed: " , m_voteGatePassed , " blocked: " , m_voteGateBlocked ,
" | at first vote trainingComplete= " , ( m_voteGateCompleteAtFirst ! = 0 ? " true " : " false " ) ,
" modelLoadedFromDisk= " , ( m_voteGateLoadedAtFirst ! = 0 ? " true " : " false " ) ,
" inferenceOnly= " , ( m_inferenceOnly ? " true " : " false " ) ,
( m_voteGateBlocked > 0 & & m_voteGatePassed = = 0
? " <-- EVERY directional call was discarded here. This is the zero-direction cause. "
: " " ) ) ;
2026-07-14 22:36:27 -04:00
}
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
//--- The ONLY place the study event is posted: arms bEventStudy with THIS instance's id (so the
//--- handler in OnChartEventHandler(), which matches on m_studyEventId, is the only member that
//--- runs it) and stamps the lost-event watchdog. sparam tags ("New Bar"/"Init"/"Resume"/...) are
//--- purely diagnostic.
bool ArmStudyEvent ( const long lparam , const string tag )
{
bEventStudy = EventChartCustom ( ChartID ( ) , m_studyEventId , lparam , 0 , tag ) ;
if ( bEventStudy )
m_studyArmedTick = GetTickCount ( ) ;
return bEventStudy ;
}
2026-07-14 22:36:27 -04:00
void StartTraining ( void )
{
if ( ! m_trainingStopRequested & & ! m_trainingPaused )
return ;
m_trainingStopRequested = false ;
m_trainingPaused = false ;
if ( ! bEventStudy )
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
ArmStudyEvent ( ( long ) dtStudied , " Resume " ) ;
2026-07-14 22:36:27 -04:00
Print ( ID + " : training (re)started by user (era " + IntegerToString ( m_eraCount ) + " ) " ) ;
}
2026-07-25 16:39:11 -04:00
//--- Has an era ever cleared the per-class recall floor and been checkpointed this run? This is the
//--- same quality bar the plateau ladder's auto-deploy requires (see PLATEAU_STAGE_DEPLOY), exposed so
//--- the panel can warn before a MANUAL deploy ships a model that ignores Buy or Sell.
bool HasRecallPassingCheckpoint ( void ) const { return m_bestPassedRecall ; }
2026-08-22 00:24:45 -04:00
//--- MANUAL deploy (panel "Deploy Model"): finalise whatever the run has found so far as THE
//--- model - exactly what the plateau ladder does on its own at stage 3, just triggered early by
//--- the operator. Reversible via RetrainDeployed().
2026-07-25 16:39:11 -04:00
bool DeployNow ( void )
{
if ( CheckPointer ( Net ) = = POINTER_INVALID | | ! m_isInitialized )
return false ;
if ( m_trainingComplete )
return true ; // already deployed - nothing to do
//--- Set BEFORE any save below: the flag is written INTO the .nnw, so persisting first would store
//--- "still training" and a restart would resume the era loop instead of running the deployed model.
m_trainingComplete = true ;
m_trainingPaused = false ;
m_trainingStopRequested = false ;
if ( m_trainRunActive | | m_haveOosCheckpoint )
FinalizeTrainRun ( ) ; // restores the best checkpoint, persists, ends the run
else
{
//--- Nothing trained this session (e.g. deploying a model that was just loaded from disk), so
//--- there is no in-memory checkpoint to restore - persist exactly what is loaded right now.
PersistDeployedModel ( ) ;
SaveChartSignals ( ) ;
}
RefreshLatestSignal ( ) ;
Print ( ID + " : model DEPLOYED by user at era " + IntegerToString ( m_eraCount ) +
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
" (selection score " + ( m_bestSelectionScore < 0 ? " n/a " : DeployScoreText ( m_bestSelectionScore ) ) +
2026-07-25 16:39:11 -04:00
" , blended OOS " + DoubleToString ( dOosForecast , 1 ) + " %) - training stopped, now running live inference " +
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
( OnlineEnableLearningFlag ( ) ? " with online continual learning " : " " ) +
2026-07-25 16:39:11 -04:00
" . Use the panel's \" Retrain Model \" to resume training from here. " ) ;
2026-08-22 00:24:45 -04:00
//--- Deliberately reported, not enforced: a manual deploy is the operator overriding the
//--- ladder, and that override stays available. See DEPLOY_FAMILY_WISE_ALPHA and
//--- HasRecallPassingCheckpoint()'s panel warning.
feat: gate deployment on the null of the MAXIMUM, not the per-era null
EDGE_MIN_SIGMAS is a PER-ERA test and the deployed model is the MAXIMUM
over every era a run ranks. A 2-sigma one-sided test passes on noise with
probability 0.0228 per era, so over N eras the chance at least one clears
it is 1-(1-0.0228)^N: 34% by era 18, 80% by era 70, 93% by era 112. The
gate was near-certain to open on a long run whatever the data held.
It did. HYBRID deployed 2026-08-08 at dir-precision 35.5% vs 34% chance -
+1.5pp, best of 112 eras whose per-era values wandered 30%..35.5%. At the
call counts these runs produce that is p_family 0.92..0.9999.
Every OTHER best-of-N decision here already carries this correction, and
every one REJECTS on this data: the barrier-geometry winner (null of the
maximum over 6, p=0.3902), the indicator tuner (Sidak, p=1.0000), the MI
lag profile (null of the maximum over 21 lags). The one decision that
ships a model to a live account had none.
BestCheckpointSurvivesSelection() re-tests the checkpoint that is about to
deploy:
z = (precision - chance)/SE, SE = sqrt(p0(1-p0)/n)
p_single = P(Z >= z)
p_family = 1 - (1-p_single)^N
against DEPLOY_FAMILY_WISE_ALPHA. It uses the checkpoint's OWN
snapshotted precision/chance/call-count, not the latest era's, because
the model that ships is the one that has to clear the bar.
N counts CANDIDATE eras (coverage measurable, at least one directional
call) - an era that called nothing directional could never have become
the best, so counting it would make the gate stricter than the search
that actually happened.
Conservative on purpose: consecutive eras share OOS bars and differ by
one gradient step, so they are nowhere near N independent draws and the
true family-wise error is below this bound. This gate decides what trades
real money and the house posture is reject-unless-demonstrated.
Effect at 2900 directional calls / N=112: required edge goes 1.76pp ->
2.92pp. A real edge clears it; +1.5pp does not.
Applied to BOTH automatic paths - the plateau ladder's stage-3 deploy and
the m_trainingComplete assignment - which must stay identical or the flag
persisted into the .nnw disagrees with the decision to stop, and a reload
runs inference on a model the ladder refused.
NOT applied to the two operator paths (era-cap deploy, panel Deploy
button). Those stay the operator's call; ReportSelectionGateVerdict()
logs the verdict beside them so an authorised deploy can never later be
misread as a validated one.
NormalUpperTail() is A&S 26.2.17 (|err| < 7.5e-8), self-contained rather
than pulling in Math\Stat. Verified against reference values to 6dp:
Q(1.645)=0.049985, Q(1.96)=0.024998, Q(3.0)=0.001350. Its locals are
ntB1..ntB5 because AI\Network.mqh line 79 does "#define b1 AdamBeta1".
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 18:01:04 -04:00
ReportSelectionGateVerdict ( " manual deploy " ) ;
2026-07-25 16:39:11 -04:00
return true ;
}
//--- The inverse of DeployNow(), and the ONLY way back: while m_trainingComplete is set,
2026-08-22 00:24:45 -04:00
//--- ScheduleTrainingIfNeeded() routes every tick to the converged/inference branch, so
//--- StartTraining() alone can never revive a deployed model (it clears the stop flag, but the
//--- complete flag still wins that branch).
2026-07-25 16:39:11 -04:00
void RetrainDeployed ( void )
{
if ( ! m_trainingComplete )
return ;
m_trainingComplete = false ;
m_trainingStopRequested = false ;
m_trainingPaused = false ;
//--- Persist the cleared flag immediately. Otherwise a terminal restart before the first era
//--- completes would reload the .nnw still marked complete and silently go back to inference-only,
//--- looking like the button did nothing.
PersistDeployedModel ( ) ;
if ( ! bEventStudy )
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
ArmStudyEvent ( ( long ) dtStudied , " Retrain " ) ;
2026-07-25 16:39:11 -04:00
Print ( ID + " : RETRAINING the deployed model from era " + IntegerToString ( m_eraCount ) +
" - keeping its current weights as the starting point (use \" Delete & Reset Weights \" to start from scratch instead). " ) ;
}
2026-07-26 12:36:56 -04:00
//--- Manual "rescan" of the drawn signal arrows: purges every arrow currently on the chart (namespaced
//--- delete - user drawings untouched) and re-infers the last SIGNAL_RESCAN_LOOKBACK_BARS bars from the
//--- CURRENTLY deployed weights, then re-runs the same end-of-era NMS declutter (PruneDirectionalClusters)
//--- used during training so the fresh set matches what a live re-render would have produced. Wired to
//--- the panel's Hide->Show Signals sequence: without this, "restore" only ever replays whatever was
//--- last saved to the .arrows sidecar, which for a long-deployed model can be a stale historical render
//--- from whenever it was last actually trained - years-old arrows crowding out anything recent. Chart-only
//--- (no persistent chart in the tester/optimizer) and a no-op until a model has something to infer with.
2026-07-26 12:52:56 -04:00
//--- This only does the cheap setup (buffer resize, arrow purge, cache alloc) and QUEUES the per-bar
//--- inference loop for AdvanceChartSignalRescan() to drain in time-boxed slices off the timer - see
//--- that method's comment for why the loop itself must never run in one blocking pass. Returns true
//--- once a rescan has been queued (check RescanPending() for completion), false if there was nothing
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
//--- to rescan (no deployed model, tester/optimizer context, etc). Body: Expert\Chart\ChartUI.mqh -
//--- it manipulates the exact same rescan queue/tally AdvanceChartSignalRescan drains, so the two
//--- halves of this state machine now live on the one object that owns the state.
bool StartChartSignalRescan ( void ) { return m_chartUI . StartChartSignalRescan ( ) ; }
2026-07-26 12:55:31 -04:00
//--- true while a queued rescan (StartChartSignalRescan above) still has slices left for
//--- AdvanceChartSignalRescan to drain - polled by Warrior_EA.mq5's FinalizeSignalsRescanIfDone() to
//--- know when it's safe to (re)apply arrow visibility and report the Show Signals click as complete.
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
bool RescanPending ( void ) const { return m_chartUI . RescanPending ( ) ; }
2026-07-14 22:36:27 -04:00
//--- forces a save of the network's current in-memory weights/state regardless of era-completion
2026-08-22 00:24:45 -04:00
//--- state; called from OnDeinit() so shutdown/chart-removal never loses more than the current
//--- tick of learning, and a subsequent restart's Train() resumes from m_eraCount rather than
//--- the last fully-completed era only.
2026-07-24 21:56:53 -04:00
bool PersistWeightsOnShutdown ( void )
2026-07-14 22:36:27 -04:00
{
if ( CheckPointer ( Net ) = = POINTER_INVALID | | ! m_isInitialized )
return false ;
2026-08-22 00:24:45 -04:00
//--- An inference-only run (any Strategy Tester pass - see m_inferenceOnly) trains NOTHING,
//--- so there is no new state to persist and this save can only do harm.
2026-07-26 10:59:46 -04:00
if ( m_inferenceOnly )
{
PrintVerbose ( ID + " : inference-only run - skipping the shutdown weight save (nothing was trained; the cached model is left exactly as seeded). " ) ;
return true ;
}
2026-08-22 00:24:45 -04:00
//--- Nothing trained and nothing loaded => there is no state to persist, and writing anyway
//--- is actively harmful.
fix(ai): stop the shutdown save from resurrecting reset weights; size HYBRID's LSTM to its real fan-in
ResetWeights already deletes the whole model set - .nnw, .cfg, _ckpt.tmp,
.stats, _shadow.nnw - and clears both the .arrows sidecar and the drawn
chart objects. What undid it was PersistWeightsOnShutdown: detaching the EA
after a reset but before an era completed re-created a .nnw from the
freshly-built, never-run net, so the next attach loaded an era-0 stub
instead of starting clean. For LSTM/HYBRID that stub is worse than nothing -
a layer that has never run a forward pass has m_iInputs<=0, so Save omits
every LSTM buffer (see 413ff7e). Skip the save when no era completed and no
model was loaded; that is exactly the post-reset and first-attach state.
Also sweep _shadowclone.tmp, which the reset did not cover.
Separately, ComputeLstmHiddenSize budgeted every topology against the
flattened input (historyBars x neuronsCount). True for LSTM, wrong for
HYBRID, where AddConvStage runs first and the LSTM is fed the conv feature
map - historyBars x convFilterCount, 160 rather than 420 at H1 defaults.
The quadratic is dominated by the inputs term, so overstating the fan-in
2.6x cost a full ladder step (16 units where the budget affords 32). New
virtual HasConvBeforeLstm() feeds LstmFanIn(), so composition decides this
rather than an AIType check. desc.window is advisory only - CNet never
passes it to the layer - but is now truthful for the same reason.
Derived values stay out of the weights-filename fingerprint and are adopted
from the .cfg, so existing models keep their saved width; only fresh ones
pick up the corrected budget.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 13:06:09 -04:00
if ( m_eraCount = = 0 & & ! m_modelLoadedFromDisk )
{
PrintVerbose ( ID + " : no era completed and no model loaded - skipping the shutdown weight save (leaving the model files absent so the next attach starts genuinely clean). " ) ;
return true ;
}
perf(deinit): I/O-free chart cleanup, dead-panel purge, skip clean weight saves
The 18:23 terminal close (20260825.log) killed two of six charts inside
OnDeinit: they printed "shutting down" then nothing for 5.9 s until
"Abnormal termination", stranding ~700 objects each - including the one
family no prefix sweep can reach, the control panel (CAppDialog names
its 15 objects <numeric instance id><control>, and a re-attach mints a
new id, so a killed panel is a permanent ghost; XTIUSD carried one
across sessions). The stall sat in the two file writes that preceded
all visible cleanup while the four sibling charts flooded the same
2013-era disk - the ~4x18MB-per-chart shutdown weight saves.
Three changes:
1. OnDeinit touches no file until the chart is clean. CVoteArrowStore
splits Save() into Snapshot() (the chart scan, in memory) and
WriteSnapshot() (the disk half, consuming). New order: status label,
vote-arrow snapshot, prefix sweep, panel destroy - all object ops -
then member sidecars, final sweep, timings, and only then the
visibility file, the vote-arrow write and the weight saves.
2. PurgeOrphanedPanelObjects() at OnInit: deletes numeric-prefix
CAppDialog ghosts by name (6 chrome + 9 buttons), qualifying a
prefix only when >=4 of OUR button names carry it, so a foreign
dialog sharing stock chrome names is never touched.
3. m_netDirty: set by every net mutation (both backProp sites, both
RestoreWeights sites, online learning conservatively, panel reset),
cleared only on a successful Net.Save. Shutdown AND the per-bar
autosave now skip the ~18MB write when the net is provably unchanged
- for converged ensembles that is every save - which removes the
very flood that starved the sibling charts. .stats still writes
every time (small; carries the vote record and calibration). A
skipped save leaves the .nnw header dtStudied stale, which is the
already-handled attach-after-offline-gap case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 18:46:51 -04:00
//--- SKIP THE WEIGHT WRITE WHEN NOTHING CHANGED IT - see m_netDirty's declaration. For a
//--- converged model with no online learning this is every save after the first, and it is
//--- what keeps a six-chart terminal close from writing ~400MB of byte-identical .nnw files
//--- into each other's OnDeinit budget. .stats is still written below: it is small and
//--- carries state (the ensemble vote record, calibration) that changes without a weight.
bool ok = true ;
bool wrote = m_netDirty ;
if ( m_netDirty )
{
double currentIndicatorParams [ ] ;
m_indicatorTuner . Flatten ( currentIndicatorParams ) ;
ok = Net . Save ( m_activeFileName + " .nnw " , dError , dUndefine , dForecast , dtStudied , m_activeFileCommon , m_eraCount , m_trainingComplete , currentIndicatorParams ) ;
if ( ok )
m_netDirty = false ;
}
else
PrintVerbose ( ID + " : weights unchanged since their last save - skipped the .nnw write (stats still saved). " ) ;
2026-07-23 19:36:34 -04:00
//--- calibration state (class priors + confidence scale) must travel with the weights so live
//--- trading behaves like training after a restart - see SaveModelStats().
2026-07-26 12:12:14 -04:00
if ( ! SaveModelStats ( m_activeFileName , m_activeFileCommon ) )
Print ( ID + " : ERROR - shutdown SaveModelStats failed for " + m_activeFileName + " . Calibration state not persisted. " ) ;
2026-08-22 00:24:45 -04:00
//--- Deliberately do NOT save the shadow net here. Worst case a shutdown loses only the
//--- shadow's in-progress-era drift, which re-converges - a far better trade than risking the
//--- whole model to an over-budget shutdown.
2026-07-14 22:36:27 -04:00
if ( ! ok )
Print ( ID + " : ERROR - failed to persist weights on shutdown for " + m_activeFileName + " , error " + IntegerToString ( GetLastError ( ) ) ) ;
else
perf(deinit): I/O-free chart cleanup, dead-panel purge, skip clean weight saves
The 18:23 terminal close (20260825.log) killed two of six charts inside
OnDeinit: they printed "shutting down" then nothing for 5.9 s until
"Abnormal termination", stranding ~700 objects each - including the one
family no prefix sweep can reach, the control panel (CAppDialog names
its 15 objects <numeric instance id><control>, and a re-attach mints a
new id, so a killed panel is a permanent ghost; XTIUSD carried one
across sessions). The stall sat in the two file writes that preceded
all visible cleanup while the four sibling charts flooded the same
2013-era disk - the ~4x18MB-per-chart shutdown weight saves.
Three changes:
1. OnDeinit touches no file until the chart is clean. CVoteArrowStore
splits Save() into Snapshot() (the chart scan, in memory) and
WriteSnapshot() (the disk half, consuming). New order: status label,
vote-arrow snapshot, prefix sweep, panel destroy - all object ops -
then member sidecars, final sweep, timings, and only then the
visibility file, the vote-arrow write and the weight saves.
2. PurgeOrphanedPanelObjects() at OnInit: deletes numeric-prefix
CAppDialog ghosts by name (6 chrome + 9 buttons), qualifying a
prefix only when >=4 of OUR button names carry it, so a foreign
dialog sharing stock chrome names is never touched.
3. m_netDirty: set by every net mutation (both backProp sites, both
RestoreWeights sites, online learning conservatively, panel reset),
cleared only on a successful Net.Save. Shutdown AND the per-bar
autosave now skip the ~18MB write when the net is provably unchanged
- for converged ensembles that is every save - which removes the
very flood that starved the sibling charts. .stats still writes
every time (small; carries the vote record and calibration). A
skipped save leaves the .nnw header dtStudied stale, which is the
already-handled attach-after-offline-gap case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 18:46:51 -04:00
if ( wrote )
PrintVerbose ( ID + " : weights persisted on shutdown (era " + IntegerToString ( m_eraCount ) + " , trainingComplete= " + ( string ) m_trainingComplete + " ) " ) ;
2026-07-24 21:56:53 -04:00
return ok ;
}
2026-08-22 00:24:45 -04:00
//--- Persist the drawn arrows to disk, then remove THIS EA's chart visuals (arrows + status
//--- label). Called early in OnDeinit(), before the heavy weight save, so a later stall/fault in
//--- the save can never leave the chart littered. Deliberately NOT part of SaveWeightsNow(): a
//--- mid-session manual save must not wipe the chart.
fix(chart): arrows survived the EA that drew them - persist, then clear
Reported: on deinit the panel and status label go, the signal arrows stay.
Two independent causes, both fixed here.
1. It was partly deliberate. ShutdownChartCleanup carried a second
behaviour selected by a `preserveChartArrows` flag derived from the
deinit reason: on RECOMPILE / PARAMETERS / CHARTCHANGE / TEMPLATE the
arrows were left on the chart on purpose, to avoid a reload flicker.
That branch IS the reported symptom, an operator cannot tell it apart
from a cleanup that failed, and it was outright wrong whenever the
reload changed the config - REASON_PARAMETERS means exactly that, and
the preserved arrows then belonged to a model the chart no longer
runs, with nothing marking them stale. It is gone, along with the flag
and m_purgeChartOnDestruct. One path now: persist, clear, restore on
the next attach.
2. Whatever remains was unfalsifiable. PurgeChart was a single
ObjectsDeleteAll(prefix) whose return value was discarded, with no
caller ever looking at the chart again - so "the arrows are still
there" and "the arrows were never there" produced identical evidence,
which is why the report survived three sessions. It now verifies:
after the bulk delete it walks the OBJ_ARROW-typed list (a handful of
objects, not the whole chart), deletes any surviving WarSig_ by name,
and says so. Costs one typed scan when the bulk delete works, which is
the normal case; names the root cause when it does not.
Every failure mode of SaveChartSignals was also silent - it returned void
and had three bare early returns. It returns bool now, logs the open
error with the filename, and the shutdown purge is CONDITIONAL on it: for
a converged model the chart objects are the only copy of its signal
history (nothing redraws them - the renderer runs per training era and a
deployed model has none left), so a chart left littered because the disk
write failed beats a clean chart bought by destroying the history. Either
way the log now says which happened.
Also states the user's rule once, where arrows come back rather than
across InitNeuralNetwork's several exits: no weights loaded for this
config => clear the sidecar and start visually clean. A fresh run must
not inherit calls it never made, and the first save would otherwise adopt
them (the sidecar is rebuilt by scanning the chart).
Compiles 0 errors / 0 warnings, standard and Market. Needs redeploy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:28:34 -04:00
void ShutdownChartCleanup ( void )
2026-07-24 21:56:53 -04:00
{
fix(chart): arrows survived the EA that drew them - persist, then clear
Reported: on deinit the panel and status label go, the signal arrows stay.
Two independent causes, both fixed here.
1. It was partly deliberate. ShutdownChartCleanup carried a second
behaviour selected by a `preserveChartArrows` flag derived from the
deinit reason: on RECOMPILE / PARAMETERS / CHARTCHANGE / TEMPLATE the
arrows were left on the chart on purpose, to avoid a reload flicker.
That branch IS the reported symptom, an operator cannot tell it apart
from a cleanup that failed, and it was outright wrong whenever the
reload changed the config - REASON_PARAMETERS means exactly that, and
the preserved arrows then belonged to a model the chart no longer
runs, with nothing marking them stale. It is gone, along with the flag
and m_purgeChartOnDestruct. One path now: persist, clear, restore on
the next attach.
2. Whatever remains was unfalsifiable. PurgeChart was a single
ObjectsDeleteAll(prefix) whose return value was discarded, with no
caller ever looking at the chart again - so "the arrows are still
there" and "the arrows were never there" produced identical evidence,
which is why the report survived three sessions. It now verifies:
after the bulk delete it walks the OBJ_ARROW-typed list (a handful of
objects, not the whole chart), deletes any surviving WarSig_ by name,
and says so. Costs one typed scan when the bulk delete works, which is
the normal case; names the root cause when it does not.
Every failure mode of SaveChartSignals was also silent - it returned void
and had three bare early returns. It returns bool now, logs the open
error with the filename, and the shutdown purge is CONDITIONAL on it: for
a converged model the chart objects are the only copy of its signal
history (nothing redraws them - the renderer runs per training era and a
deployed model has none left), so a chart left littered because the disk
write failed beats a clean chart bought by destroying the history. Either
way the log now says which happened.
Also states the user's rule once, where arrows come back rather than
across InitNeuralNetwork's several exits: no weights loaded for this
config => clear the sidecar and start visually clean. A fresh run must
not inherit calls it never made, and the first save would otherwise adopt
them (the sidecar is rebuilt by scanning the chart).
Compiles 0 errors / 0 warnings, standard and Market. Needs redeploy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:28:34 -04:00
PersistAndClearChartSignals ( ) ;
2026-07-24 21:56:53 -04:00
}
//--- Full shutdown persistence (weights + arrows), preserved for the panel's manual "save weights"
//--- button (SaveWeightsNow) - does NOT purge the chart. OnDeinit no longer calls this; it runs
//--- ShutdownChartCleanup() then PersistWeightsOnShutdown() so cleanup can't be starved by the save.
bool PersistOnShutdown ( void )
{
bool ok = PersistWeightsOnShutdown ( ) ;
//--- Persist the drawn arrows too so a re-add/recompile restores them without a retrain.
2026-07-24 11:52:19 -04:00
SaveChartSignals ( ) ;
2026-07-14 22:36:27 -04:00
return ok ;
}
//--- explicit manual save, identical persistence to PersistOnShutdown() but user-triggered from the panel
bool SaveWeightsNow ( void ) { return PersistOnShutdown ( ) ; }
//--- reloads this signal's current-config weights file from disk, discarding any unsaved in-memory
//--- training progress since the last successful save
bool LoadWeightsNow ( void )
{
if ( CheckPointer ( Net ) = = POINTER_INVALID )
return false ;
double loadedIndicatorParams [ ] ;
bool netLoaded = Net . Load ( m_activeFileName + " .nnw " , dError , dUndefine , dForecast , dtStudied , m_activeFileCommon , m_eraCount , m_trainingComplete , loadedIndicatorParams ) ;
if ( ! netLoaded )
{
Print ( ID + " : ERROR - failed to load weights from " + m_activeFileName + " .nnw, error " + IntegerToString ( GetLastError ( ) ) ) ;
return false ;
}
2026-07-27 11:28:23 -04:00
m_modelLoadedFromDisk = true ;
2026-07-29 12:00:40 -04:00
//--- the file may carry a superseded architecture - correct it before anything reads the net
EnforceTopologyContract ( ) ;
2026-07-23 19:36:34 -04:00
//--- restore the calibration state that pairs with these weights (priors + confidence scale) so a
//--- manual reload keeps live decisions calibrated exactly as the saved model was - see LoadModelStats().
LoadModelStats ( m_activeFileName , m_activeFileCommon ) ;
2026-07-14 22:36:27 -04:00
if ( ArraySize ( loadedIndicatorParams ) = = AD_TUNE_PARAM_COUNT )
{
2026-08-13 10:23:11 -04:00
//--- same no-change guard as the resume path - see AdoptIndicatorParams
2026-07-14 22:36:27 -04:00
if ( m_indicatorsPtr ! = NULL )
2026-08-13 10:23:11 -04:00
AdoptIndicatorParams ( loadedIndicatorParams , m_indicatorsPtr ) ;
else
m_indicatorTuner . Unflatten ( loadedIndicatorParams ) ;
2026-07-14 22:36:27 -04:00
}
2026-08-22 00:24:45 -04:00
//--- Discard resumable state: the load just swapped dtStudied/m_eraCount/weights out from under
//--- whatever era a chunked Train() was mid-way through.
2026-07-14 22:36:27 -04:00
m_trainRunActive = false ;
m_eraResumePending = false ;
m_haveOosCheckpoint = false ;
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
m_checkpointEra = -1 ; // the joint-checkpoint era stamp goes with the snapshot it describes
2026-07-14 22:36:27 -04:00
m_oosWindow . Clear ( ) ;
m_tuneTrialIndex = -1 ;
RefreshLatestSignal ( ) ;
Print ( ID + " : weights reloaded from disk (era " + IntegerToString ( m_eraCount ) + " , trainingComplete= " + ( string ) m_trainingComplete + " ) " ) ;
return true ;
}
//--- deletes this signal's current-config saved files only (weights, topology config, in-progress
2026-08-22 00:24:45 -04:00
//--- Deletes this config's saved files only and rebuilds a fresh untrained topology, so training
//--- restarts from era 0. m_fileName embeds symbol+period+id+outputs+algo, so no other model's
//--- files are reachable from here.
2026-07-14 22:36:27 -04:00
bool ResetWeights ( void )
{
bool stopped = m_trainingStopRequested ;
m_trainingStopRequested = true ; // hold off any in-flight Train() scheduling while we reset
2026-08-22 00:24:45 -04:00
//--- Whichever file this run trains against (see InitNeuralNetwork): the shared production
//--- weights, or the tester cache during a backtest - so a panel reset mid-backtest cannot wipe
//--- the live model.
2026-07-14 22:36:27 -04:00
int flags = m_activeFileCommon ? FILE_COMMON : 0 ;
string nnw = m_activeFileName + " .nnw " ;
string cfg = m_activeFileName + " .cfg " ;
string ckpt = m_activeFileName + " _ckpt.tmp " ;
2026-08-22 00:24:45 -04:00
//--- Sidecars pair with the weights being erased: .stats carries the calibration, _shadow.nnw the
//--- deployed EMA, and _shadowclone.tmp is the clone staging file. Leave any behind and a fresh
//--- retrain inherits the OLD model's calibration or blends into a stale shadow.
2026-07-24 11:52:19 -04:00
string stats = m_activeFileName + " .stats " ;
string shadow = m_activeFileName + " _shadow.nnw " ;
fix(ai): stop the shutdown save from resurrecting reset weights; size HYBRID's LSTM to its real fan-in
ResetWeights already deletes the whole model set - .nnw, .cfg, _ckpt.tmp,
.stats, _shadow.nnw - and clears both the .arrows sidecar and the drawn
chart objects. What undid it was PersistWeightsOnShutdown: detaching the EA
after a reset but before an era completed re-created a .nnw from the
freshly-built, never-run net, so the next attach loaded an era-0 stub
instead of starting clean. For LSTM/HYBRID that stub is worse than nothing -
a layer that has never run a forward pass has m_iInputs<=0, so Save omits
every LSTM buffer (see 413ff7e). Skip the save when no era completed and no
model was loaded; that is exactly the post-reset and first-attach state.
Also sweep _shadowclone.tmp, which the reset did not cover.
Separately, ComputeLstmHiddenSize budgeted every topology against the
flattened input (historyBars x neuronsCount). True for LSTM, wrong for
HYBRID, where AddConvStage runs first and the LSTM is fed the conv feature
map - historyBars x convFilterCount, 160 rather than 420 at H1 defaults.
The quadratic is dominated by the inputs term, so overstating the fan-in
2.6x cost a full ladder step (16 units where the budget affords 32). New
virtual HasConvBeforeLstm() feeds LstmFanIn(), so composition decides this
rather than an AIType check. desc.window is advisory only - CNet never
passes it to the layer - but is now truthful for the same reason.
Derived values stay out of the weights-filename fingerprint and are adopted
from the .cfg, so existing models keep their saved width; only fresh ones
pick up the corrected budget.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 13:06:09 -04:00
string shadowClone = m_activeFileName + " _shadowclone.tmp " ;
2026-08-22 00:24:45 -04:00
//--- SAY WHAT HAPPENED TO EVERY FILE. This once printed only on a delete FAILURE, so a four-member
//--- reset was 24 silent operations and "it only wiped the first model" could not be settled from
//--- a log. "absent" on a member that should have had a .nnw is a different fault from "deleted".
fix(reset): say what the reset actually did, per member and per file
The user reports "Delete & Reset Weights only wipes the first NN". I could not
find a code path that skips ensemble members, and I am not going to assert one:
the handler loops g_aiSignals[0..g_aiSignalCount), all four topologies register
unconditionally in OnInit, and SetIdentity gives each its own State\<id>\ folder
so the six deleted paths are genuinely distinct per member. What IS true is that
the whole success path was SILENT - six FileDelete calls per member printing only
on failure, and one chart-wide Alert - so a four-member reset and a one-member
reset produce byte-identical output. The symptom could be neither confirmed nor
refuted from a log. That is the defect I can fix today.
- COMPILED <timestamp> (__DATETIME__) beside the build tag. The hand-edited tag
had sat at scan-nofwd-v5 across a week of commits, so it could not answer the
question it exists for. The compile stamp cannot be forgotten. Tag bumped to
reset-census-v6.
- RegistryLine() (public): ID, active file path, common/local, era, deployed vs
training, ensemble index. The reset handler prints a numbered census of the
whole registry BEFORE the confirm dialog. If that says 1 on an AI_HYBRID chart
the fault is registration, not the reset - and RegisterAISignal already has a
loud MAX_AI_SIGNALS message for exactly that.
- The confirmation dialog now names the count, so a wrong registry is visible
before anything is deleted rather than after.
- ResetWeights prints one line per member: N deleted / N already absent / N
FAILED, plus a per-suffix breakdown. "absent" on a member that should have had
a .nnw is a completely different fault from "deleted"; they were identical.
- ResetWeights' return value was discarded. A member whose BuildFreshTopology
fails has had its files deleted and has no network - and the Alert still said
"weights reset". Counted now, with an INCOMPLETE alert when they disagree.
- Same for dbm.ResetDatabase(), whose bool was also dropped. The DB is one shared
file for every signal on the chart, so there is nothing per-member to loop -
the log now says that explicitly, since it is the question being asked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 17:57:26 -04:00
int filesDeleted = 0 , filesAbsent = 0 , filesFailed = 0 ;
string wipeReport = " " ;
string targets [ 6 ] ;
targets [ 0 ] = nnw ;
targets [ 1 ] = cfg ;
targets [ 2 ] = ckpt ;
targets [ 3 ] = stats ;
targets [ 4 ] = shadow ;
targets [ 5 ] = shadowClone ;
for ( int fi = 0 ; fi < 6 ; fi + + )
{
ResetLastError ( ) ;
string leaf = targets [ fi ] ;
2026-08-22 00:24:45 -04:00
string shortName = StringSubstr ( targets [ fi ] , StringLen ( m_activeFileName ) ) ; // suffix only; full path prints below
fix(reset): say what the reset actually did, per member and per file
The user reports "Delete & Reset Weights only wipes the first NN". I could not
find a code path that skips ensemble members, and I am not going to assert one:
the handler loops g_aiSignals[0..g_aiSignalCount), all four topologies register
unconditionally in OnInit, and SetIdentity gives each its own State\<id>\ folder
so the six deleted paths are genuinely distinct per member. What IS true is that
the whole success path was SILENT - six FileDelete calls per member printing only
on failure, and one chart-wide Alert - so a four-member reset and a one-member
reset produce byte-identical output. The symptom could be neither confirmed nor
refuted from a log. That is the defect I can fix today.
- COMPILED <timestamp> (__DATETIME__) beside the build tag. The hand-edited tag
had sat at scan-nofwd-v5 across a week of commits, so it could not answer the
question it exists for. The compile stamp cannot be forgotten. Tag bumped to
reset-census-v6.
- RegistryLine() (public): ID, active file path, common/local, era, deployed vs
training, ensemble index. The reset handler prints a numbered census of the
whole registry BEFORE the confirm dialog. If that says 1 on an AI_HYBRID chart
the fault is registration, not the reset - and RegisterAISignal already has a
loud MAX_AI_SIGNALS message for exactly that.
- The confirmation dialog now names the count, so a wrong registry is visible
before anything is deleted rather than after.
- ResetWeights prints one line per member: N deleted / N already absent / N
FAILED, plus a per-suffix breakdown. "absent" on a member that should have had
a .nnw is a completely different fault from "deleted"; they were identical.
- ResetWeights' return value was discarded. A member whose BuildFreshTopology
fails has had its files deleted and has no network - and the Alert still said
"weights reset". Counted now, with an INCOMPLETE alert when they disagree.
- Same for dbm.ResetDatabase(), whose bool was also dropped. The DB is one shared
file for every signal on the chart, so there is nothing per-member to loop -
the log now says that explicitly, since it is the question being asked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 17:57:26 -04:00
if ( ! FileIsExist ( leaf , flags ) )
{
filesAbsent + + ;
wipeReport + = StringFormat ( " %s%s=absent " , ( wipeReport = = " " ? " " : " " ) , shortName ) ;
continue ;
}
if ( FileDelete ( leaf , flags ) )
{
filesDeleted + + ;
wipeReport + = StringFormat ( " %s%s=deleted " , ( wipeReport = = " " ? " " : " " ) , shortName ) ;
}
else
{
filesFailed + + ;
wipeReport + = StringFormat ( " %s%s=FAILED(%d) " , ( wipeReport = = " " ? " " : " " ) ,
shortName , GetLastError ( ) ) ;
Print ( ID + " : ERROR - failed to delete " + leaf + " , error " + IntegerToString ( GetLastError ( ) ) ) ;
}
}
PrintFormat ( " %s: RESET WIPE of %s - %d deleted, %d already absent, %d FAILED | %s " ,
ID , m_activeFileName , filesDeleted , filesAbsent , filesFailed , wipeReport ) ;
2026-08-22 00:24:45 -04:00
//--- The arrows and their .arrows sidecar belong to the model being erased, like the sidecars above.
fix: clear stale signal arrows when a fresh model starts at era 0
Arrow cleanup existed on two paths - the panel's reset-weights, and the
topology-mismatch discard - but both are gated on there being a saved .nnw to
delete. The third case had no cleanup at all: a fresh topology at era 0 with no
weights behind it, which is what a changed config produces. A new fingerprint
makes a new m_fileName, so the previous model's files are not "discarded", they
are simply not this model's files, and nothing ever cleared the chart.
That is not cosmetic. Arrows outlive the model that drew them twice over:
1. The chart objects live in the CHART, not the sidecar, so they survive a
remove/re-add, a recompile, a restart and a fresh deploy no matter what
happens to any file on disk.
2. SaveChartSignals() rebuilds the sidecar by SCANNING the chart for
SIG_ARROW_PREFIX objects. So the first save of the fresh run adopts the
dead model's calls and writes them out under the NEW model's filename -
laundering them into the new model's history where nothing can separate
them afterwards.
Extracted the duplicated cleanup into ClearPersistedChartSignals(reason) - it
cancels the deferred restore queue, deletes m_fileName + ".arrows", clears the
namespaced chart objects and logs why - and called it from all three paths.
The call sits at the BuildFreshTopology() call site, not inside it: the genetic
tuner rebuilds a throwaway topology per candidate (AutoTune.mqh) and must never
touch the chart. All three sites run after m_fileName has its config fingerprint
appended, so they target the right sidecar.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 10:13:01 -04:00
ClearPersistedChartSignals ( " weights reset from the panel " ) ;
2026-07-14 22:36:27 -04:00
m_eraCount = 0 ;
m_trainingComplete = false ;
2026-07-27 11:28:23 -04:00
m_modelLoadedFromDisk = false ;
perf(deinit): I/O-free chart cleanup, dead-panel purge, skip clean weight saves
The 18:23 terminal close (20260825.log) killed two of six charts inside
OnDeinit: they printed "shutting down" then nothing for 5.9 s until
"Abnormal termination", stranding ~700 objects each - including the one
family no prefix sweep can reach, the control panel (CAppDialog names
its 15 objects <numeric instance id><control>, and a re-attach mints a
new id, so a killed panel is a permanent ghost; XTIUSD carried one
across sessions). The stall sat in the two file writes that preceded
all visible cleanup while the four sibling charts flooded the same
2013-era disk - the ~4x18MB-per-chart shutdown weight saves.
Three changes:
1. OnDeinit touches no file until the chart is clean. CVoteArrowStore
splits Save() into Snapshot() (the chart scan, in memory) and
WriteSnapshot() (the disk half, consuming). New order: status label,
vote-arrow snapshot, prefix sweep, panel destroy - all object ops -
then member sidecars, final sweep, timings, and only then the
visibility file, the vote-arrow write and the weight saves.
2. PurgeOrphanedPanelObjects() at OnInit: deletes numeric-prefix
CAppDialog ghosts by name (6 chrome + 9 buttons), qualifying a
prefix only when >=4 of OUR button names carry it, so a foreign
dialog sharing stock chrome names is never touched.
3. m_netDirty: set by every net mutation (both backProp sites, both
RestoreWeights sites, online learning conservatively, panel reset),
cleared only on a successful Net.Save. Shutdown AND the per-bar
autosave now skip the ~18MB write when the net is provably unchanged
- for converged ensembles that is every save - which removes the
very flood that starved the sibling charts. .stats still writes
every time (small; carries the vote record and calibration). A
skipped save leaves the .nnw header dtStudied stale, which is the
already-handled attach-after-offline-gap case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 18:46:51 -04:00
m_netDirty = true ; // the on-disk copy is gone; whatever the net holds next must be written
2026-07-14 22:36:27 -04:00
dtStudied = 0 ;
dError = -1 ;
dUndefine = 0 ;
dForecast = 0 ;
dPrevSignal = 0 ;
2026-07-21 00:03:45 -04:00
m_nmsLiveBuyTime = 0 ;
m_nmsLiveSellTime = 0 ;
2026-07-21 12:30:29 -04:00
m_nmsLiveBuyAccept = false ;
m_nmsLiveSellAccept = false ;
m_nmsLiveKeptTime = 0 ;
m_nmsLiveKeptDir = Neutral ;
m_nmsLiveKeptConf = 0 ;
2026-07-14 22:36:27 -04:00
dOosError = -1 ;
dOosForecast = 0 ;
m_oosSamples = 0 ;
2026-08-22 00:24:45 -04:00
//--- Lifetime counters: reset ONLY here. A normal restart restores them from .stats.
2026-07-25 00:02:34 -04:00
m_cumIsCorrect = 0 ;
m_cumIsTotal = 0 ;
m_cumOosCorrect = 0 ;
m_cumOosTotal = 0 ;
2026-08-16 21:08:41 -04:00
if ( m_ensembleMember )
{
g_ensCumOosCorrect = 0 ;
g_ensCumOosTotal = 0 ;
}
2026-08-22 00:24:45 -04:00
//--- In-progress chunked run/tuning state references buffers and checkpoints from before the reset.
2026-07-14 22:36:27 -04:00
m_trainRunActive = false ;
m_eraResumePending = false ;
m_haveOosCheckpoint = false ;
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
m_checkpointEra = -1 ; // the joint-checkpoint era stamp goes with the snapshot it describes
2026-07-14 22:36:27 -04:00
m_oosWindow . Clear ( ) ;
m_syncWaitStartTick = 0 ;
m_tuneTrialIndex = -1 ;
2026-08-22 00:24:45 -04:00
//--- Re-verify history sync and rebuild the label cache: both reference bars from before the reset.
2026-07-14 22:36:27 -04:00
m_warmupPassesRemaining = 3 ;
m_labelCacheBars = 0 ;
m_labelCacheAnchorTime = 0 ;
m_labelCachePrebuilt = false ;
m_labelPrebuildActive = false ;
m_prebuildSeedPending = false ;
fix(geometry): a weights reset could never change the barrier - the pair laundered itself through the wipe
Reported as "it still seems leaned towards 2x atr" after a full Delete && Reset
Weights on SP500 H4. It was not the .cfg pin, and it was not the new ratio floor
failing to take: the geometry is held in members ResetWeights never cleared, so
a reset wiped the .cfg, built a fresh net, restarted at era 0 - and then relabelled
under the PREVIOUS model's pair, before SaveTopologyConfiguration wrote that stale
pair back into the brand new .cfg. Reset, re-derive, re-pin, with the middle step
missing. No number of resets could ever have moved it.
Evidence in the 2026-08-19 journal: RESET WIPE at 16:23:38.699 (.cfg=deleted),
"rebuilt a fresh topology" at .790, and a label cache at .890 with a distribution
byte-identical to the pre-reset 2.00/2.00 one (Buy 3221 / Sell 3303 / Neutral 4837,
mean lifespan 6.4 bars) - 100 ms later, with no DeriveBarrierGeometry between them.
The member that actually blocked it is m_geometryDerivePasses.
LoadAndCompareTopologyConfiguration pins it to BARRIER_DERIVE_MAX_PASSES to block
the fixed-point iteration, which is correct for a LOAD - an existing model must
never re-derive or the target moves under fitted weights - and exactly wrong for a
RESET, which is the act of declaring there are no fitted weights left to protect.
State that is correctly sticky for one lifecycle event, silently inherited by
another; the same shape as the .cfg pin sitting beside it.
ResetWeights now returns the whole derivation to its constructor state: the three
latches, the pair, the horizon and its flags, the swing/lifespan measurements, the
scan's own outputs, and m_spreadAtr - that last one matters because the cost filter
is deliberately inert on pass 1 (m_spreadAtr <= 0.0) and an inherited spread makes
a reset model walk a different ladder than a genuinely new one. m_sl_mode/m_tp_mode
go back to the SL_Mode/TP_Mode inputs, since the adopt path overwrites them in
place with no copy of what was configured.
NOT COMPILED - user compiles in MetaEditor.
2026-08-19 16:31:18 -04:00
m_sl_mode = SL_Mode ;
m_tp_mode = TP_Mode ;
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
m_onlineLearning . AbortSimIfActive ( ) ;
2026-08-22 00:24:45 -04:00
//--- Salted with this model's id so two members reset in the same millisecond cannot collide,
//--- and re-seeded so weight init is not dominated by the tuner's last candidate evaluation.
feat(rng): ALGLIB's L'Ecuyer generator replaces MathRand, and a seed collision goes with it
MQL5's MathRand() is the 15-bit MSVC LCG - 32768 distinct values and
the lattice structure that shape of generator has. Two places here
actually lean on randomness and both were hurt by it:
WEIGHT INIT. Six He/LeCun-uniform sites drew
((MathRand()+1)/32768.0 - 0.5) * 2 * scale, so a first dense layer of
~250k weights had only 32768 possible values and thousands of
connections started byte-identical. Breaking that symmetry is the whole
job of random init.
SHUFFLING. ShuffleRandomIndex() already had to splice TWO MathRand()
draws to reach 30 bits, and its own comment documented the residual
modulo bias it still carried. HQRndUniformI() is rejection-sampled and
exactly uniform, so the splice and the bias note both go.
CHighQualityRand is L'Ecuyer's combined multiplicative congruential
generator - two differenced streams, 31-bit output, period ~2.3e18 -
and it ships with the terminal.
AND A BUG THE MIGRATION EXPOSED. The three MathSrand(GetTickCount())
calls sit immediately before "build a fresh topology", once per model.
GetTickCount() steps in ~15.6 ms on Windows and an ensemble builds every
member inside one OnInit, so members could be handed the SAME seed and
draw the SAME weights wherever their shapes coincide - and members that
start identical are not an ensemble. WarriorRandSeed() takes a salt (the
model id) plus a never-reset call counter, so a collision is impossible
rather than merely unlikely, while the tick keeps the run itself
genuinely unrepeatable the way those call sites asked for.
Seeds are masked positive rather than trusted: HQRndSeed computes
s % (M-1) + 1 and MQL5's % keeps the sign, so a negative seed leaves the
generator in a state its own assertions reject. GetTickCount() is a uint
and goes negative as an int after ~24 days of uptime - a fault that
would surface as "training is broken" on a long-running terminal and
nowhere else.
The indicator tuner's 52 draws move across too: its random search is
where sample quality earns its keep.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 00:29:12 -04:00
WarriorRandSeed ( ID ) ;
fix(topology): size the network against observations, not bars
The capacity budget is stated in weights per INDEPENDENT observation
and divides by the mean label lifespan to get there. It never once
did: EstimatedInSampleBars() deflates via m_labelOverlap, but it is
only ever called from InitNeuralNetwork, where the label cache does
not exist yet (that same function sets m_labelCachePrebuilt = false
a few lines below), so MeanLifespan() returned its "nothing measured"
default of 1.0 at every call. Every fresh model was sized as though
its labels did not overlap - over-budgeting the first dense layer by
a factor of L, which is several rungs of a power-of-two ladder. The
"expect overfitting, reduce the feature set or pool instruments"
warning is the branch that should fire on H1 and structurally could
not.
Fixed at the source rather than by reordering the boot sequence (the
prebuild is chunked across Train() calls and cannot complete inside
init): MeasureSwingGeometry() walks the ZigZag ONCE at init and
answers both questions from it - the median leg gives the window,
and the leg series gives the mean label lifespan analytically.
SwingPivotDirectionLabel resolves bar i when the SECOND pivot after
it commits, so a bar d bars before pivot P waits d + (the leg
leaving P); summed over every bar of every leg that is exactly the
mean the label walk accumulates.
That also closes the coherence gap the swing target opened: the
window was measured with a private +/-12-bar fractal while the label
aimed at ZigZag(12,5,3) pivots, so it was sized against a leg
distribution the label never used. One pivot source now, the
label's.
Also:
- ResetWeights() re-derives the shape. It rebuilt from the members a
history-starved init had pinned and re-saved them - so the "let
history download, then reset from the panel" advice in both
fallback warnings did nothing at all.
- The CAPACITY line prints the measured lifespan beside the one the
topology was sized for, and warns when they differ by more than a
ladder rung. That is the check that makes the estimator falsifiable.
- Topology reads the view's symbol, not _Symbol (latent for pooling).
- Unmeasured geometry defaults to HISTORY_BARS_FALLBACK, never 1.0:
under-sizing is recoverable, over-sizing silently is not.
Compile: 0 errors, 0 warnings (stage).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 21:21:05 -04:00
//--- RE-DERIVE THE SHAPE, in InitNeuralNetwork()'s order (window first - the other four read
//--- what it measures). Until 2026-08-24 this rebuilt from whatever the members already held,
//--- so a model whose window and capacity had been sized from a half-synced history stayed that
//--- shape and was re-pinned to disk by the very action both fallback warnings tell the operator
//--- to take. The .cfg was deleted above, so nothing on disk is being contradicted.
m_topology . RemeasureSwingGeometry ( ) ;
m_historyBars = DeriveHistoryBars ( ) ;
m_convFilterCount = ComputeConvFilterCount ( ) ;
m_lstmHiddenSize = ComputeLstmHiddenSize ( ) ;
m_initialNeuronsCount = ComputeFirstLayerWidth ( ) ;
m_hiddenLayersCount = ComputeHiddenLayerCount ( ) ;
2026-07-14 22:36:27 -04:00
bool rebuilt = BuildFreshTopology ( ) ;
if ( ! rebuilt )
Print ( ID + " : ERROR - failed to rebuild fresh topology after weights reset " ) ;
else
{
2026-08-22 00:24:45 -04:00
//--- isInitialized=FALSE, never m_isInitialized: every other writer runs before init sets it
//--- true, so passing true here makes this the only .cfg on disk that fails its own compare
//--- on the next attach. Runtime lifecycle state must never gate reuse.
feat(nn): derive dense depth, train on all history, pin the shape in .cfg
Completes the derived-topology work. Three inputs removed.
AIType loses its depth suffix - AI_MLP/AI_CONV/AI_LSTM/AI_HYBRID, five
entries instead of eight. Depth is now derived from the two endpoints
the taper already has to connect (derived first-layer width, output-tied
final width) at a 2x per-layer compression target, clamped [2..5].
Asking a user to pick a layer count while the code derives the widths
those layers taper between was asking for half a decision: at 64 units
tapering to 12, four layers compress by 1.4x per step and five by 1.3x,
so the extra depth bought no abstraction. On the shipping H1/10y default
the derivation lands on 3 layers - the depth that actually won Run 2.
StudyPeriods removed. There is no case for training on less data than
the broker provides at a ~6% directional base rate; the honest
generalization read comes from the OOS holdout, not from withholding
history. Training now starts at the earliest available bar, floored by
MinTrainYear, which answers a different question (excluding dubious
pre-history) and stays.
That required closing the hazard the old code documented: the capacity
budget now MEASURES the symbol's real bar count, and a topology derived
from a measurement would widen as history downloads. Both ends are now
pinned. Every derived value left the weights-filename fingerprint -
keying a filename on a measured quantity means the EA looks for a file
that does not exist, starts from era 0 and orphans a trained model,
silently, because a missing cache is the normal first-run state. The
shape lives in the .cfg instead, where LoadAndCompare now ADOPTS the
four derived fields rather than diffing them; a mismatch there would
discard a fully-trained model over nothing the user did. Two fields
appended to the .cfg for the conv/LSTM stages, length-guarded on read
because FileReadInteger past EOF returns 0 with no error.
ForceHiddenLayers, a compile-time constant like DebuggingMode, pins
depth for diagnostic comparisons. It joins the fingerprint only when
non-zero, so forced depths get their own files - sequential comparisons
only, not simultaneous from one .ex5.
Derived shape, H1/10y defaults (21 features x 20 bars): first layer 64,
3 dense, 8 conv filters, 16 LSTM units. The LSTM block halves from
~58k to ~28k weights.
Both builds compile 0 errors, 0 warnings. Re-keys existing models.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 10:05:40 -04:00
SaveTopologyConfiguration ( m_activeFileName , m_initialNeuronsCount , m_hiddenLayersCount , m_neuronsReduction , m_minNeuronsCount , m_optimizationAlgo , m_historyBars , m_outputNeuronsCount , m_neuronsCount , LEGACY_STUDY_PERIOD_SLOT , m_minTrainYear , false , LEGACY_CONVERGE_WR_SLOT , m_fractalPeriods , m_convFilterCount , m_lstmHiddenSize , m_activeFileCommon ) ;
2026-07-14 22:36:27 -04:00
Print ( ID + " : weights reset - training will restart from era 0 (current config only: " + m_activeFileName + " ) " ) ;
}
m_trainingStopRequested = stopped ;
if ( ! stopped & & ! bEventStudy )
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
ArmStudyEvent ( 0 , " Reset " ) ;
2026-07-14 22:36:27 -04:00
return rebuilt ;
}
} ;
//+------------------------------------------------------------------+
2026-08-22 00:24:45 -04:00
//| IMPLEMENTATION. Method bodies, included after the declaration |
//| above and nowhere else. Order between them is irrelevant. |
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
//+------------------------------------------------------------------+
# include "AIBase\Training.mqh"
2026-08-01 11:27:28 -04:00
# include "AIBase\Lifecycle.mqh"
# include "AIBase\Topology.mqh"
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
# include "AIBase\Labels.mqh"
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
//--- Body in Expert\OnlineLearning\OnlineLearning.mqh.
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
# include "AIBase\AutoTune.mqh"
refactor(mi): split AutoTune.mqh - SEARCH vs MEASUREMENT
Session C of the feature-selection/labeling refactor track. AutoTune.mqh was
two responsibilities in one 1,584-line file: SEARCH (TuneIndicatorsByFilter,
coordinate-descent over indicator settings) and MEASUREMENT (the "does this
feature vector predict this label at all" evidence screen and its three
sub-reports). Split along that seam into a new Expert/AIBase/FeatureScreen.mqh.
Moved, verbatim (diffed byte-for-byte against the pre-split content - zero
lines differ beyond the file-boundary comment headers): ReportFeatureLabel-
Information, ReportExcursionInformation, ReportFeatureLagProfile, Report-
BarrierGeometryScan, ApplyAdoptedGeometry.
Stayed in AutoTune.mqh: the MI engine (FeatureColumnMI/BuildMiSample/
ScoreMiSample) both files call - a shared dependency used by two consumers is
not itself a reason to split further; TuneIndicatorsByFilter; the export
utilities; and TuneIndicatorsAndTrain, the entry point that decides which of
the two branches a given model runs - it is the coordinator, not a member of
either side.
Still body-only method definitions of CExpertSignalAIBase, same as every
other Expert\AIBase\*.mqh file - MQL5 has no partial classes, so this is a
file-organisation move (legibility, SRP-per-file), not a coupling reduction.
The include site says order between AIBase\*.mqh files is irrelevant, so the
new include was added next to AutoTune.mqh's; the file-scope g_ensembleChart*
globals both files reference stay declared in AutoTune.mqh's header, ahead of
the new include either way.
Verified: brace counts split exactly 105 -> 57+48; every one of the 14
function definitions HEAD had in AutoTune.mqh accounted for in exactly one of
the two files, no duplicates; the WARRIOR_EXPORT_FEATURES ifdef/endif pair
(unrelated, lines 35/147) untouched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 19:08:36 -04:00
# include "AIBase\FeatureScreen.mqh"
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
# include "AIBase\Inference.mqh"
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
//--- Body in Expert\Persistence\ModelPersistence.mqh.
2026-08-24 18:26:25 -04:00
//--- Now holds only InitOpen/InitClose/InitHigh/InitLow/InitTime/InitZigZag/ResizeBuffers/
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
//--- RefreshData - the rest of the old Features.mqh is in Expert\Features\FeatureBuilder.mqh.
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
# include "AIBase\Features.mqh"
refactor(arch): a read-only training-data view, so modules stop being #included code
The AIBase\*.mqh files are not modules. They are method bodies of one
3,400-line class, textually #included after its declaration. Every one
of them can touch every member of every other, which is why "move this
out" has so far meant "move the whole class".
Introduce the seam that ends that:
CTrainingDataView abstract - the ONLY thing a training-side
collaborator may see: a feature row, a label, an
outcome, an excursion, the shape they share, and
the identity to log under.
CAIBaseTrainingData the adapter. MQL5 gives a class exactly one base
and CExpertSignalAIBase is already a
CExpertSignalCustom, so it cannot implement the
view itself. It owns one of these instead.
Data*() on the the published read API the adapter forwards to.
signal MQL5 has no `friend`, so reaching in from outside
was never an option - and making it explicit is
the point rather than a workaround.
Every row accessor OWNS ITS BOUNDS TEST and answers false for a bar it
has nothing for. Thirty-odd call sites currently carry their own
ArraySize() guard; one that forgets reads past a cache that is shorter
than the bar count for the whole warm-up. The -2.0 "never scored"
sentinel on the arrow cache is folded in the same way, so it can no
longer be mistaken for a small confidence.
Nothing uses it yet - this is the seam only, kept as its own commit so
the pattern compiles before 951 lines of Baselines move onto it. The
pattern is the stdlib's own: abstract base with =0 (Canvas\DX\DXObject),
concrete override, forward-declared owner pointer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:53:00 -04:00
# include "Training\AIBaseTrainingDataImpl.mqh"
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
# include "Chart\AIBaseChartViewImpl.mqh"
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
# include "Persistence\AIBasePersistenceViewImpl.mqh"
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
# include "OnlineLearning\AIBaseOnlineLearningViewImpl.mqh"
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
# include "Topology\AIBaseTopologyViewImpl.mqh"
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
# include "Features\AIBaseFeaturesViewImpl.mqh"
2026-08-24 04:39:17 -04:00
# include "ConfigLock\AIBaseConfigLockViewImpl.mqh"
2026-07-16 00:56:33 -04:00
//+------------------------------------------------------------------+