forked from mnbvc188199/Warrior_EA
c393497fd6
Full-pipeline analysis after "threshold 30, attained often, nothing drawn,
still glued to buy". The log falsified the premise before any code did:
21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0%
21:42:07 swept 4999, 0 voters, drew 0
21:51:30 swept 4999, 0 voters, drew 0
21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0%
The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root
cause, three symptoms: every display path read m_arrowSignalCache, which is
wiped to sentinel at each era start and only complete again when pass 3
finishes. With eras at ~30s and a sweep at ~17s:
* ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its
else-branch deleted the arrow on every voteless bar - erasing the previous
sweep's entire output. The chart cycled populated -> blank -> populated;
the user kept catching the blank phase.
* READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every
era and fell through to dPrevSignal - the frozen purge-band edge bar that
reads Buy. 659638e fixed which bar was frozen, not the freezing.
* VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a
different fraction of half-rebuilt caches.
THE FIX, structural rather than another patch:
1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one
moment the cache is complete - and now copies it (raw signals, newest
LOOKBACK+16 bars) into member-owned snapshot state, unconditionally,
BEFORE its early return: an all-Neutral era is a snapshot worth showing,
not an absence of one. Raw signals rather than votes, so a tier re-rank
between eras reprices them at read time via LiveVoteContribution for free.
2. The sweep (SnapshotVoteAt) and the prospective readout both read
snapshots; the readout's fallback chain is live-cache -> snapshot ->
dPrevSignal, and the snapshot leg is the one that fires most of the time.
3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual
sub-threshold vote takes an arrow down. This alone ends the wipe half of
the flicker even where snapshots are missing (before the first era).
4. Arming moved from an era-counter diff (which fires at era BOUNDARIES,
i.e. precisely when caches are about to be wiped) to
g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's
snapshot just got fresher", the only event a redraw can act on. 60s rate
limit collapses the four members' burst into one sweep. Classic-only
charts arm once at start.
5. Census now reports the direction split - "922 had a voter (610 buy / 312
sell)" - so "the vote leans buy" is checkable from the log instead of
inferred from arrow colours.
Also visible in the log and worth knowing: the threshold flip-flopped
30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51
ran at 40), so part of the observed blankness was configuration, not code.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1127 lines
57 KiB
MQL5
1127 lines
57 KiB
MQL5
//+------------------------------------------------------------------+
|
|
//| Lifecycle.mqh |
|
|
//| |
|
|
//| Construction/destruction, the CExpertSignal vote API |
|
|
//| (LongCondition/ShortCondition/ConfidenceTier/pattern weights), |
|
|
//| tick + chart-event dispatch, and the per-config chart lock. |
|
|
//| |
|
|
//| PARTIAL IMPLEMENTATION FILE - not standalone. |
|
|
//| CExpertSignalAIBase method BODIES only. The class declaration |
|
|
//| lives in Expert\ExpertSignalAIBase.mqh, which includes this file |
|
|
//| at the bottom, after the declaration. Do not include it |
|
|
//| anywhere else and do not compile it on its own. |
|
|
//+------------------------------------------------------------------+
|
|
#ifndef WARRIOR_AIBASE_LIFECYCLE_MQH
|
|
#define WARRIOR_AIBASE_LIFECYCLE_MQH
|
|
//+------------------------------------------------------------------+
|
|
//| Constructor |
|
|
//+------------------------------------------------------------------+
|
|
//--- These are only fallback defaults for a fresh object before the EA's OnInit() applies the
|
|
//--- active input values via the public setters in Warrior_EA.mq5. The input-driven values are the
|
|
//--- source of truth for the actual run configuration.
|
|
CExpertSignalAIBase::CExpertSignalAIBase(void) :
|
|
ID("NULL"),
|
|
m_neuronsCount(0),
|
|
m_minTrainYear(1970),
|
|
m_optimizationAlgo(TrainingOptimizer), // see the member declaration comment
|
|
//--- Placeholder only; InitNeuralNetwork() replaces it with ComputeFirstLayerWidth() before anything
|
|
//--- reads it. Deliberately the floor rather than 0, so a hypothetical path that built a topology
|
|
//--- without going through init would produce a small usable net instead of a zero-width layer.
|
|
m_initialNeuronsCount(FIRST_LAYER_MIN_WIDTH),
|
|
m_outputNeuronsCount(OUTPUT_CLASSIFICATION),
|
|
//--- Frozen. Nothing reads these to build a topology any more - the taper derives its own endpoints
|
|
//--- (BuildFreshTopology) - but they still occupy positional slots in the .cfg sidecar and the weights
|
|
//--- fingerprint. Held at their historical defaults so both stay byte-stable; changing either value
|
|
//--- would re-key every model on disk for no behavioural reason whatsoever.
|
|
m_minNeuronsCount(MIN_NEURONS_20),
|
|
m_neuronsReduction(RF_70),
|
|
m_hiddenLayersCount(3),
|
|
m_lstmHiddenSize(32),
|
|
m_convFilterCount(16),
|
|
m_historyBars(14),
|
|
m_fractalPeriods(5),
|
|
m_pattern_0(25),
|
|
m_pattern_1(50),
|
|
m_pattern_2(75),
|
|
m_pattern_3(100),
|
|
m_trainTarget(0),
|
|
m_ensembleMember(false),
|
|
m_ensemblePanelSlot(-1),
|
|
m_fracLegCount(0),
|
|
m_metaCandCount(0),
|
|
m_useVolumes(true),
|
|
m_useTime(true),
|
|
m_useATR(true),
|
|
m_useMA(false),
|
|
m_useRSI(false),
|
|
m_useMACD(false),
|
|
m_useIchimoku(false),
|
|
m_useSwingContext(false),
|
|
m_useNews(false),
|
|
m_useCrossAsset(false),
|
|
m_useSpreadFeature(false),
|
|
m_spreadSeriesBars(0),
|
|
m_spreadSeriesAnchor(0),
|
|
m_crossAssetAnchor(0),
|
|
m_crossAssetPairsPinned(""),
|
|
m_crossAssetCfgSaved(false),
|
|
m_useAltData(false),
|
|
m_altDataEnabled(true),
|
|
m_altDataLateWarned(false),
|
|
m_altDataNamesPinned(""),
|
|
m_newsFeatureWindowMinutes(60),
|
|
m_useADCumulativeDelta(false),
|
|
m_useADShorteningOfThrust(false),
|
|
m_useADWyckoffEventStream(false),
|
|
m_useADWyckoffFailedStructure(false),
|
|
m_useADWyckoffSignificantBarInversion(false),
|
|
m_autoTuneIndicators(false),
|
|
m_indicatorsPtr(NULL),
|
|
Net(NULL),
|
|
m_shadowNet(NULL),
|
|
TempData(NULL),
|
|
dError(-1),
|
|
dUndefine(0),
|
|
dForecast(0),
|
|
dPrevSignal(0),
|
|
m_refreshOk(0),
|
|
m_refreshFailFeatures(0),
|
|
m_refreshFailShort(0),
|
|
m_refreshBuy(0),
|
|
m_refreshSell(0),
|
|
m_refreshNeutral(0),
|
|
m_voteGateBlocked(0),
|
|
m_voteGatePassed(0),
|
|
m_voteGateCompleteAtFirst(-1),
|
|
m_voteGateLoadedAtFirst(-1),
|
|
m_signalClusterWindow(6),
|
|
m_nmsLiveBuyTime(0),
|
|
m_nmsLiveSellTime(0),
|
|
m_nmsLiveBuyAccept(false),
|
|
m_nmsLiveSellAccept(false),
|
|
m_nmsLiveKeptTime(0),
|
|
m_nmsLiveKeptDir(Neutral),
|
|
m_nmsLiveKeptConf(0),
|
|
dtStudied(0),
|
|
m_eraCount(0),
|
|
m_trainingComplete(false),
|
|
m_inferenceOnly(false),
|
|
m_modelLoadedFromDisk(false),
|
|
m_topologySuperseded(false),
|
|
m_mqlInferenceValidated(false),
|
|
m_shadowBootstrapAttempted(false),
|
|
m_enableOnlineLearning(true),
|
|
m_freezePriorCalibration(false),
|
|
m_onlineLearnedUpToTime(0),
|
|
m_onlineRollingAcc(-1.0),
|
|
m_onlineSamples(0),
|
|
m_onlineBarsSincePersist(0),
|
|
m_onlineBlendFrozen(false),
|
|
bEventStudy(false),
|
|
m_oosSplitPct(30),
|
|
dOosError(-1),
|
|
dOosForecast(0),
|
|
m_oosSamples(0),
|
|
m_cumIsCorrect(0),
|
|
m_cumIsTotal(0),
|
|
m_cumOosCorrect(0),
|
|
m_cumOosTotal(0),
|
|
m_oosOutSpreadSum(0),
|
|
m_oosOutCount(0),
|
|
m_spreadAtr(0.0),
|
|
m_oosNeutralStrict(0),
|
|
m_oosNeutralTie(0),
|
|
m_oosTieBuySell(0),
|
|
m_oosRailBars(0),
|
|
m_countBuySignals(0),
|
|
m_countSellSignals(0),
|
|
m_countNeutralSignals(0),
|
|
m_trueBuyCount(0),
|
|
m_trueSellCount(0),
|
|
m_trueNeutralCount(0),
|
|
m_logitAdjustTau(1.0),
|
|
m_logitAdjustLogged(false),
|
|
m_logitAdjustSkipWarned(false),
|
|
m_prevEraTrueBuyCount(0),
|
|
m_prevEraTrueSellCount(0),
|
|
m_prevEraTrueNeutralCount(0),
|
|
m_oosBuyHits(0),
|
|
m_oosBuyTotal(0),
|
|
m_oosSellHits(0),
|
|
m_oosSellTotal(0),
|
|
m_oosNeutralHits(0),
|
|
m_oosNeutralTotal(0),
|
|
m_oosBuyPredicted(0),
|
|
m_oosBuyPredictedHits(0),
|
|
m_oosSellPredicted(0),
|
|
m_oosSellPredictedHits(0),
|
|
m_oosNeutralPredicted(0),
|
|
m_oosNeutralPredictedHits(0),
|
|
m_oosBuyPredictedWins(0),
|
|
m_oosSellPredictedWins(0),
|
|
m_oosWinLongTotal(0),
|
|
m_oosWinShortTotal(0),
|
|
m_oosConfidenceSum(0),
|
|
m_confidenceCalScale(1.0),
|
|
m_minDirectionalRecallPct(40),
|
|
//--- No vote-driven exit until the inputs say otherwise - matches Min_Vote_Close's shipped Disabled.
|
|
m_exitVoteThreshold(0.0),
|
|
m_exitHoldToBarrier(false),
|
|
m_simRSum(0.0),
|
|
m_simRSumSq(0.0),
|
|
m_simTrades(0),
|
|
m_simVoteExits(0),
|
|
m_simBarrierWins(0),
|
|
m_exitReplayReported(false),
|
|
m_lastRecallFloorPct(0.0),
|
|
m_detectabilityReported(false),
|
|
m_priorBuy(0.0),
|
|
m_priorSell(0.0),
|
|
m_priorNeutral(0.0),
|
|
m_oosBuyFired(0),
|
|
m_oosBuyFiredHits(0),
|
|
m_oosSellFired(0),
|
|
m_oosSellFiredHits(0),
|
|
m_oosNmsFired(0),
|
|
m_oosNmsHits(0),
|
|
m_oosNmsLastBuyIdx(-1),
|
|
m_oosNmsLastSellIdx(-1),
|
|
m_oosNmsKeptIdx(-1),
|
|
m_oosNmsKeptConf(0.0),
|
|
m_oosNmsKeptDir(Neutral),
|
|
m_lastBuyFiredPrecPct(-1),
|
|
m_lastSellFiredPrecPct(-1),
|
|
m_lastBuyFired(0),
|
|
m_lastSellFired(0),
|
|
m_maxClassSampleWeight(1.5),
|
|
m_swingConfirmationBars(100),
|
|
m_barrierHorizonBars(BARRIER_HORIZON_FALLBACK),
|
|
m_barrierHorizonResolved(false),
|
|
m_barrierFallbackWarned(false),
|
|
m_lastBarrierTimedOut(false),
|
|
m_barrierHorizonLegStarved(false),
|
|
m_horizonStarvedWarned(false),
|
|
m_geometryCfgSaved(false),
|
|
m_geometryAdopted(false),
|
|
m_dirEvidence(false),
|
|
m_dirEvidenceWhy("not measured yet"),
|
|
m_lastBarrierBothWon(false),
|
|
m_lastBarrierBothWonTied(false),
|
|
m_labelPrebuildTimeoutCount(0),
|
|
m_labelPrebuildBothWonCount(0),
|
|
m_labelPrebuildBothWonTieCount(0),
|
|
m_maxErasPerRun(300),
|
|
m_arrowRestoreIndex(0),
|
|
m_arrowRestorePending(false),
|
|
m_arrowRestoreStartMs(0),
|
|
m_rescanIndex(0),
|
|
m_rescanHi(0),
|
|
m_rescanBarsNow(0),
|
|
m_rescanPending(false),
|
|
m_rescanStartMs(0),
|
|
m_rescanRawBuy(0),
|
|
m_rescanRawSell(0),
|
|
m_rescanRawNeutral(0),
|
|
m_trainRunActive(false),
|
|
m_eraResumePending(false),
|
|
m_resumeBars(0),
|
|
m_resumeTotalIter(0),
|
|
m_resumeOosCutoff(0),
|
|
m_resumeBarIndex(0),
|
|
m_resumeAddLoop(false),
|
|
m_isTrainQueueCount(0),
|
|
m_isTrainCursor(0),
|
|
m_isPass2Active(false),
|
|
m_isPass2Done(false),
|
|
m_isPass3Active(false),
|
|
m_oosScoreIndex(0),
|
|
m_oosScoreStartIndex(0),
|
|
m_isCalibActive(false),
|
|
m_isCalibDone(false),
|
|
m_calibIndex(0),
|
|
m_calibStartIndex(0),
|
|
m_excNet(NULL),
|
|
m_excTgt(NULL),
|
|
m_excOut(NULL),
|
|
m_excHeadFailed(false),
|
|
m_excBaseTotal(0),
|
|
m_excScored(0),
|
|
m_excScoredD(0),
|
|
m_excMonoViol(0),
|
|
m_excTrainTick(0),
|
|
m_excUs(0),
|
|
m_excTrailHead(0),
|
|
m_excTrailCount(0),
|
|
m_excTrailN(0),
|
|
m_excTrailScored(0),
|
|
m_lastStatusLabelUpdateTick(0),
|
|
m_lastBuyRecallPct(-1),
|
|
m_lastSellRecallPct(-1),
|
|
m_lastDisplayNeuron0(0),
|
|
m_lastDisplayNeuron1(0),
|
|
m_lastDisplayNeuron2(0),
|
|
m_lastDisplaySignal(0),
|
|
m_lastBarTime(0),
|
|
m_modelEta(InitialEtaForOptimizer()),
|
|
m_etaCeiling(InitialEtaForOptimizer()),
|
|
m_erasSinceCooldown(0),
|
|
m_bestOosForecast(-1),
|
|
m_bestBalancedOos(-1),
|
|
m_bestPassedRecall(false),
|
|
m_bestBothSidesLive(false),
|
|
m_eraStartTick(0),
|
|
m_passFeatUs(0),
|
|
m_passNetUs(0),
|
|
m_passHeartbeatPrints(0),
|
|
m_passWindowOk(0),
|
|
m_passWindowFail(0),
|
|
m_consecutiveRegressions(0),
|
|
m_featureFailTransient(false),
|
|
m_featureFailBlock(""),
|
|
m_featureFailIdx(-1),
|
|
m_windowFailSlot(-2),
|
|
m_windowFailTotal(0),
|
|
m_featureWidthWarned(false),
|
|
m_featureHealthReported(false),
|
|
m_lastHeartbeatTick(0),
|
|
m_passProgressPct(0),
|
|
m_passLabel("starting"),
|
|
m_lastEraCompleteTick(0),
|
|
m_lastStallReportTick(0),
|
|
m_haveOosCheckpoint(false),
|
|
m_bestDirPrecPct(-1.0),
|
|
m_bestChancePrecPct(-1.0),
|
|
m_bestDirCalls(0),
|
|
m_deployCandidateEras(0),
|
|
m_oosStable(false),
|
|
m_objectiveMet(false),
|
|
m_erasSinceBestBalanced(0),
|
|
m_plateauStage(0),
|
|
m_bestIsError(-1.0),
|
|
m_erasSinceBestIsError(0),
|
|
m_isErrorPlateaued(false),
|
|
m_restartBoostErasLeft(0),
|
|
m_syncWaitStartTick(0),
|
|
m_warmupPassesRemaining(0),
|
|
m_coldSweepTick(0),
|
|
m_indicatorDepthCapBars(0),
|
|
m_indicatorDepthDeadWarned(false),
|
|
m_handleRepairTick(0),
|
|
m_barrierEraSeen(-1),
|
|
m_barrierEraTick(0),
|
|
m_barrierExcluded(false),
|
|
m_barrierHoldReportTick(0),
|
|
m_inferenceDepthRefusalWarned(false),
|
|
m_prebuildBlockWarned(false),
|
|
m_depthSettleStart(0),
|
|
m_depthProbeTick(0),
|
|
m_depthProbeLast(0),
|
|
m_depthProbeStable(0),
|
|
m_labelCacheBars(0),
|
|
m_labelCacheAnchorTime(0),
|
|
m_labelCachePrebuilt(false),
|
|
m_lastExcUp(0.0),
|
|
m_lastExcDown(0.0),
|
|
//--- 0 samples => MeanLabelLifespan() returns 1.0 => EffectiveSampleSize() is the identity, so an
|
|
//--- un-prebuilt model behaves exactly as it did before the overlap correction existed rather than
|
|
//--- shrinking its own samples on a guess. See m_lastLabelLifespan.
|
|
m_lastLabelLifespan(0),
|
|
m_labelLifespanSum(0.0),
|
|
m_labelLifespanCount(0),
|
|
//--- -1 = the deploy gate has not run its arithmetic yet this era; the era line then omits the bar
|
|
//--- rather than printing a stale one from a previous era.
|
|
m_lastEdgeFloorPct(-1.0),
|
|
m_lastPrecSE(-1.0),
|
|
m_lastEffN(-1.0),
|
|
m_lastRungLifespan(0.0),
|
|
m_poolWriteWarned(false),
|
|
m_lastPoolPasses(false),
|
|
m_lastPoolReport(""),
|
|
m_derivedSlMult(0.0),
|
|
m_derivedTpMult(0.0),
|
|
//--- 0.0 = unthresholded until the first pass 2 fits it (see DIR_CONF_THRESHOLD_BINS). Deliberately
|
|
//--- the permissive value: a model that has not measured its own operating point must not silently
|
|
//--- abstain on everything.
|
|
m_dirConfThreshold(0.0),
|
|
m_bestDirConfThreshold(0.0),
|
|
m_dirConfPrimaryBars(0),
|
|
m_dirConfSparseWarned(false),
|
|
m_geometryDerived(false),
|
|
m_geometryDerivePasses(0),
|
|
m_swingMedianBars(0),
|
|
m_labelPrebuildActive(false),
|
|
m_prebuildSeedPending(false),
|
|
m_labelPrebuildBars(0),
|
|
m_labelPrebuildOosCutoff(0),
|
|
m_labelPrebuildIndex(-1),
|
|
m_labelPrebuildBuyCount(0),
|
|
m_labelPrebuildSellCount(0),
|
|
m_labelPrebuildNeutralCount(0),
|
|
m_simOosNet(NULL),
|
|
m_simOosRunActive(false),
|
|
m_simOosCutoff(0),
|
|
m_simOosBarIndex(-1),
|
|
m_simOosForecast(0),
|
|
m_simOosSamples(0),
|
|
m_dbBackfillActive(false),
|
|
m_dbBackfillDone(false),
|
|
m_dbBackfillIndex(0),
|
|
m_dbBackfillStartIndex(0),
|
|
m_dbBackfillStopIndex(2),
|
|
m_dbBackfillBars(0),
|
|
m_dbBackfillFired(0),
|
|
m_dbBackfillEra(-1),
|
|
m_tuneTrialIndex(-1),
|
|
m_tuneBestOosForecast(-1),
|
|
m_tuneLastTrialWasWin(true),
|
|
m_tuneHaveBestCheckpoint(false),
|
|
m_tuneStartTrainBar(0),
|
|
m_tuneFilterDone(false),
|
|
m_trainingPaused(false),
|
|
m_trainingStopRequested(false),
|
|
m_activeFileCommon(true),
|
|
m_configLockName(""),
|
|
m_isInitialized(false),
|
|
m_shutdownInProgress(false),
|
|
m_lastArrowsSaved(0),
|
|
m_purgeMismatchWarned(false),
|
|
m_miBestColumn(0.0),
|
|
m_miLabelEntropy(0.0),
|
|
m_miStrideBars(0),
|
|
m_miNullBlocks(0),
|
|
m_miReportDone(false),
|
|
m_tiersSelfRanked(false),
|
|
m_overlaySnapBars(0),
|
|
m_prospectiveSigSnap(-2.0),
|
|
m_miReportDeferrals(0),
|
|
m_barrierScanSlMult(0.0),
|
|
m_barrierScanTpMult(0.0),
|
|
m_barrierScanLiveLabels(false),
|
|
m_barrierScanTimeouts(0),
|
|
m_barrierHorizonClamped(false)
|
|
{
|
|
//--- Claim this instance's study-event id - see STUDY_EVENT_ID_BASE (ExpertSignalAIBase.mqh) for why
|
|
//--- these are per instance and offset above the Controls library's event codes.
|
|
m_studyEventId = (ushort)(STUDY_EVENT_ID_BASE + g_warriorStudyEventSeq++);
|
|
m_studyArmedTick = 0;
|
|
m_ensembleIndex = -1; // not an ensemble member until EnsembleMember(true) registers one (the flag itself is in the init list)
|
|
//--- per-era stash the ensemble verdict reads (see EnsembleStashEraStats); -1/false = "no era yet"
|
|
m_eraStatPrecPct = -1.0;
|
|
m_eraStatChancePct = -1.0;
|
|
m_eraStatCalls = 0;
|
|
m_eraStatTradeable = false;
|
|
m_eraStatTwoSided = false;
|
|
m_eraStatScore = 0.0;
|
|
m_eraStatBlended = 0.0;
|
|
m_eraStatThreshold = 0.0;
|
|
m_checkpointEra = -1;
|
|
//--- indicator tuning defaults live in CADIndicatorTuner's own constructor (Expert\ADIndicatorTuner.mqh),
|
|
//--- which runs automatically for the m_indicatorTuner member above.
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Destructor |
|
|
//+------------------------------------------------------------------+
|
|
CExpertSignalAIBase::~CExpertSignalAIBase(void)
|
|
{
|
|
//--- deliberately NOT calling PersistOnShutdown() here: OnDeinit() (Warrior_EA.mq5) already calls
|
|
//--- it explicitly for every signal, one call stack frame shallower, BEFORE Expert.Deinit() tears
|
|
//--- these objects down. Doing it again here nested inside that same teardown cascade doubled the
|
|
//--- stack depth of an already-deep recursive save (layers -> neurons -> connections) right at the
|
|
//--- point in MT5's lifecycle (EA recompile while attached) that has the least stack headroom, and
|
|
//--- reliably crashed the terminal with a stack overflow. Keep this destructor cheap.
|
|
if(CheckPointer(Net) != POINTER_INVALID)
|
|
delete Net;
|
|
if(CheckPointer(m_shadowNet) != POINTER_INVALID)
|
|
delete m_shadowNet;
|
|
if(CheckPointer(TempData) != POINTER_INVALID)
|
|
delete TempData;
|
|
if(CheckPointer(m_simOosNet) != POINTER_INVALID)
|
|
delete m_simOosNet;
|
|
//--- Excursion head: never persisted (Stage 1 is a measurement), so teardown is the whole lifecycle.
|
|
if(CheckPointer(m_excNet) != POINTER_INVALID)
|
|
delete m_excNet;
|
|
if(CheckPointer(m_excTgt) != POINTER_INVALID)
|
|
delete m_excTgt;
|
|
if(CheckPointer(m_excOut) != POINTER_INVALID)
|
|
delete m_excOut;
|
|
//--- Unconditional now that the warm-reload "leave the arrows up" branch is gone (see
|
|
//--- ShutdownChartCleanup). Cheap and idempotent: OnDeinit already purged, so this normally deletes
|
|
//--- nothing - it exists for the teardown paths that never reach OnDeinit (a failed OnInit).
|
|
PurgeChart();
|
|
//--- Last, and cheap by design (one global-variable delete): the claim must outlive every save above
|
|
//--- it, or a chart re-attaching during this teardown could start writing the same files mid-save.
|
|
ReleaseConfigLock();
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Sets the file/id identity a subclass constructor would otherwise |
|
|
//| repeat verbatim (ID, m_id, m_folderPath, m_fileName, pattern count)|
|
|
//+------------------------------------------------------------------+
|
|
void CExpertSignalAIBase::SetIdentity(string id, string shortId, int patternCount = 4)
|
|
{
|
|
ID = id;
|
|
m_id = shortId;
|
|
m_folderPath = eaName + "\\" + "Neural Networks" + "\\" + "State" + "\\" + m_id + "\\";
|
|
m_fileName = m_folderPath + _Symbol + "_" + IntegerToString(_Period);
|
|
m_pattern_count = patternCount;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| "Voting" that price will grow. |
|
|
//+------------------------------------------------------------------+
|
|
int CExpertSignalAIBase::LongCondition(void)
|
|
{
|
|
int result = 0;
|
|
//--- Readiness gate: live trading still requires a converged model, but an inference-only tester run
|
|
//--- may replay a model that was ACTUALLY loaded from disk even if its persisted trainingComplete flag
|
|
//--- is false. Without that exception the tester could seed/refresh dPrevSignal from the deployed model
|
|
//--- and draw chart arrows from those weights, yet this gate would still hard-zero the trading vote.
|
|
//--- A fresh random topology still cannot trade in the tester because m_modelLoadedFromDisk stays false.
|
|
//--- Census: this gate is invisible to the refresh-path counters and is a live candidate for the
|
|
//--- all-bars-zero-direction backtest - see m_voteGateBlocked.
|
|
NoteVoteGate(DoubleToSignal(dPrevSignal) == Buy);
|
|
if(!m_trainingComplete && !(m_inferenceOnly && m_modelLoadedFromDisk))
|
|
return 0;
|
|
//--- No alternation gate any more - see the removal note at m_voteGateBlocked's declaration. Under
|
|
//--- triple-barrier labels consecutive same-direction setups are ordinary and correct.
|
|
//--- "not yet studied" sentinel - dPrevSignal == -2 is not a real Sell. Its MAGNITUDE is 2, so it
|
|
//--- passed straight through the confidence floor that used to sit here (|-2| exceeds any 0..1
|
|
//--- threshold); only the m_trainingComplete gate above was keeping it out. Checked explicitly now
|
|
//--- that the floor is gone, rather than left resting on that.
|
|
if(dPrevSignal == -2)
|
|
return 0;
|
|
//--- NO confidence floor here, by design - see m_minSignalConfidence's former declaration site. A
|
|
//--- weak call is not blocked at the AI's own boundary; it votes at its tier weight (as low as
|
|
//--- m_pattern_0) and is then filtered by Min vote to open, exactly like a weak classic vote.
|
|
if(DoubleToSignal(dPrevSignal) == Buy)
|
|
{
|
|
int tier = ConfidenceTier();
|
|
result = PatternWeightForTier(tier);
|
|
m_active_pattern = "Pattern_" + IntegerToString(tier);
|
|
m_active_direction = "Buy";
|
|
}
|
|
return(result);
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| "Voting" that price will fall. |
|
|
//+------------------------------------------------------------------+
|
|
int CExpertSignalAIBase::ShortCondition(void)
|
|
{
|
|
int result = 0;
|
|
//--- Readiness gate - see LongCondition's matching comment.
|
|
NoteVoteGate(DoubleToSignal(dPrevSignal) == Sell);
|
|
if(!m_trainingComplete && !(m_inferenceOnly && m_modelLoadedFromDisk))
|
|
return 0;
|
|
//--- "not yet studied" sentinel, and no confidence floor - see LongCondition's matching comments.
|
|
if(dPrevSignal == -2)
|
|
return 0;
|
|
if(DoubleToSignal(dPrevSignal) == Sell)
|
|
{
|
|
int tier = ConfidenceTier();
|
|
result = PatternWeightForTier(tier);
|
|
m_active_pattern = "Pattern_" + IntegerToString(tier);
|
|
m_active_direction = "Sell";
|
|
}
|
|
return result;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Buckets a confidence magnitude into one of 4 equal bands between |
|
|
//| the head's own structural floor and 1.0 - see m_pattern_0's |
|
|
//| declaration comment for the resulting tier/weight table. Neither |
|
|
//| head has a configurable floor: the boundary is 1/3 for the 3-class|
|
|
//| softmax and 0.5 for regression (DoubleToSignal's own threshold), |
|
|
//| both arithmetic properties of the head rather than settings. |
|
|
//+------------------------------------------------------------------+
|
|
int CExpertSignalAIBase::ConfidenceTierFor(const double signal)
|
|
{
|
|
//--- The head's own STRUCTURAL decision boundary - the lowest confidence magnitude that head can
|
|
//--- possibly report for a directional call - not a user setting:
|
|
//--- - 3-class classification: the winning class of a 3-way softmax is arithmetically >= 1/3, since
|
|
//--- three probabilities summing to 1 cannot all be below it. Nothing can ever be read below this.
|
|
//--- - single-neuron regression: 0.5, DoubleToSignal()'s own decision boundary.
|
|
//--- Quartiling from HERE (rather than from an input, as the classification branch used to) is what
|
|
//--- makes the tier boundaries a fixed property of the model instead of something that silently moves
|
|
//--- whenever the trader adjusts an unrelated vote threshold - and it is what lets the same tier
|
|
//--- weights mean the same thing on both heads. See m_pattern_0's declaration comment for the weights.
|
|
double floorConf = (m_outputNeuronsCount == 3) ? (1.0 / 3.0) : 0.5;
|
|
double span = MathMax(1.0 - floorConf, 0.0001);
|
|
//--- RAW magnitude, not the calibrated one (fixed 2026-08-16). The bounds above describe the range
|
|
//--- the HEAD can emit - a 3-way softmax winner is arithmetically >= 1/3 - but
|
|
//--- CalibratedConfidenceMagnitude() multiplies by m_confidenceCalScale, which is clamped to
|
|
//--- [0.3, 1.5]. That lower clamp sits BELOW this floor of 1/3, so the moment calibration bottoms
|
|
//--- out the quartiling is fed values it considers impossible: t goes negative, MathFloor takes it
|
|
//--- further negative, and MathMax(0, ...) pins EVERY call to tier 0.
|
|
//---
|
|
//--- Not hypothetical - it is what the live SP500 H4 run does. m_confidenceCalScale is EMA'd toward
|
|
//--- empiricalAccuracy / avgClaimedConfidence (see its update in Training.mqh); with the model
|
|
//--- over-calling Neutral, 3-class agreement accuracy sits near 10% against a claimed confidence
|
|
//--- near 0.9, so the ratio is ~0.11 and clamps to the 0.3 floor every era. Logged result:
|
|
//--- "tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0)" - 828 calls, one bucket, and the four
|
|
//--- tier weights plus the whole per-tier pattern-DB ranking reduced to a single number.
|
|
//---
|
|
//--- Tiering wants RELATIVE confidence bucketing, which is exactly what the raw head output gives,
|
|
//--- on precisely the [floorConf, 1] range these bounds were written for. Calibration is still the
|
|
//--- right thing for consumers that need an absolute probability - AIConfidence() for MM sizing and
|
|
//--- SignedAIConfidence() for the vote both keep using it, unchanged.
|
|
double rawMag = MathAbs(signal);
|
|
if(!MathIsValidNumber(rawMag))
|
|
return 0;
|
|
double t = (MathMin(1.0, rawMag) - floorConf) / span;
|
|
int tier = (int)MathFloor(t * 4.0);
|
|
return MathMax(0, MathMin(tier, 3));
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| The live bar's tier - the only caller shape that existed before |
|
|
//| ConfidenceTierFor() was split out, kept so LongCondition()/ |
|
|
//| ShortCondition() read exactly as they did. |
|
|
//+------------------------------------------------------------------+
|
|
int CExpertSignalAIBase::ConfidenceTier(void)
|
|
{
|
|
return ConfidenceTierFor(dPrevSignal);
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| The signed vote this member would cast for a given decision, in |
|
|
//| the units CExpertSignalCustom::Direction() sums - see the |
|
|
//| declaration comment for why this is a shared function and not two |
|
|
//| parallel expressions. |
|
|
//| |
|
|
//| Mirrors the live path line for line: |
|
|
//| LongCondition/ShortCondition -> PatternWeightForTier(tier) |
|
|
//| CExpertSignalCustom::Direction() -> m_weight * (long - short) |
|
|
//| Both m_weight and the four tier weights are rewritten from the |
|
|
//| signal DB by UpdateSignalsWeights(), so this number MOVES as |
|
|
//| ranking lands. That is deliberate and it is the point: the gate |
|
|
//| has to score the vote the EA would actually have cast, not a |
|
|
//| frozen idealisation of it. |
|
|
//+------------------------------------------------------------------+
|
|
double CExpertSignalAIBase::LiveVoteContribution(const double signal)
|
|
{
|
|
ENUM_SIGNAL s = DoubleToSignal(signal);
|
|
if(s != Buy && s != Sell)
|
|
return 0.0; // abstention - live drops it from the sum AND the divisor
|
|
double w = (double)PatternWeightForTier(ConfidenceTierFor(signal));
|
|
if(!MathIsValidNumber(w) || w <= 0.0)
|
|
return 0.0; // a pattern ranked to weight 0 contributes nothing and is not a voter
|
|
double contribution = m_weight * w;
|
|
if(!MathIsValidNumber(contribution))
|
|
return 0.0;
|
|
return (s == Buy) ? contribution : -contribution;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Returns the given tier's current pattern weight (0-100) |
|
|
//+------------------------------------------------------------------+
|
|
int CExpertSignalAIBase::PatternWeightForTier(int tier)
|
|
{
|
|
switch(tier)
|
|
{
|
|
case 0:
|
|
return m_pattern_0;
|
|
case 1:
|
|
return m_pattern_1;
|
|
case 2:
|
|
return m_pattern_2;
|
|
default:
|
|
return m_pattern_3;
|
|
}
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| TURN THIS ERA'S HELD-OUT OUTCOMES INTO THE VOTE WEIGHTS. |
|
|
//| |
|
|
//| The problem this solves, in the user's own words (2026-08-18): |
|
|
//| "NNs are different from classic signals - 0.33 could change |
|
|
//| meaning as the neurons weights change". Exactly so. A classic |
|
|
//| Pattern_2 is a fixed geometric condition, so win rates for it can |
|
|
//| be accumulated over years and stay meaningful. An AI Pattern_2 |
|
|
//| means "confidence landed in tier 2", and tier 2 under era 100's |
|
|
//| weights is a different statement from tier 2 under era 500's. Any |
|
|
//| accumulated ledger of AI rows therefore describes models that no |
|
|
//| longer exist, and averages them together. |
|
|
//| |
|
|
//| WHY THIS DOES NOT WRITE TO THE SIGNAL DB, which was the obvious |
|
|
//| reading of "fill the database during training": the DB's value is |
|
|
//| accumulation, and accumulation is precisely what is wrong here. |
|
|
//| Synthetic rows would also collide with the per-table row cap and |
|
|
//| mix measured-on-holdout outcomes into the same tables the LIVE |
|
|
//| ledger uses. What the DB actually supplies is a measured win rate |
|
|
//| per pattern - and pass 3 already computes exactly that, on held- |
|
|
//| out bars, thousands at a time instead of a handful of live trades.|
|
|
//| So the AI ranks itself from that, once per era, replacing rather |
|
|
//| than accumulating - which makes the weights describe the CURRENT |
|
|
//| weights by construction. |
|
|
//| |
|
|
//| SHRINKAGE IS NOT OPTIONAL HERE. The OOS window holds ~63 |
|
|
//| EFFECTIVE observations (overlapping triple-barrier labels - see |
|
|
//| EffectiveSampleSize), so four tiers hold ~8 apiece and a raw |
|
|
//| per-tier ratio would be noise dressed as a probability. Each tier |
|
|
//| is shrunk toward this model's POOLED holdout win rate with |
|
|
//| MIN_TRADES_FOR_WIN_RATE pseudo-trades - the same estimator, and |
|
|
//| the same prior strength, UpdateSignalsWeights() applies to the |
|
|
//| classic ladders. A thin tier therefore reads as the model's own |
|
|
//| overall win rate and the tiers separate only as evidence earns it.|
|
|
//| That is the correct behaviour, not a failure to differentiate. |
|
|
//| |
|
|
//| NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather |
|
|
//| than needing a guard: weights are computed at the END of era N, |
|
|
//| so the vote scored during era N was cast with era N-1's weights. |
|
|
//| The deploy gate never grades a vote whose weights were fitted on |
|
|
//| the very bars it is scoring. Residual leakage remains and is not |
|
|
//| papered over - it is the same OOS bars each era, under a |
|
|
//| different model - which is why this feeds the VOTE and not the |
|
|
//| gate's own pass/fail arithmetic. |
|
|
//+------------------------------------------------------------------+
|
|
void CExpertSignalAIBase::RankTiersFromOos(void)
|
|
{
|
|
//--- SNAPSHOT FIRST, unconditionally - this runs at pass-3 completion, the single moment the
|
|
//--- arrow cache is complete for the era, and everything display-side (the overlay sweep, the
|
|
//--- prospective readout) reads the snapshot instead of the cache precisely because the cache is
|
|
//--- about to be wiped when the next era starts. Deliberately BEFORE the pooledFired early
|
|
//--- return: an all-Neutral era still scored every bar, and "the model says Neutral everywhere"
|
|
//--- is a snapshot worth showing, not an absence of one. Raw signals, not votes: a tier re-rank
|
|
//--- between eras then reprices them at read time (LiveVoteContribution) for free.
|
|
int snapN = MathMin(ArraySize(m_arrowSignalCache), SIGNAL_RESCAN_LOOKBACK_BARS + 16);
|
|
if(snapN > 0)
|
|
{
|
|
ArrayResize(m_overlaySigSnap, snapN);
|
|
ArrayCopy(m_overlaySigSnap, m_arrowSignalCache, 0, 0, snapN);
|
|
}
|
|
m_overlaySnapBars = snapN;
|
|
m_prospectiveSigSnap = -2.0;
|
|
for(int pi = 1; pi <= 16 && pi < snapN; pi++)
|
|
if(m_overlaySigSnap[pi] != -2.0 && MathIsValidNumber(m_overlaySigSnap[pi]))
|
|
{
|
|
m_prospectiveSigSnap = m_overlaySigSnap[pi];
|
|
break;
|
|
}
|
|
//--- ...and tell the EA a fresh snapshot exists, so the overlay redraws from it. Every member
|
|
//--- sets this each era; the EA's rate limit collapses the burst into one sweep.
|
|
g_warriorOverlayArmRequest = true;
|
|
int pooledFired = 0, pooledHits = 0;
|
|
for(int t = 0; t < 4; t++)
|
|
{
|
|
pooledFired += m_oosTierFired[t];
|
|
pooledHits += m_oosTierHits[t];
|
|
}
|
|
//--- Nothing fired this era (all-Neutral, or a stopped/cap-hit era): leave the previous era's
|
|
//--- weights standing rather than collapsing every tier to a prior built on no evidence at all.
|
|
if(pooledFired <= 0)
|
|
return;
|
|
double pooledPct = 100.0 * pooledHits / pooledFired;
|
|
//--- WHY NOT WinRateFromCounts(), which is the estimator the classic ladders use: it returns
|
|
//--- NO_DATA_WIN_RATE for anything under MIN_TRADES_FOR_WIN_RATE raw trades, BEFORE it shrinks. That
|
|
//--- floor is right there - a classic pattern with 12 live rows should be left at its default, not
|
|
//--- re-weighted - but here it would fire on every tier every era and hand all four the pooled rate,
|
|
//--- so the tiers could never separate and the whole mechanism would be inert. Shrinkage is the
|
|
//--- answer to a small sample; a floor in front of it means the shrinkage never runs.
|
|
//---
|
|
//--- ...and the sample is measured EFFECTIVE, not raw. Triple-barrier labels overlap, so N fires
|
|
//--- resolving over a mean lifespan of L bars are worth about N/L independent observations (see
|
|
//--- EffectiveSampleSize / [[project_label_overlap_effective_n]]). A tier showing 800 raw fires may
|
|
//--- carry ~12 real ones. Shrinking on the raw count would treat that as overwhelming evidence and
|
|
//--- reproduce exactly the "clears by N sigma" error that overlap invalidated everywhere else.
|
|
string line = "";
|
|
for(int t = 0; t < 4; t++)
|
|
{
|
|
int fired = m_oosTierFired[t];
|
|
int hits = m_oosTierHits[t];
|
|
double w = pooledPct;
|
|
if(fired > 0)
|
|
{
|
|
double effN = MathMax(0.0, EffectiveSampleSize((double)fired));
|
|
double effHits = effN * ((double)hits / fired);
|
|
//--- Beta prior of TIER_PRIOR_EFF_N pseudo-observations centred on this model's pooled
|
|
//--- holdout rate. Deliberately small: the prior is counted in the same EFFECTIVE units as
|
|
//--- the evidence, and a tier holds ~8-15 of those, so a prior of 100 (the classic path's)
|
|
//--- would drown every tier in the pool. At 10 a tier with ~10 effective observations sits
|
|
//--- half on its own evidence and half on the pool, which is an honest reading of what one
|
|
//--- era's holdout can support.
|
|
w = (effHits + TIER_PRIOR_EFF_N * (pooledPct / 100.0)) * 100.0 / (effN + TIER_PRIOR_EFF_N);
|
|
}
|
|
//--- Rounded to the nearest INTEGER, not to the nearest 10 as NormalizeWinRate() does. After
|
|
//--- shrinkage the tiers legitimately sit within a few points of each other, and decade rounding
|
|
//--- would collapse them back into one number - undoing the separation this exists to produce.
|
|
int wi = (int)MathMax(0, MathMin(100, MathRound(w)));
|
|
ApplyTierWeight(t, wi);
|
|
line += StringFormat(" T%d=%d(%d fires, %.1f eff)", t, wi, fired,
|
|
(fired > 0 ? EffectiveSampleSize((double)fired) : 0.0));
|
|
}
|
|
//--- MODULE WEIGHT = how much this model's opinion COUNTS in the weighted mean, which since the
|
|
//--- 2026-08-18 currency change is a trust weight and no longer a discount on the estimate. The
|
|
//--- pooled holdout win rate is the honest measure of that trust.
|
|
Weight(MathMax(0.0, MathMin(1.0, pooledPct / 100.0)));
|
|
m_tiersSelfRanked = true;
|
|
Print(ID + StringFormat(": tier weights re-ranked from %d held-out fires (pooled %.1f%%) ->%s"
|
|
" | module weight %.2f. These are the weights the NEXT era votes with.",
|
|
pooledFired, pooledPct, line, ModuleWeight()));
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Set the specified pattern's weight to the specified value |
|
|
//+------------------------------------------------------------------+
|
|
void CExpertSignalAIBase::ApplyPatternWeight(int patternNumber, int weight)
|
|
{
|
|
//--- THE SIGNAL DB DOES NOT OUTRANK THE HOLDOUT. UpdateSignalsWeights() calls this hourly for every
|
|
//--- filter it can find rows for, and for an AI filter those rows are LIVE-journaled fires
|
|
//--- accumulated across eras - i.e. across models. Once this model has measured its own tiers on
|
|
//--- held-out bars (RankTiersFromOos), that measurement describes the weights actually loaded and
|
|
//--- the DB's does not, so the DB call is declined rather than allowed to overwrite it.
|
|
//--- Without this the ranking pass would silently undo every era's self-ranking within the hour.
|
|
if(m_tiersSelfRanked)
|
|
return;
|
|
switch(patternNumber)
|
|
{
|
|
case 0:
|
|
Pattern_0(weight);
|
|
break;
|
|
case 1:
|
|
Pattern_1(weight);
|
|
break;
|
|
case 2:
|
|
Pattern_2(weight);
|
|
break;
|
|
case 3:
|
|
Pattern_3(weight);
|
|
break;
|
|
default:
|
|
break;
|
|
}
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| OnTick function |
|
|
//+------------------------------------------------------------------+
|
|
void CExpertSignalAIBase::OnTickHandler(void)
|
|
{
|
|
ScheduleTrainingIfNeeded();
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Schedules the next training pass (if one is due) and refreshes |
|
|
//| the per-tick status label. Factored out of OnTickHandler() so |
|
|
//| Warrior_EA.mq5's always-on timer (see PollTraining()) can drive |
|
|
//| this on a fixed wall-clock schedule too - training must not stall |
|
|
//| just because the market is closed and no ticks are arriving. |
|
|
//+------------------------------------------------------------------+
|
|
void CExpertSignalAIBase::ScheduleTrainingIfNeeded(void)
|
|
{
|
|
//--- stopped: no new training passes get scheduled at all (StartTraining() re-arms this).
|
|
//--- paused: still schedule so bEventStudy/dtStudied bookkeeping stays current, but Train() itself
|
|
//--- blocks at the next era boundary until resumed - keeps in-memory state coherent either way.
|
|
//--- complete: training already converged - a plain new bar must NOT re-enter Train()'s full era
|
|
//--- loop, which would otherwise reset the best-checkpoint/eta-decay tracking and run real
|
|
//--- Net.backProp() passes again, forever, once per bar, on an already-converged model (see
|
|
//--- RefreshConvergedSignal()'s declaration comment). Just keep the live signal current instead.
|
|
//--- publish this signal's current signed confidence for the intelligent trailing (and any other
|
|
//--- live-confidence consumer) - see g_LiveAISignedConfidence in Variables\ConfidenceBridge.mqh.
|
|
//--- Cheap: SignedAIConfidence() just reads the already-computed dPrevSignal.
|
|
//--- ENSEMBLE: the AVERAGE across members, not this member's own reading. Every member ran this line
|
|
//--- unconditionally, every tick, so the global was simply whichever member's OnTick happened to run
|
|
//--- last - and its two consumers are the AI early-exit route (CExpertSignalCustom::LiveSignedConfidence)
|
|
//--- and the intelligent trailing stop. So on a four-model chart an LSTM entry could be closed, and its
|
|
//--- stop moved, on the Perceptron's opinion alone, purely by scheduling order. Not the vote, not a
|
|
//--- weighted blend - an arbitrary member. (User-identified 2026-08-17.)
|
|
//--- Averaging matches how the ensemble actually trades: the open decision is the weighted-average
|
|
//--- vote, and a member that abstains contributes 0 and dilutes, exactly as it does there. Members
|
|
//--- still training read 0 from SignedAIConfidence(), so a half-trained ensemble reads WEAKER rather
|
|
//--- than louder, which is the safe direction for an exit trigger.
|
|
//--- Currently latent, and worth keeping that way deliberately: Min_Vote_Close ships Disabled (101,
|
|
//--- unreachable on both scales it drives) and TrailingStrategy is off, so neither consumer fires
|
|
//--- today. This is fixed now precisely because the plan is to enable vote exits once the models are
|
|
//--- accurate - at which point a scheduling-order exit would be actively harmful and very hard to see.
|
|
//--- PUBLISH ONLY THIS SIGNAL'S OWN VOTE. Combining is the orchestrator's job, never a member's - see
|
|
//--- the vote board in Variables\ConfidenceBridge.mqh for the scheduling bug this replaces and for why
|
|
//--- "have a member average its siblings" was the wrong shape of fix in a codebase whose whole point is
|
|
//--- that signals do not reach into each other. A solo AI signal owns slot 0 and the aggregate is then
|
|
//--- just its own value, so the non-ensemble path is unchanged.
|
|
PublishAIVote(m_ensembleMember ? m_ensembleIndex : 0, SignedAIConfidence());
|
|
datetime lastBarDate = (datetime)SeriesInfoInteger(m_symbol.Name(), m_period, SERIES_LASTBAR_DATE);
|
|
// A failed lookup (0) must not silently read as "dtStudied is already caught up, nothing pending" -
|
|
// that would freeze this function into never re-triggering training/signal refresh again until some
|
|
// other path happens to bump dtStudied. Treat a failed lookup as pending instead (same >0-guard
|
|
// philosophy as the SERIES_FIRSTDATE lookup elsewhere in this class) so a transient history-sync
|
|
// hiccup costs one extra harmless check, not a silent stall.
|
|
bool newBarPending = (dPrevSignal == -2 || lastBarDate <= 0 || ((m_inferenceOnly ? m_lastBarTime : dtStudied) < lastBarDate));
|
|
//--- A MODEL THAT HAS NOT FINISHED TRAINING IS ALWAYS PENDING. The watermark test above answers
|
|
//--- "has a new bar closed since the last one we processed", which is the right question for a
|
|
//--- CONVERGED model (one inference refresh per bar, see the branch below) and the wrong one for a
|
|
//--- training run: Train() is chunked - it does ~TRAIN_TIME_BUDGET_MS of work and yields, needing
|
|
//--- thousands of calls to finish a single era - yet every one of those calls has to be armed from
|
|
//--- here. Gating them on the watermark meant training could only advance when a new BAR closed.
|
|
//--- On H1 that is one 120ms chunk per hour.
|
|
//--- The trap is that dtStudied is two different things: Train() sets it to the training WINDOW
|
|
//--- START (TrainWindowStart, ~2008) while FinalizeTrainRun() sets it to the last bar SCANNED
|
|
//--- (~now). So the moment any run finalized, dtStudied >= lastBarDate and this function went
|
|
//--- silent until the next candle - no era lines, no heartbeats, no stall branch, nothing, because
|
|
//--- Train() was not being CALLED at all. Observed 2026-08-10: four charts, 28 minutes of silence
|
|
//--- between two bursts exactly one H1 bar apart, and the TRAIN STALL line that finally caught it
|
|
//--- reported runActive=Y only because m_trainRunActive had been set microseconds earlier in that
|
|
//--- same call. Before 0c85c54 this was survivable rather than correct: eras were nearly free (the
|
|
//--- saved watermark left almost no bars eligible), so one call per bar still looked like progress.
|
|
//--- There is never a reason to withhold a Train() call from an unconverged model - pause/stop are
|
|
//--- handled by m_trainingPaused/m_trainingStopRequested, which Train() checks for itself.
|
|
//--- A POST-TRAINING WALK IS ALSO PENDING. Both of these are armed at the moment convergence is
|
|
//--- declared (Training.mqh's era-end block: StartOosContinualSimulation + StartPatternDatabaseBackfill)
|
|
//--- and both advance ONLY from inside Train(), one time-boxed chunk per call - so they need calls
|
|
//--- armed from here exactly like a training run does. Without this term they never got any: the
|
|
//--- convergence that arms them also sets m_trainingComplete, and FinalizeTrainRun() clears
|
|
//--- m_trainRunActive one line earlier, so the branch below took the converged path from that instant
|
|
//--- on and ArmStudyEvent() - the only per-tick arming site in the EA - was never reached again.
|
|
//--- Train() was simply never called, so both walks sat at their start index forever: the
|
|
//--- continual-learning OOS simulation never produced its "simulation complete" line, and the
|
|
//--- pattern-database backfill never wrote a row (the DB it exists to fill stayed empty, which is
|
|
//--- indistinguishable from the feature being absent). Only a manual Resume/Retrain click - which
|
|
//--- arms an event through a different path - could ever unstick them. Found 2026-08-16.
|
|
bool postTrainWalkPending = (m_simOosRunActive || m_dbBackfillActive);
|
|
bool trainingPending = !(m_trainingComplete || m_inferenceOnly) || postTrainWalkPending;
|
|
//--- m_inferenceOnly (single backtest) takes the converged/inference branch even if the seeded model
|
|
//--- wasn't flagged complete, so a backtest never drops into Train()'s era loop - see m_inferenceOnly.
|
|
if((m_trainingComplete || m_inferenceOnly) && !m_trainingStopRequested && !m_trainRunActive &&
|
|
!postTrainWalkPending)
|
|
{
|
|
if(newBarPending)
|
|
RefreshConvergedSignal();
|
|
}
|
|
else
|
|
{
|
|
//--- Lost-event watchdog: an armed event that never arrived (chart-event queue overflow) would
|
|
//--- otherwise leave bEventStudy true forever and silently stall training - the accidental rescue
|
|
//--- (any sibling's event clearing this flag) went away with the per-instance ids. See
|
|
//--- STUDY_EVENT_LOST_MS for why a false trip is not a realistic concern.
|
|
if(bEventStudy && GetTickCount() - m_studyArmedTick > STUDY_EVENT_LOST_MS)
|
|
{
|
|
Print(ID + ": study event armed " + IntegerToString(STUDY_EVENT_LOST_MS / 1000) +
|
|
"s ago never arrived (chart-event queue overflow?) - re-arming.");
|
|
bEventStudy = false;
|
|
}
|
|
if(!m_trainingStopRequested && !bEventStudy && (newBarPending || trainingPending))
|
|
ArmStudyEvent((long)MathMax(0, MathMin(iTime(m_symbol.Name(), PERIOD_CURRENT, (int)(100 * Net.recentAverageSmoothingFactor * (m_trainingComplete ? 1 : 10))), dtStudied)), "New Bar");
|
|
}
|
|
//--- Train() (see its declaration comment) now yields every ~TRAIN_TIME_BUDGET_MS instead of
|
|
//--- blocking for a whole era, so while a run is active this per-tick line would otherwise
|
|
//--- overwrite Train()'s own full-detail status label on every single tick between chunks -
|
|
//--- flickering between the two instead of showing one steady picture. Only write this terse
|
|
//--- summary when nothing else is actively updating the status label (idle/stopped/paused/cooldown).
|
|
//--- ...and the same reasoning covers a member held at the ENSEMBLE ERA BARRIER, which the
|
|
//--- !m_trainRunActive test above does not: a held member returns from Train() before it ever sets
|
|
//--- m_trainRunActive, so BOTH writers considered themselves the only one updating the label and
|
|
//--- fought over it every tick. That is the "Getting ready..." <-> "Waiting at era N for slower
|
|
//--- ensemble members" flicker reported on 2026-08-17 - and it appeared on Perceptron but not
|
|
//--- Convolutional purely because Convolutional had a run active from a completed era and Perceptron,
|
|
//--- resumed from disk, never did. Train()'s message is the specific one, so it wins.
|
|
if(m_ensembleMember && EnsembleEraBarrierHolds())
|
|
return;
|
|
if(!m_trainRunActive)
|
|
{
|
|
//--- Compact, accurate end-state text. The completed state distinguishes a model that is genuinely
|
|
//--- adapting live (online learning active - a live chart with EnableOnlineLearning, not the tester)
|
|
//--- from one running pure inference (the Strategy Tester, or online learning off), so the label is
|
|
//--- literally true either way and never over-promises "keeps learning" when it doesn't - see
|
|
//--- OnlineLearnStep()'s gate for exactly when adaptation runs.
|
|
bool onlineActive = m_enableOnlineLearning && !m_inferenceOnly
|
|
&& !MQLInfoInteger(MQL_TESTER) && !MQLInfoInteger(MQL_OPTIMIZATION) && !MQLInfoInteger(MQL_FORWARD)
|
|
&& CheckPointer(Net) != POINTER_INVALID && !Net.CpuInference();
|
|
//--- Simple end-state panel (default, VerboseMode off): plain-language status + the model's
|
|
//--- compounded/persistent Buy/Sell win-rate (directional accuracy, Neutral excluded - persisted in
|
|
//--- .stats WST5, so it survives a fresh chart reload and is meaningful the moment a drop-and-go user
|
|
//--- attaches the EA) + the current call. The verbose era/forecast dump below stays for power users.
|
|
if(!VerboseMode)
|
|
{
|
|
string statusPlain;
|
|
if(m_trainingComplete)
|
|
statusPlain = onlineActive ? "Live - learning from new bars" : "Ready for live trading";
|
|
else if(m_trainingStopRequested)
|
|
statusPlain = "Paused - progress saved";
|
|
else if(m_trainingPaused)
|
|
statusPlain = "Paused";
|
|
else
|
|
statusPlain = "Getting ready...";
|
|
ENUM_SIGNAL liveSig = DoubleToSignal(dPrevSignal);
|
|
string liveSigPlain = (liveSig == Buy) ? "Buy" : (liveSig == Sell) ? "Sell" : "Neutral (no trade)";
|
|
string simpleLive = DisplayName() + " - " + statusPlain + "\n";
|
|
//--- Only show the accuracy line once at least one signal has been validated (compounded counts
|
|
//--- persist across restarts, so a deployed model shows real numbers immediately, not "measuring").
|
|
if(m_cumIsTotal > 0 || m_cumOosTotal > 0)
|
|
simpleLive += ComputeCompoundedAccuracyLine() + "\n";
|
|
//--- The ensemble headline is the FIRST line, so lead with the signal - on the combined
|
|
//--- panel each member's one line must answer "what is this model saying right now".
|
|
if(m_ensembleMember)
|
|
{
|
|
simpleLive = statusPlain + " -> " + liveSigPlain;
|
|
//--- one-line accuracy on the member headline (user request 2026-08-16: the solo panel
|
|
//--- shows accuracy, the ensemble panel did not). Same counters and same
|
|
//--- always-show-the-break-even doctrine as ComputeCompoundedAccuracyLine - a win rate
|
|
//--- without its geometry's base rate reads as skill when it is chance.
|
|
if(m_cumOosTotal > 0)
|
|
{
|
|
double slBeH = 0.0, tpBeH = 0.0;
|
|
BarrierMultiples(slBeH, tpBeH);
|
|
simpleLive += StringFormat(" | win %d%%", (int)MathRound(m_cumOosCorrect * 100.0 / m_cumOosTotal));
|
|
if(slBeH > 0.0 && tpBeH > 0.0)
|
|
simpleLive += StringFormat(" (need %d%%)", (int)MathRound(100.0 * slBeH / (slBeH + tpBeH)));
|
|
}
|
|
}
|
|
PublishStatus(simpleLive);
|
|
return;
|
|
}
|
|
string completeText = onlineActive ? "Complete - live (adapting to new bars)" : "Complete - ready for live (inference)";
|
|
string trainingState = m_trainingStopRequested
|
|
? (m_trainingComplete ? completeText : "Stopped - resumable (weights kept)")
|
|
: (m_trainingPaused ? "Paused" : (m_trainingComplete ? completeText : "In progress"));
|
|
//--- same "Forecast: <signal> -> <value>" line the active training loop's status label ends on
|
|
//--- (see the classLine-terminated StringFormat below), instead of a raw bEventStudy/dPrevSignal/
|
|
//--- dtStudied debug dump - this is what stays on screen once training stops/pauses/completes.
|
|
PublishStatus(StringFormat(
|
|
ID + " : Era %d -> Training %s\n" +
|
|
"Forecast: %s -> %.2f",
|
|
m_eraCount, trainingState,
|
|
EnumToString(DoubleToSignal(dPrevSignal)), dPrevSignal));
|
|
}
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Timer-driven equivalent of OnTickHandler()'s scheduling, called |
|
|
//| from Warrior_EA.mq5's always-on OnTimer() so training keeps |
|
|
//| progressing purely on wall-clock time - no dependency on ticks, |
|
|
//| which simply don't arrive while the market is closed. |
|
|
//+------------------------------------------------------------------+
|
|
void CExpertSignalAIBase::PollTraining(void)
|
|
{
|
|
//--- Drain a slice of the queued chart-arrow restore FIRST, and skip training work on any tick where
|
|
//--- restoring is still in flight. Both compete for the one MQL5 thread; letting the arrows finish
|
|
//--- quickly (a few hundred ms of slices) means the user sees a complete chart almost immediately,
|
|
//--- whereas interleaving them with 80ms training chunks would stretch the restore over minutes.
|
|
//--- STOP BEFORE ANY OF IT. Everything below either draws objects on the chart (the arrow restore and
|
|
//--- the rescan) or starts a training chunk, and OnDeinit is about to delete every one of those objects
|
|
//--- again - while the budget it has to do it in is already running. Drawing arrows for a chart that is
|
|
//--- unloading is the worst case of the three: it lengthens the very sweep that has to finish.
|
|
if(ShutdownRequested())
|
|
return;
|
|
if(m_arrowRestorePending)
|
|
{
|
|
AdvanceChartSignalRestore();
|
|
return;
|
|
}
|
|
//--- Same one-thread reasoning as the arrow restore above: a manual rescan (Show Signals) also competes
|
|
//--- for the single MQL5 thread, and its per-bar feedForward is real compute rather than a cheap object
|
|
//--- write, so it must finish its own slices before training resumes rather than interleaving with it.
|
|
if(m_rescanPending)
|
|
{
|
|
AdvanceChartSignalRescan();
|
|
return;
|
|
}
|
|
if(m_isInitialized)
|
|
ScheduleTrainingIfNeeded();
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| |
|
|
//+------------------------------------------------------------------+
|
|
void CExpertSignalAIBase::OnChartEventHandler(const int id,
|
|
const long &lparam,
|
|
const double &dparam,
|
|
const string &sparam)
|
|
{
|
|
//--- Match THIS instance's id only. CExpertCustom broadcasts every chart event to every filter, so
|
|
//--- matching a shared id here (the old `id == 1001`) made each posted event run a train chunk in
|
|
//--- ALL ensemble members - N members posting per round times N members handling each = N*N chunks,
|
|
//--- and the chart thread never idled long enough to deliver the control panel's clicks and drags.
|
|
//--- See STUDY_EVENT_ID_BASE (ExpertSignalAIBase.mqh) for the full failure shape.
|
|
if(id == CHARTEVENT_CUSTOM + m_studyEventId)
|
|
{
|
|
//--- The study event IS the training driver, so a queued one landing after the stop request would
|
|
//--- open a whole era inside the teardown window. Checked here as well as in Warrior_EA.mq5's
|
|
//--- OnChartEvent because CExpertCustom re-posts these between members.
|
|
if(ShutdownRequested())
|
|
return;
|
|
TuneIndicatorsAndTrain(lparam);
|
|
bEventStudy = false;
|
|
OnTickHandler();
|
|
}
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Claim m_activeFileName for this chart, terminal-wide. |
|
|
//| |
|
|
//| Two charts running the same AIType with the same retrain-affecting|
|
|
//| inputs resolve to the SAME .nnw/.cfg/.stats/checkpoint set. Both |
|
|
//| then train independently and save over each other, so whichever |
|
|
//| writes last wins and the other's eras are discarded - silently, |
|
|
//| because every individual file operation succeeds. A five-chart |
|
|
//| comparison run on 2026-07-29 lost both its HYBRID models this way |
|
|
//| (one chart left at the AIType default), and the only evidence |
|
|
//| anywhere was that model path appearing twice as often in the log. |
|
|
//| |
|
|
//| A terminal-wide global variable is the right lock rather than a |
|
|
//| lock FILE: GlobalVariableTemp() is an atomic create-if-absent, |
|
|
//| and a TEMPORARY variable dies with the terminal, so a crash can |
|
|
//| never leave a stale lock that blocks the next start. Within one |
|
|
//| session a stale entry is still possible (an EA removed without a |
|
|
//| clean deinit), so the owner's chart id is stored and revalidated. |
|
|
//+------------------------------------------------------------------+
|
|
bool CExpertSignalAIBase::AcquireConfigLock(void)
|
|
{
|
|
//--- FNV-1a over the resolved filename: every retrain-affecting input is already folded into that
|
|
//--- name, so equal names mean genuinely equal configs and nothing else has to be compared. Hashed
|
|
//--- because MQL5 caps global-variable names at 63 characters and the path alone exceeds that.
|
|
uint h = 2166136261;
|
|
int len = StringLen(m_activeFileName);
|
|
for(int i = 0; i < len; i++)
|
|
{
|
|
h ^= (uint)StringGetCharacter(m_activeFileName, i);
|
|
h *= 16777619;
|
|
}
|
|
string name = "WarriorAI_" + m_id + "_" + StringFormat("%08x", h);
|
|
long self = ChartID();
|
|
//--- Atomic: true means it did not exist and is now ours.
|
|
if(GlobalVariableTemp(name))
|
|
{
|
|
GlobalVariableSet(name, (double)self);
|
|
m_configLockName = name;
|
|
return true;
|
|
}
|
|
long owner = (long)GlobalVariableGet(name);
|
|
//--- Our own entry: this chart is re-initializing after a parameter change or a recompile whose
|
|
//--- OnDeinit never reached ReleaseConfigLock(). Reclaim it instead of refusing to start.
|
|
if(owner == self)
|
|
{
|
|
m_configLockName = name;
|
|
return true;
|
|
}
|
|
//--- Owner recorded but its chart no longer runs an expert - take the claim over. owner == 0 is
|
|
//--- deliberately NOT treated as stale: it means another instance created the variable microseconds
|
|
//--- ago and has not stamped its id yet, which is a live claim, not a dead one.
|
|
bool ownerAlive = false;
|
|
if(owner != 0)
|
|
{
|
|
long id = ChartFirst();
|
|
while(id >= 0)
|
|
{
|
|
if(id == owner)
|
|
{
|
|
ownerAlive = (StringLen(ChartGetString(id, CHART_EXPERT_NAME)) > 0);
|
|
break;
|
|
}
|
|
id = ChartNext(id);
|
|
}
|
|
}
|
|
if(owner != 0 && !ownerAlive)
|
|
{
|
|
GlobalVariableSet(name, (double)self);
|
|
m_configLockName = name;
|
|
return true;
|
|
}
|
|
Print(ID + ": REFUSED to start - another chart is already training this exact configuration. Both" +
|
|
" would save into the same files (" + m_activeFileName + ".nnw plus its .cfg/.stats/checkpoints)" +
|
|
" and overwrite each other's progress with no error reported anywhere. Owner: " +
|
|
(owner != 0 ? "chart " + IntegerToString(owner) + " (" + ChartSymbol(owner) + " " +
|
|
EnumToString((ENUM_TIMEFRAMES)ChartPeriod(owner)) + ")" : "another chart, still initializing") +
|
|
". Change AIType or any retrain-affecting input on THIS chart so it trains its own model, or" +
|
|
" remove one of the two charts. Note AIType defaults to " + EnumToString(AI_HYBRID) +
|
|
" - a chart whose AIType was never actually changed lands here.");
|
|
return false;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Drop this instance's claim (see AcquireConfigLock). |
|
|
//+------------------------------------------------------------------+
|
|
void CExpertSignalAIBase::ReleaseConfigLock(void)
|
|
{
|
|
if(StringLen(m_configLockName) == 0)
|
|
return;
|
|
//--- Only delete a claim we still hold: if a later instance took this entry over via the stale-owner
|
|
//--- path above, deleting it here would silently hand the config to a third chart.
|
|
if((long)GlobalVariableGet(m_configLockName) == ChartID())
|
|
GlobalVariableDel(m_configLockName);
|
|
m_configLockName = "";
|
|
}
|
|
#endif
|