ответвлён от animatedread/Warrior_EA
afe1038d11
1. THE POOL FIX WAS LANDING ON A TOPOLOGY THAT COULD NOT SEE IT. ComputeFirstLayerWidth budgets against EstimatedInSampleBars, which counts this chart's own bars PLUS the training pool. On a COLD fleet start every chart derives and pins its topology BEFORE any chart has published a pool file - measured on the 18:13 start, model creation at 18:13:21 against a first publish at 18:13:48. All six sized as if training alone, wrote that into .cfg, and adopted it back on every later start even with the pool full. SP500 ran a first layer floored to 16 while adopting 30229 peer rows. Adopt-don't-compare exists to protect weights shaped by those sizes. It was also running for a model with NO .nnw, where there is nothing to protect and the .cfg is just a record of one unlucky moment. The four derived sizes are now re-measured when no weights exist. Safe on all three counts that matter: free (nothing to discard), cannot loop (once weights exist the .cfg is authoritative again), and cannot fragment the pool - the derived width is NOT in BuildModelFingerprint, which keys only on the FEATURE layout. Verified: field 2 of the fingerprint is LEGACY_HISTORY_BARS_SLOT, not the first-layer width. TO TAKE EFFECT the weights must be wiped while the TrainPool is KEPT - the census has to be non-empty at derivation time. A full wipe empties the pool and reproduces the original condition exactly. 2. THE KEEP-SCREEN LATCHED ON AN UNDERPOWERED SAMPLE. MI_MIN_SAMPLES is a floor for "can this be computed", and it was being used as the bar for "is this answer final". The screen fired on the first era clearing 200 rows and latched, measuring at 202-773 samples where a warm chart gives ~2065. Columns kept then tracked SAMPLE SIZE rather than information - EURUSD kept 0 of 49 at n=202, SP500 kept 15 at n=773, and the ordering across all six charts was very nearly monotone in n. A thin sample is still measured and printed, but it no longer closes the question: below MI_GOOD_SAMPLE_FRACTION of the target the result is labelled underpowered and a later era supersedes it, bounded by the same attempt budget. An underpowered screen that latches is worse than one that waits, because it looks like a result. Build tag -> fleet-pool-v2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
511 строки
34 КиБ
MQL5
511 строки
34 КиБ
MQL5
//+------------------------------------------------------------------+
|
|
//+------------------------------------------------------------------+
|
|
//| Topology.mqh |
|
|
//| |
|
|
//| The network BOOT SEQUENCE: InitNeuralNetwork() (indicator init, |
|
|
//| config-lock, tester-cache seeding, load/save the .cfg, net load |
|
|
//| with backend fallback, chart-signal restore, arm the first study |
|
|
//| event) and InitFeatureIndicators() (the ~15 per-feature indicator|
|
|
//| Init* calls that size m_neuronsCount). This is orchestration |
|
|
//| across nearly every other collaborator - chart, persistence, |
|
|
//| online-learning, cross-asset - not shape derivation, so it stays |
|
|
//| a raw-include partial of CExpertSignalAIBase rather than moving |
|
|
//| into a collaborator of its own; see project_oop_module_pattern |
|
|
//| memory for the "diagnose before applying the pattern" doctrine |
|
|
//| this follows. The FINGERPRINT, the derived shape (width/taper/ |
|
|
//| depth/conv filters/LSTM hidden), the conv/LSTM/batch-norm stages |
|
|
//| and BuildFreshTopology are a genuinely separable job and now |
|
|
//| live in Expert\Topology\Topology.mqh as CTopology, reached |
|
|
//| through the one-line forwards on CExpertSignalAIBase. |
|
|
//+------------------------------------------------------------------+
|
|
#ifndef WARRIOR_AIBASE_TOPOLOGY_MQH
|
|
#define WARRIOR_AIBASE_TOPOLOGY_MQH
|
|
//+------------------------------------------------------------------+
|
|
//| Common network bootstrap shared by every AI signal: sets up |
|
|
//| indicators, then loads a saved network or builds a fresh one |
|
|
//| whose only per-signal-type difference is AddCustomLayers(). |
|
|
//+------------------------------------------------------------------+
|
|
bool CExpertSignalAIBase::InitNeuralNetwork(CIndicators *indicators)
|
|
{
|
|
if(m_isInitialized)
|
|
return true;
|
|
if(indicators == NULL)
|
|
return false;
|
|
m_indicatorsPtr = indicators;
|
|
if(!CExpertSignalCustom::InitIndicators(indicators))
|
|
return false;
|
|
if(!InitFeatureIndicators(indicators))
|
|
return false;
|
|
//--- Kick the terminal's async history sync for every cross-asset reference symbol NOW, at init,
|
|
//--- so the ~minute of cross-symbol download runs while the model loads and the label cache
|
|
//--- prebuilds - instead of starting only when the first Build() call finds the symbols unselected
|
|
//--- and the first era (and the one-shot MI report) runs with the panel absent. Non-blocking.
|
|
if(m_useCrossAsset)
|
|
m_crossAsset.Warm((ENUM_TIMEFRAMES)m_period);
|
|
Net = new CNet(NULL);
|
|
if(CheckPointer(Net) == POINTER_INVALID)
|
|
return false;
|
|
//--- Size the first dense layer to the data. InitIndicators() above is what finalises
|
|
//--- m_neuronsCount, so this is the earliest point the input width is actually known. ORDER
|
|
//--- MATTERS, and it changed on 2026-08-09.
|
|
m_historyBars = DeriveHistoryBars();
|
|
m_convFilterCount = ComputeConvFilterCount();
|
|
m_lstmHiddenSize = ComputeLstmHiddenSize();
|
|
m_initialNeuronsCount = ComputeFirstLayerWidth();
|
|
//--- Depth LAST of the four: it is derived from the first-layer width above, so it cannot be settled
|
|
//--- before that one is. All four are overwritten from the .cfg further below if this configuration
|
|
//--- already has a trained model - see the adopt-don't-compare block there.
|
|
m_hiddenLayersCount = ComputeHiddenLayerCount();
|
|
//--- The name used to carry a dense-depth tag ("Perceptron 3L"), from when AIType let a user
|
|
//--- pick MLP_3L vs MLP_4L and the depth was the only thing separating two charts of the same
|
|
//--- family.
|
|
string fp = BuildModelFingerprint();
|
|
//--- FNV-1a 32-bit -> 8 hex chars: compact, deterministic, order-stable, collision-safe enough for
|
|
//--- the small optimizer grids in play (a collision would merely fail the .cfg guard and retrain).
|
|
uint fpHash = 2166136261;
|
|
int fpLen = StringLen(fp);
|
|
for(int fpi = 0; fpi < fpLen; fpi++)
|
|
{
|
|
fpHash ^= (uint)StringGetCharacter(fp, fpi);
|
|
fpHash *= 16777619;
|
|
}
|
|
m_fileName += "_" + DoubleToString(MathRound(m_outputNeuronsCount)) + "_" + DoubleToString(MathRound(m_optimizationAlgo)) + "_" + StringFormat("%08x", fpHash);
|
|
//--- Finish the display name with the model's short id and the leading 4 hex digits of that same
|
|
//--- fingerprint, so every log line and panel names the model file it belongs to. Their files
|
|
//--- were never at risk; the TAG was simply unable to do its one job.
|
|
string cfgTag = " [" + m_id + "-" + StringSubstr(StringFormat("%08x", fpHash), 0, 4) + "]";
|
|
if(StringFind(ID, cfgTag) < 0)
|
|
ID += cfgTag;
|
|
//--- One self-verifying config line per chart, deliberately NOT gated on VerboseMode. A multi-
|
|
//--- chart comparison is only valid if every chart is identical except the axis under test, and
|
|
//--- until now a drifted setting was invisible: the filename carries a HASH, so two charts that
|
|
//--- should match and do not look merely "different" with no indication of WHICH field moved.
|
|
Print(ID + ": config - " + IntegerToString(m_hiddenLayersCount) + " dense from " +
|
|
IntegerToString(m_initialNeuronsCount) + " units | batchnorm " +
|
|
((EnableBatchNorm && BatchNormWindow > 1) ? "ON(" + IntegerToString(BatchNormWindow) + ")" : "OFF") +
|
|
//--- "requested", not effective: the delivered tau is capped against the head's usable
|
|
//--- logit range and cannot be known until the class priors are measured -
|
|
//--- ApplyLogitAdjustment logs the value in force.
|
|
" | class-imbalance logit-adjust(tau 1.00 requested)" +
|
|
" | input " + IntegerToString((int)m_historyBars * m_neuronsCount) +
|
|
" (" + IntegerToString((int)m_historyBars) + " bars x " + IntegerToString(m_neuronsCount) + ")" +
|
|
//--- The front-end stages are DERIVED (see ComputeConvFilterCount/ComputeLstmHiddenSize),
|
|
//--- so without them this "self-verifying" line verified only half the topology - it
|
|
//--- printed the dense taper while the conv/recurrent stages that actually dominate
|
|
//--- CONV/LSTM/HYBRID were invisible.
|
|
FrontEndConfigSummary());
|
|
//--- Kept as its own line and deliberately free of any per-chart prefix INSIDE the string, so
|
|
//--- the six startup lines diff textually against each other.
|
|
Print(ID + ": fingerprint - " + fp);
|
|
//--- The resolved path is DebuggingMode-only: the tag above already names the folder (its m_id half)
|
|
//--- and the file's hash suffix (its hex half), so this line is derivable rather than new information,
|
|
//--- and a third startup line per chart is not worth spending on a user who will never open the file.
|
|
if(DebuggingMode)
|
|
Print(ID + ": model file - " + m_fileName + ".nnw");
|
|
//--- Strategy Tester / optimizer: target a LOCAL (agent-sandboxed, non-FILE_COMMON) cache file
|
|
//--- instead of the shared production weights, so genetic/complete optimization passes on this
|
|
//--- same agent can reuse an already-trained model whenever the topology-relevant inputs
|
|
//--- (neuron counts, layers, history bars, output count, opt algo, study period, ...) are
|
|
//--- unchanged from a previous pass, instead of re-running every training era from scratch each
|
|
//--- pass. The live/manual-chart production .nnw/.cfg under FILE_COMMON are never touched by
|
|
//--- this path, so a backtest can never corrupt or overwrite the deployed live model.
|
|
bool inTesterOrOpt = MQLInfoInteger(MQL_TESTER) || MQLInfoInteger(MQL_OPTIMIZATION) || MQLInfoInteger(MQL_FORWARD);
|
|
m_activeFileName = inTesterOrOpt ? (m_fileName + "_optcache") : m_fileName;
|
|
m_activeFileCommon = !inTesterOrOpt;
|
|
//--- Claim these files before anything reads or writes them, and refuse to start if another
|
|
//--- chart in this terminal already holds them (see AcquireConfigLock).
|
|
if(!inTesterOrOpt && !AcquireConfigLock())
|
|
return false;
|
|
//--- Any Strategy-Tester run - a single backtest OR an optimization pass - runs pure inference
|
|
//--- on the deployed model, never trains.
|
|
m_inferenceOnly = MQLInfoInteger(MQL_TESTER);
|
|
//--- Seed the agent-local optcache from the deployed production model on the first tester/opt
|
|
//--- pass. Re-seeds when the cache is MISSING *or* STALE. Copies FROM FILE_COMMON (the
|
|
//--- live/manual-chart model) INTO the agent-local sandbox only; the production files are read,
|
|
//--- never written, so a backtest still can't corrupt the deployed model.
|
|
bool cacheMissing = !FileIsExist(m_activeFileName + ".nnw");
|
|
bool cacheStale = false;
|
|
if(inTesterOrOpt && !cacheMissing && FileIsExist(m_fileName + ".nnw", FILE_COMMON))
|
|
{
|
|
datetime prodModified = (datetime)FileGetInteger(m_fileName + ".nnw", FILE_MODIFY_DATE, true);
|
|
datetime cacheModified = (datetime)FileGetInteger(m_activeFileName + ".nnw", FILE_MODIFY_DATE, false);
|
|
//--- both timestamps must be readable before trusting the comparison; a 0 means "couldn't tell",
|
|
//--- and re-seeding on an unreadable timestamp every single pass would be worse than not checking.
|
|
cacheStale = (prodModified > 0 && cacheModified > 0 && prodModified > cacheModified);
|
|
if(cacheStale)
|
|
Print(__FUNCTION__ + ": the deployed model is newer than this agent's cached copy - re-seeding so the backtest runs the CURRENT model, not the previously cached one.");
|
|
}
|
|
if(inTesterOrOpt && (cacheMissing || cacheStale))
|
|
{
|
|
if(FileIsExist(m_fileName + ".nnw", FILE_COMMON))
|
|
{
|
|
//--- The .nnw is the only copy that MUST succeed - retried (see CopyFileWithRetry's
|
|
//--- declaration comment) because a live chart's own atomic Save() can be mid-rename on
|
|
//--- this exact file.
|
|
if(CopyFileWithRetry(m_fileName + ".nnw", m_activeFileName + ".nnw"))
|
|
{
|
|
//--- Best-effort sidecars: not retried - losing one just means a cold
|
|
//--- calibration/shadow-blend start rather than a wrong/untrained model, which the .nnw
|
|
//--- copy above already guards against.
|
|
if(FileIsExist(m_fileName + ".cfg", FILE_COMMON))
|
|
CopySharedFile(m_fileName + ".cfg", m_activeFileName + ".cfg", false);
|
|
if(FileIsExist(m_fileName + "_shadow.nnw", FILE_COMMON))
|
|
CopySharedFile(m_fileName + "_shadow.nnw", m_activeFileName + "_shadow.nnw", false);
|
|
//--- carry the calibration sidecar into the agent sandbox too, so a seeded backtest calibrates its
|
|
//--- live decisions with the deployed model's priors instead of the un-adjusted cold defaults.
|
|
if(FileIsExist(m_fileName + ".stats", FILE_COMMON))
|
|
CopySharedFile(m_fileName + ".stats", m_activeFileName + ".stats", false);
|
|
Print(__FUNCTION__ + ": seeded tester cache from the deployed production model (" + m_fileName + ") - this run reuses the deployed weights instead of retraining");
|
|
}
|
|
//--- else: CopyFileWithRetry already logged why. Fall through - the Net.Load() below will
|
|
//--- correctly report "no file" and BuildFreshTopology() takes over, same as a genuine first pass.
|
|
}
|
|
else if(m_inferenceOnly)
|
|
//--- Name the exact file (symbol + timeframe + config fingerprint) it looked for: the
|
|
//--- model is keyed on the CHART TIMEFRAME, so the #1 cause of this is running the tester
|
|
//--- on a different timeframe than the model was trained on (e.g. an H4 model, tester set
|
|
//--- to H1) - which reads as "no model" when one exists under a different timeframe.
|
|
Print(__FUNCTION__ + ": WARNING - no deployed production model found at '" + m_fileName +
|
|
".nnw' (shared folder) for " + _Symbol + " " + EnumToString((ENUM_TIMEFRAMES)_Period) +
|
|
". A single backtest runs inference only and will NOT train. Most common cause: the tester" +
|
|
" timeframe differs from the one the model was trained on (the filename is keyed on timeframe)." +
|
|
" Otherwise, train this configuration on a chart first, then re-run the backtest.");
|
|
}
|
|
//--- ADOPT-DON'T-COMPARE PROTECTS TRAINED WEIGHTS. WITH NO WEIGHTS THERE IS NOTHING TO PROTECT.
|
|
//---
|
|
//--- The .cfg pins four DERIVED sizes (first-layer width, depth, conv filters, LSTM hidden), and
|
|
//--- adopting them is right for a model that has weights shaped by them. It was also being done
|
|
//--- for a model with NO .nnw at all, and that turned one unlucky moment into a permanent
|
|
//--- property of the fleet:
|
|
//---
|
|
//--- ComputeFirstLayerWidth budgets against EstimatedInSampleBars, which counts this chart's own
|
|
//--- bars PLUS the training pool. On a COLD fleet start every chart derives its topology before
|
|
//--- any chart has published a pool file - measured 2026-08-26, model creation 18:13:21 against a
|
|
//--- first publish at 18:13:48 - so all six sized as if training alone, wrote that into .cfg, and
|
|
//--- then adopted it back on every subsequent start even though the pool had been full for hours.
|
|
//--- SP500 sat at a first layer floored to 16 while adopting 30229 peer rows.
|
|
//---
|
|
//--- Re-deriving when there are no weights is free (nothing to discard), cannot loop (once weights
|
|
//--- exist the .cfg is authoritative again), and cannot fragment the pool: the derived width is
|
|
//--- NOT part of BuildModelFingerprint, which keys only on the FEATURE layout.
|
|
bool haveWeights = FileIsExist(m_activeFileName + ".nnw", m_activeFileCommon ? FILE_COMMON : 0);
|
|
bool cfgAdopted = haveWeights &&
|
|
LoadAndCompareTopologyConfiguration(m_activeFileName, m_initialNeuronsCount, m_hiddenLayersCount, m_neuronsReduction, m_minNeuronsCount, m_optimizationAlgo, m_historyBars, m_outputNeuronsCount, m_neuronsCount, m_minTrainYear, m_isInitialized, LEGACY_CONVERGE_WR_SLOT, m_fractalPeriods, m_convFilterCount, m_lstmHiddenSize, m_activeFileCommon);
|
|
if(!haveWeights && FileIsExist(m_activeFileName + ".cfg", m_activeFileCommon ? FILE_COMMON : 0))
|
|
Print(__FUNCTION__ + ": " + ID + " - a .cfg exists but no weights do, so its DERIVED sizes were"
|
|
" re-measured rather than adopted (first layer " + IntegerToString(m_initialNeuronsCount) +
|
|
", depth " + IntegerToString(m_hiddenLayersCount) + "). A .cfg written before the training"
|
|
" pool had any peer files would otherwise pin a training-alone topology forever.");
|
|
if(!cfgAdopted)
|
|
{
|
|
//--- Topology/input params diverged from what produced the saved .nnw (or no .cfg exists yet;
|
|
//--- for inTesterOrOpt this is also the normal "first pass on this agent" case).
|
|
if(FileIsExist(m_activeFileName + ".nnw", m_activeFileCommon ? FILE_COMMON : 0))
|
|
{
|
|
Print(__FUNCTION__ + ": " + m_activeFileName + " - topology/input params changed since last save; discarding incompatible saved weights and starting fresh");
|
|
FileDelete(m_activeFileName + ".nnw", m_activeFileCommon ? FILE_COMMON : 0);
|
|
//--- Reaching here means a TRAINED model was just thrown away, so its drawn signals are
|
|
//--- stale for exactly the same reason ResetWeights() clears them: they would otherwise be
|
|
//--- restored moments later (LoadChartSignals runs at the end of this function) and shown
|
|
//--- as if they belonged to the model about to be trained.
|
|
ClearPersistedChartSignals("saved weights discarded - topology/input params changed");
|
|
}
|
|
if(FileIsExist(m_activeFileName + "_ckpt.tmp", m_activeFileCommon ? FILE_COMMON : 0))
|
|
FileDelete(m_activeFileName + "_ckpt.tmp", m_activeFileCommon ? FILE_COMMON : 0);
|
|
// Same reasoning applies to the EMA shadow-weight file (see m_shadowNet's declaration comment) -
|
|
// it's shaped for the OLD topology too, and EnsureShadowNet() has no independent way to detect
|
|
// that mismatch on Load() (CNet::Load() doesn't cross-validate against an expected shape). Drop
|
|
// it so EnsureShadowNet() cleanly misses and re-bootstraps from the fresh Net instead.
|
|
if(FileIsExist(m_activeFileName + "_shadow.nnw", m_activeFileCommon ? FILE_COMMON : 0))
|
|
FileDelete(m_activeFileName + "_shadow.nnw", m_activeFileCommon ? FILE_COMMON : 0);
|
|
//--- the calibration sidecar is tied to the discarded weights - drop it too so a fresh run
|
|
//--- re-measures priors from scratch instead of adjusting with a stale model's base rates.
|
|
if(FileIsExist(m_activeFileName + ".stats", m_activeFileCommon ? FILE_COMMON : 0))
|
|
FileDelete(m_activeFileName + ".stats", m_activeFileCommon ? FILE_COMMON : 0);
|
|
//--- and the pattern-database backfill marker (see StartPatternDatabaseBackfill): it records
|
|
//--- the era of the model whose OOS calls were written into the ranking tables.
|
|
if(FileIsExist(m_activeFileName + ".dbfill", m_activeFileCommon ? FILE_COMMON : 0))
|
|
FileDelete(m_activeFileName + ".dbfill", m_activeFileCommon ? FILE_COMMON : 0);
|
|
SaveTopologyConfiguration(m_activeFileName, m_initialNeuronsCount, m_hiddenLayersCount, m_neuronsReduction, m_minNeuronsCount, m_optimizationAlgo, m_historyBars, m_outputNeuronsCount, m_neuronsCount, LEGACY_STUDY_PERIOD_SLOT, m_minTrainYear, m_isInitialized, LEGACY_CONVERGE_WR_SLOT, m_fractalPeriods, m_convFilterCount, m_lstmHiddenSize, m_activeFileCommon);
|
|
}
|
|
double loadedIndicatorParams[];
|
|
//--- Inference-only backtest: if this deployed model was validated MQL5-inference-safe at deploy
|
|
//--- (marker in its .stats), load it host-only and run the pure-MQL5 forward path so the backtest
|
|
//--- never loads WarriorDML/WarriorCPU.dll - no DLL file-lock class of failure, and the exact math
|
|
//--- the Market build ships. Falls back to a compute backend just below if that load fails.
|
|
if(m_inferenceOnly && CheckPointer(Net) != POINTER_INVALID)
|
|
{
|
|
LoadModelStats(m_activeFileName, m_activeFileCommon); // reads m_mqlInferenceValidated (and priors)
|
|
if(m_mqlInferenceValidated)
|
|
{
|
|
Net.SetCpuInference(true);
|
|
PrintVerbose(__FUNCTION__ + ": " + ID + " - inference-only backtest running pure-MQL5 (DLL-free): the deployed model is validated MQL5-inference-safe");
|
|
}
|
|
}
|
|
bool netLoaded = LoadNetWithRetry(loadedIndicatorParams);
|
|
//--- Pure-MQL5 load failed unexpectedly (should not happen for a validated model) - drop back to a
|
|
//--- compute backend and retry once so the backtest still runs via the DLL rather than on a fresh net.
|
|
if(!netLoaded && CheckPointer(Net) != POINTER_INVALID && Net.CpuInference())
|
|
{
|
|
Print(__FUNCTION__ + ": " + ID + " - pure-MQL5 load failed; retrying with a compute backend (DLL)");
|
|
Net.SetCpuInference(false);
|
|
netLoaded = LoadNetWithRetry(loadedIndicatorParams);
|
|
}
|
|
//--- the file may carry a superseded architecture - correct it before anything reads the net
|
|
if(netLoaded)
|
|
EnforceTopologyContract();
|
|
//--- A superseded conv receptive field cannot be repaired in place (different weight-tensor shape),
|
|
//--- so the loaded net is discarded and the fresh-topology path below rebuilds and retrains.
|
|
if(netLoaded && m_topologySuperseded)
|
|
netLoaded = false;
|
|
//--- restore the calibration sidecar (priors + confidence scale) that pairs with these weights, so a
|
|
//--- restart - including a buyer's inference-only backtest - calibrates live decisions exactly as the
|
|
//--- saved model did instead of running with cold defaults (priors 0 => no adjustment). See LoadModelStats().
|
|
if(netLoaded)
|
|
{
|
|
LoadModelStats(m_activeFileName, m_activeFileCommon);
|
|
//--- ...and rebuild the ensemble headline from whatever record that just restored. Without this
|
|
//--- a reloaded DEPLOYED chart had no aggregate line at all: the only other caller runs at
|
|
//--- pass-3 completion, and a deployed ensemble runs no further eras to reach it - so the panel
|
|
//--- fell back to one row per member, which is exactly the readout the operator asked to be rid
|
|
//--- of. No "this era" figure is passed: there has not been one this session, and showing the
|
|
//--- stored era's number here would read as live.
|
|
if(m_ensembleMember)
|
|
PublishEnsembleAccuracyLine(-1.0, 0);
|
|
//--- CONVERGED BUT MUTE. Say so, loudly and once, because every downstream symptom of it looks
|
|
//--- like something else: no arrows reads as a drawing fault, "0 vote/4 flat" reads as models
|
|
//--- that disagree, and "measuring..." reads as a panel that has not caught up. All three are
|
|
//--- the same thing - a model with no measured tier ladder returns 0 from LiveVoteContribution
|
|
//--- by design, so it cannot vote, cannot be counted in the reconstruction divisor, and cannot
|
|
//--- contribute to the aggregate win rate. A .stats written before WST7 has no ladder in it.
|
|
if(m_trainingComplete && !m_tiersSelfRanked)
|
|
Print(ID + ": WARNING - resumed CONVERGED but with NO MEASURED TIER LADDER, so this model"
|
|
" CANNOT VOTE and will place no trades. The tier weights are produced only by a"
|
|
" completed scoring pass and were not stored by the build that trained this model"
|
|
" (.stats predates WST7). It will mint and store them at the end of its next scoring"
|
|
" pass, after which restarts keep them. Until then this member is silent - that is the"
|
|
" cause of an empty chart, a '0 vote' readout and a 'measuring...' win rate, all three.");
|
|
}
|
|
m_modelLoadedFromDisk = netLoaded;
|
|
//--- Make a successful resume visible (the counterpart to the fresh-start / mismatch messages below):
|
|
//--- on a live chart this confirms the saved model was found and loaded rather than silently retrained.
|
|
if(netLoaded && !inTesterOrOpt)
|
|
Print(ID + ": resumed saved model from era " + IntegerToString(m_eraCount) + " (trainingComplete=" + (string)m_trainingComplete + ") - continuing, not retraining from era 0.");
|
|
//--- RESUMED MODELS GET THE SAME WARM-UP AS FRESH ONES (2026-08-13; was `netLoaded ? 0 : 3`).
|
|
//--- Three no-op passes cost seconds. The label cache itself, however, is NEVER restored from
|
|
//--- the .nnw checkpoint - it lives only in the in-memory m_labelCacheBuy/Sell/HasValue arrays,
|
|
//--- which start empty every process start regardless of netLoaded.
|
|
m_warmupPassesRemaining = 3;
|
|
m_labelCachePrebuilt = false;
|
|
if(inTesterOrOpt && netLoaded)
|
|
Print(__FUNCTION__ + ": " + ID + " - reused cached weights from a previous optimization/tester pass on this agent (era " + IntegerToString(m_eraCount) + ", trainingComplete=" + (string)m_trainingComplete + ") - skipping redundant training for this unchanged config");
|
|
if(netLoaded && ArraySize(loadedIndicatorParams) == AD_TUNE_PARAM_COUNT)
|
|
{
|
|
//--- Restart deploying previously AutoTune-d indicator params even with
|
|
//--- AutoTuneIndicators=false now.
|
|
AdoptIndicatorParams(loadedIndicatorParams, indicators);
|
|
}
|
|
if(!netLoaded)
|
|
{
|
|
int error_code = GetLastError();
|
|
//--- Do NOT present error_code as the cause: on a no-GPU/CPU-DLL box it is the harmless 5100
|
|
//--- (OpenCL-not-found) left by the compute probe inside CNet::Load, NOT the reason the file
|
|
//--- was rejected.
|
|
if(error_code != 5004) // not "file not found"
|
|
ResetLastError();
|
|
//--- CRITICAL: a failed load may have ALREADY overwritten the training-state out-params from
|
|
//--- the bad file's header before it was rejected - notably a corrupt/empty 0-layer stub
|
|
//--- whose header still says trainingComplete=1 (see CNet::Load's 0-layer guard).
|
|
m_trainingComplete = false;
|
|
m_eraCount = 0;
|
|
dtStudied = 0;
|
|
dForecast = 0;
|
|
//--- Cold the in-memory calibration so the freshly-rebuilt (untrained) topology below runs
|
|
//--- with no stale prior-correction until a retrain re-measures it (priors 0 =>
|
|
//--- AdjustedSignalFromSoftmax is a no-op; scale 1.0 = the constructor default).
|
|
m_priorBuy = 0.0;
|
|
m_priorSell = 0.0;
|
|
m_priorNeutral = 0.0;
|
|
m_confidenceCalScale = 1.0;
|
|
//--- Accurate diagnostic (do NOT cite GetLastError() - inside CNet::Load the OpenCL probe leaves 5100
|
|
//--- there on a no-GPU/CPU-DLL box, which has nothing to do with the file). Distinguish an ordinary
|
|
//--- fresh start (no file yet) from a real read failure of an existing file by testing existence.
|
|
if(!inTesterOrOpt)
|
|
{
|
|
int loadFlags = m_activeFileCommon ? FILE_COMMON : 0;
|
|
if(FileIsExist(m_activeFileName + ".nnw", loadFlags))
|
|
Print(ID + ": could not read the existing model file " + m_activeFileName + ".nnw - rebuilding a fresh topology to retrain from era 0. Existing .stats/_shadow.nnw are KEPT (they refresh as training runs). If this recurs, that .nnw is likely corrupt - back it up, then use the panel's reset-weights to start clean.");
|
|
else
|
|
Print(ID + ": no saved model for this config yet - starting a fresh training run from era 0.");
|
|
}
|
|
//--- Re-seed before building a fresh topology so weight init is genuinely random. See
|
|
//--- System\Random.mqh. Matches ResetWeights() and OnInit.
|
|
WarriorRandSeed(ID);
|
|
//--- Era 0 with no weights behind it, so any arrow currently on this chart was drawn by a
|
|
//--- DIFFERENT model - the previous fingerprint's, or a corrupt .nnw's. Deliberately at
|
|
//--- this call site rather than inside BuildFreshTopology(): the genetic tuner calls that
|
|
//--- for every throwaway candidate (AutoTune.mqh) and must not touch the chart.
|
|
ClearPersistedChartSignals("fresh topology at era 0 - arrows belong to a previous model");
|
|
if(!BuildFreshTopology())
|
|
return false;
|
|
}
|
|
TempData = new CArrayDouble();
|
|
if(CheckPointer(TempData) == POINTER_INVALID)
|
|
return false;
|
|
if(netLoaded)
|
|
// Populate dPrevSignal from the just-loaded weights immediately, rather than leaving it at
|
|
// its blank constructor default until the next (asynchronous, queued) training pass happens
|
|
// to run - matters most for the tester cache-reuse path above, where training may be skipped
|
|
// entirely for this run because dtStudied already covers the whole backtest window.
|
|
RefreshLatestSignal();
|
|
//--- Status line must match what the gate below (if(!m_trainingComplete && !m_inferenceOnly)) will
|
|
//--- actually do - otherwise an inference-only single backtest logs "resuming full training now" right
|
|
//--- under the "runs inference only and will NOT train" warning, which reads as a contradiction.
|
|
string trainState = m_trainingComplete
|
|
? "already complete - staying converged, no full retrain on this restart"
|
|
: (m_inferenceOnly
|
|
? "NOT complete, but this is an inference-only backtest - NOT training (see warning above); deploy a trained model for meaningful results"
|
|
: "NOT complete (interrupted or never converged) - resuming full training now");
|
|
Print(__FUNCTION__ + ": " + m_activeFileName + " - training " + trainState);
|
|
//--- Only kick off a full Train() run here if the loaded model genuinely isn't converged yet - an
|
|
//--- already-complete model used to get one full era-loop retrain (real Net.backProp() over the
|
|
//--- whole IS window) on every single EA restart/reattach for no reason, since this "Init" event
|
|
//--- bypassed ScheduleTrainingIfNeeded()'s m_trainingComplete gate entirely. dPrevSignal is already
|
|
//--- fresh from RefreshLatestSignal() above; ScheduleTrainingIfNeeded()'s normal per-tick check
|
|
//--- will call RefreshConvergedSignal() itself once a genuinely new bar closes.
|
|
if(!m_trainingComplete && !m_inferenceOnly)
|
|
ArmStudyEvent((long)MathMax(0, MathMin(iTime(_Symbol, PERIOD_CURRENT, (int)(100 * Net.recentAverageSmoothingFactor * (m_trainingComplete ? 1 : 10))), dtStudied)), "Init");
|
|
//--- Restore arrows persisted from a previous session (see SaveChartSignals). MUST run here, not in
|
|
//--- InitIndicators(): the arrows file is keyed on the FULL m_fileName including the per-config
|
|
//--- fingerprint, which is only appended above - see the note left at InitIndicators()'s old call site.
|
|
LoadChartSignals();
|
|
//--- bootstrap (or restore) the EMA shadow net now rather than waiting for the first
|
|
//--- RefreshLatestSignal()/era-blend call to lazily trigger it - see m_shadowNet's declaration
|
|
//--- comment.
|
|
EnsureShadowNet();
|
|
m_isInitialized = true;
|
|
return true;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Creates the OHLC + ZigZag indicators the feature builder reads. |
|
|
//| Called by InitNeuralNetwork(), not by the framework - the public |
|
|
//| InitIndicators() override is the framework entry point. |
|
|
//+------------------------------------------------------------------+
|
|
bool CExpertSignalAIBase::InitFeatureIndicators(CIndicators *indicators)
|
|
{
|
|
//--- Reset only the status label on (re-)init; deliberately do NOT PurgeChart() here so previously drawn
|
|
//--- signal arrows survive an EA re-init (recompile / param change / timeframe switch) instead of
|
|
//--- vanishing every time - see SIG_ARROW_PREFIX. Full cleanup still happens in the destructor.
|
|
ClearStatusLabel();
|
|
//--- NOTE: LoadChartSignals() is deliberately NOT called here any more. The mismatch made the
|
|
//--- restore silently no-op on every restart from the moment the fingerprint was introduced. Same
|
|
//--- family as the fingerprint trap documented at BuildConfigFingerprint: anything keyed on
|
|
//--- m_fileName must run AFTER it is fully built.
|
|
if(!InitOpen(indicators))
|
|
return false;
|
|
if(!InitClose(indicators))
|
|
return false;
|
|
if(!InitLow(indicators))
|
|
return false;
|
|
if(!InitHigh(indicators))
|
|
return false;
|
|
//--- label source, always created unconditionally, same as the OHLC indicators above - see
|
|
//--- m_zigZag's declaration comment. Optionally ALSO read as an input feature (m_useSwingContext,
|
|
//--- below) using the same already-running indicator instance - no separate init needed for that.
|
|
if(!InitZigZag(indicators))
|
|
return false;
|
|
m_neuronsCount = 4; // (close-open)/atr, (high-open)/atr, (low-open)/atr, bullish/bearish flag
|
|
if(m_useVolumes)
|
|
{
|
|
// change ratio, level vs 50-bar baseline, absorption (range per unit volume), volume x range -
|
|
// see BufferTempDataCompute()'s matching block, and research/test_volume.py for the measurement
|
|
// that justified widening this from 1. m_neuronsCount is already in the config fingerprint, so
|
|
// this re-keys existing caches on its own: correct, the input vector genuinely changed shape.
|
|
m_neuronsCount += 4;
|
|
if(!InitVolumes(indicators))
|
|
return false;
|
|
}
|
|
// Unconditional, same reasoning as m_ATR/m_zigZag below: m_Time.GetData() is read
|
|
// unconditionally elsewhere (label-eligibility gate, cache anchor, online-learning watermark,
|
|
// arrow timestamps) regardless of whether the cyclical time-of-day/day-of-week values are also
|
|
// opted into as an explicit feature via m_useTime - so the indicator itself must always exist.
|
|
if(!InitTime(indicators))
|
|
return false;
|
|
if(m_useTime)
|
|
{
|
|
m_neuronsCount += 6;
|
|
}
|
|
if(m_useATR)
|
|
{
|
|
//already init in the base class
|
|
m_neuronsCount++;
|
|
}
|
|
if(m_useMA)
|
|
{
|
|
if(!InitMA(indicators))
|
|
return false;
|
|
m_neuronsCount += 5; // (open-MA)/atr, (high-MA)/atr, (low-MA)/atr, (close-MA)/atr, (MA-MA[1])/atr
|
|
}
|
|
if(m_useSwingContext)
|
|
m_neuronsCount += 9; // 5 confirmed-pivot features (direction, distance-since-pivot, prior-leg magnitude, retracement ratio, bars-since-pivot) + 4 recent-context features (Donchian pos 20/50, 20-bar return, 20-bar SMA extension) - see BufferTempDataCompute()'s matching block
|
|
if(m_useNews)
|
|
m_neuronsCount += 2; // NewsRecency, NewsProximity - see BufferTempDataCompute()'s matching block
|
|
if(m_useSpreadFeature)
|
|
m_neuronsCount += 2; // spread/ATR (volatility-regime reading), spread change ratio
|
|
// - see BufferTempDataCompute()'s matching block
|
|
if(m_useCrossAsset)
|
|
m_neuronsCount += CROSSASSET_FEATURES; // FX: base/quote strength + divergence; index: denom/risk-proxy strength
|
|
//--- ALT DATA (2026-08-16). Externally collected, publication-stamped features (COT positioning,
|
|
//--- VIX complex, macro) exported by research/altdata/export.py into
|
|
//--- Common\Files\Warrior_EA\AltData\{SYMBOL}_{TF}.csv - see System\AltData.mqh for the
|
|
//--- lookahead/degradation contracts.
|
|
if(m_altDataEnabled)
|
|
{
|
|
//--- A MODEL'S OWN .cfg STILL WINS. Adopt-don't-compare: an existing model keeps the column set
|
|
//--- its weights were trained against, exactly as it keeps its topology. Only a FRESH model
|
|
//--- takes the fleet set - which is what makes this change retrain-forcing rather than
|
|
//--- silently re-keying a trained model's inputs.
|
|
string altPin = ReadAltDataPinFromCfg();
|
|
bool fleetPin = false;
|
|
if(altPin == "")
|
|
{
|
|
//--- FRESH MODEL: pin the FLEET set, not this symbol's file header. Letting the file decide
|
|
//--- is what split the fleet into three incompatible training pools and orphaned SP500 -
|
|
//--- see ALTDATA_FLEET_COLUMNS for the full reasoning and the cost.
|
|
altPin = ALTDATA_FLEET_COLUMNS;
|
|
fleetPin = true;
|
|
}
|
|
m_altData.SetPinnedNames(altPin);
|
|
m_altData.Load(m_symbol.Name(), (ENUM_TIMEFRAMES)m_period); // logs its own outcome; absence is normal
|
|
m_altDataNamesPinned = altPin; // stamped into the .cfg on the first save
|
|
if(fleetPin)
|
|
Print(ID + ": alt-data pinned to the FLEET column set (" +
|
|
IntegerToString(m_altData.FeatureCount()) + " columns) rather than this symbol's file"
|
|
" header. Every chart therefore publishes the same feature layout and can pool with"
|
|
" every other; a per-symbol set is what left SP500 training alone.");
|
|
m_useAltData = (m_altData.FeatureCount() > 0);
|
|
if(m_useAltData)
|
|
m_neuronsCount += m_altData.FeatureCount();
|
|
}
|
|
else
|
|
{
|
|
//--- Operator opt-out (EnableAltData=false): zero width, nothing pinned. On a model trained
|
|
//--- WITH alt features this shrinks neuronsCount, mismatches the .cfg compare and correctly
|
|
//--- starts fresh - stated in the input's comment rather than silently absorbed.
|
|
m_useAltData = false;
|
|
m_altDataNamesPinned = "";
|
|
}
|
|
if(!FolderCreate(m_folderPath, FILE_COMMON))
|
|
{
|
|
if(GetLastError() != 5010) // If the error is not because the folder already exists
|
|
{
|
|
Print("Failed to create folder: " + m_folderPath);
|
|
}
|
|
else
|
|
{
|
|
ResetLastError(); // Reset the error code
|
|
}
|
|
}
|
|
return true;
|
|
}
|
|
#endif
|