forked from animatedread/Warrior_EA
994fe3899c
The old target asked "which way is the next pivot", which every bar of a
~13-20 bar leg answers identically - so the net could not tell a fresh turn
from mid-trend and learned the prevailing direction instead. Its own
zero-skill reference showed it: chance sat at 56/44, i.e. the label WAS the
drift, and the gate's standing warning ("a model that only reproduces it has
found the drift, not an edge") applied to the target itself.
Buy now means a swing LOW commits within PIVOT_LABEL_TOLERANCE_BARS bars,
Sell a swing HIGH, Neutral no turn that close. Pivot type is read from
ZigZagBuffer[p] == Low[p], exact by construction in ZigZag.mq5. The existing
P1-final-once-P2-commits rule is kept and now also settles the NEGATIVE
verdict, so the Neutral majority is permanent rather than provisional.
Measured on a full fresh run, all 6 charts:
class balance 56/44/~0 -> 13.7/13.7/72.6 (imbalance 5.3:1)
label overlap ~31 bars -> 5 bars
independent obs 368-1086 -> 2331-7032
weights/obs 9.2-26.2 -> 1.1-4.2
coverage 100% of bars -> 17-48%
23 of 24 models fire all three classes at precision 18-32% vs 13-15%
chance; SP500's ensemble reaches DEPLOYABLE (32.3% vs a 24.0% bar).
Two bindings had to move with the label:
- The capacity deflator. m_swingLifespan fed EstimatedInSampleBars() as
raw/31, measured from the legs. Overlap is now a property of the LABEL -
one turn is callable by exactly the tolerance window - so it is the
window, not a leg measurement. Missing this would have kept every model
sized for a sixth of its real evidence.
- A dormant cold-start seed. Labels.mqh seeds the output bias toward the
dominant class above COLD_START_SEED_MIN_DOMINANCE (0.70); at 56/44 it
never armed, at 72.6% Neutral it does - writing a fixed +-3.0 against a
true prior spread of ~1.75, which would start every net predicting Neutral
~95% of the time. Now seeds the measured log-prior, zero-centred and
capped by the same guard rail the logit adjustment uses (Lin et al. 2017).
TGT:SWG1 -> TGT:PVT1:<tolerance>, with the window in the token because it is
part of the label: every .nnw is invalidated and the fleet retrains.
Depth is still gated, and now for a precise reason: the first dense layer
stays at FIRST_LAYER_MIN_WIDTH because budget = effN/(inputWidth+1) is 11.2
at input 624. Reaching the next rung needs inputWidth <= ~218, i.e. feature
pruning - not architecture.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1219 lines
62 KiB
MQL5
1219 lines
62 KiB
MQL5
//+------------------------------------------------------------------+
|
|
//| Warrior_EA |
|
|
//| AnimateDread |
|
|
//| |
|
|
//| Indicator lifecycle for the feature-only handles, and the per-bar |
|
|
//| input feature vector. STATEFUL, unlike CModelPersistence/ |
|
|
//| CTopology: owns m_Volumes and m_MA directly (the RSI, MACD, |
|
|
//| Ichimoku and five AD/Wyckoff handles it also owned were removed |
|
|
//| 2026-08-24 with their feature groups), plus the depth-probe/ |
|
|
//| handle-repair/spread-series state. m_Open/m_Close/ |
|
|
//| m_High/m_Low/m_Time/m_ATR/m_zigZag stay signal-owned (genuinely |
|
|
//| shared with Labels.mqh/AutoTune.mqh/Training.mqh) and are reached |
|
|
//| read-only through the view. InitOpen/InitClose/InitHigh/InitLow/ |
|
|
//| InitTime/InitZigZag/ResizeBuffers/RefreshData are NOT here: they|
|
|
//| manage those shared indicators' Create/BufferResize/Refresh |
|
|
//| lifecycle, which would need a pure-relay wrapper per operation per|
|
|
//| indicator for no coupling benefit - same judgment as Topology's |
|
|
//| boot sequence. They stay in Expert\AIBase\Features.mqh, calling |
|
|
//| back into the Feature*BufferResize()/Feature*Refresh() forwards |
|
|
//| this class exposes for its OWN owned indicators. |
|
|
//+------------------------------------------------------------------+
|
|
#ifndef WARRIOR_FEATURES_FEATUREBUILDER_MQH
|
|
#define WARRIOR_FEATURES_FEATUREBUILDER_MQH
|
|
//--- Plausibility ceiling for any single input value, enforced once over the whole bar at the end
|
|
//--- of BufferTempDataCompute(). See the sanitize loop at the end of BufferTempDataCompute() for
|
|
//--- what it protects.
|
|
#define FEATURE_ABS_MAX 1.0e4
|
|
class CFeatureBuilder
|
|
{
|
|
private:
|
|
CFeaturesView *m_view; // BORROWED - the signal owns the adapter, not the reverse
|
|
//--- Optional classic-indicator input FEATURES, independent of the CSignal* instances used for
|
|
//--- voting. Periods come from the tuner. Built-in Ci* wrappers throughout, so none of them
|
|
//--- carries a CiCustom depth limit.
|
|
CiVolumes m_Volumes;
|
|
CiMA m_MA;
|
|
//--- Last depth ServableBars() had to clamp to. 0 = never clamped.
|
|
int m_indicatorDepthCapBars;
|
|
//--- One-shot latch for "an enabled tunable indicator reports NO calculated bars".
|
|
bool m_indicatorDepthDeadWarned;
|
|
//--- Cooldown between attempts to rebuild a dead handle.
|
|
uint m_handleRepairTick;
|
|
//--- SettledBars() probe state. m_depthSettleStart doubles as the "a wait is in progress" flag.
|
|
uint m_depthSettleStart;
|
|
uint m_depthProbeTick;
|
|
int m_depthProbeLast;
|
|
int m_depthProbeStable;
|
|
//--- ONE-SHOT DETECTABILITY REPORT latch.
|
|
bool m_detectabilityReported;
|
|
//--- Set by BufferTempDataCompute when it rejected a bar because the data had not ARRIVED yet
|
|
//--- (price buffer EMPTY_VALUE, or an ATR the terminal has not finished calculating) as opposed
|
|
//--- to the bar being genuinely unusable.
|
|
bool m_featureFailTransient;
|
|
bool m_featureWidthWarned; // one-shot: the width contract is a structural fault
|
|
bool m_featureHealthReported;
|
|
//--- Cross-asset panel cache key.
|
|
datetime m_crossAssetAnchor;
|
|
//--- Historical spread series copied onto the current bar grid.
|
|
int m_spreadSeries[];
|
|
int m_spreadSeriesBars;
|
|
datetime m_spreadSeriesAnchor;
|
|
public:
|
|
CFeatureBuilder(void) :
|
|
m_view(NULL),
|
|
m_indicatorDepthCapBars(0),
|
|
m_indicatorDepthDeadWarned(false),
|
|
m_handleRepairTick(0),
|
|
m_depthSettleStart(0),
|
|
m_depthProbeTick(0),
|
|
m_depthProbeLast(0),
|
|
m_depthProbeStable(0),
|
|
m_detectabilityReported(false),
|
|
m_featureFailTransient(false),
|
|
m_featureWidthWarned(false),
|
|
m_featureHealthReported(false),
|
|
m_crossAssetAnchor(0),
|
|
m_spreadSeriesBars(0),
|
|
m_spreadSeriesAnchor(0)
|
|
{ }
|
|
void Bind(CFeaturesView *view) { m_view = view; }
|
|
|
|
int TunableBarsCalculated(int &enabled);
|
|
int TunableBarsCalculated(void);
|
|
int ServableBars(int want, string context);
|
|
int SettledBars(int want, string context);
|
|
void ReportDetectability(int oosBars);
|
|
bool RepairDeadIndicatorHandles(void);
|
|
int NoteHandleMove(const string name, const int oldHandle, const int newHandle, string &moves);
|
|
string IndicatorDepthField(const string name, const int depth, const int handle);
|
|
string FeatureSlotName(const int slot);
|
|
string IndicatorDepthReport(void);
|
|
bool AdoptIndicatorParams(const double &loaded[], CIndicators *indicators);
|
|
bool ReInitTunableIndicators(CIndicators *indicators);
|
|
bool BufferTempData(int idx);
|
|
void ReportFeatureHealth(int bars);
|
|
bool BuildFeatureWindow(int r);
|
|
bool BuildCrossAssetPanel(int bars);
|
|
bool EnsureSpreadSeries(int bars);
|
|
bool BufferTempDataCompute(int idx);
|
|
bool ADIndicatorCold(CiCustom &ind, string block);
|
|
bool InitVolumes(CIndicators *indicators);
|
|
bool InitMA(CIndicators *indicators, bool addToCollection = true);
|
|
//--- Called by the signal's own (still-signal-owned) ResizeBuffers()/RefreshData() - see this
|
|
//--- file's class comment for why those two stay behind.
|
|
bool VolumesBufferResize(const int n) { return m_Volumes.BufferResize(n); }
|
|
void VolumesRefresh(void) { m_Volumes.Refresh(OBJ_ALL_PERIODS); }
|
|
bool MaBufferResize(const int n) { return m_MA.BufferResize(n); }
|
|
void MaRefresh(void) { m_MA.Refresh(OBJ_ALL_PERIODS); }
|
|
};
|
|
//+------------------------------------------------------------------+
|
|
//| Rebuilds only the enabled AD* CiCustom handles in place, so a |
|
|
//| new trial's member-struct param values take effect. |
|
|
//+------------------------------------------------------------------+
|
|
int CFeatureBuilder::TunableBarsCalculated(int &enabled)
|
|
{
|
|
enabled = 0;
|
|
int worst = INT_MAX;
|
|
if(m_view.UseMA())
|
|
{
|
|
enabled++;
|
|
worst = (int)MathMin(worst, m_MA.BarsCalculated());
|
|
}
|
|
return (worst == INT_MAX) ? -1 : worst;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Back-compatible form for the callers that only want the number. |
|
|
//+------------------------------------------------------------------+
|
|
int CFeatureBuilder::TunableBarsCalculated(void)
|
|
{
|
|
int enabled = 0;
|
|
return TunableBarsCalculated(enabled);
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| See the declaration. THE one place that decides how much history |
|
|
//| may be asked of the indicators; every ResizeBuffers() call site |
|
|
//| goes through it. |
|
|
//+------------------------------------------------------------------+
|
|
int CFeatureBuilder::ServableBars(int want, string context)
|
|
{
|
|
if(want <= 0)
|
|
return want;
|
|
int enabled = 0;
|
|
int servable = TunableBarsCalculated(enabled);
|
|
//--- THE BLIND SPOT THAT COST 2026-08-17 (fixed the same day, after the fact). Fine. enabled >
|
|
//--- 0, servable == -1 -> a handle answered INVALID. The dead case is now REPORTED and REPAIRED.
|
|
if(enabled == 0 || servable >= want)
|
|
{
|
|
//--- Cleared on the healthy path too, not only on the clamp path below: a handle that recovers
|
|
//--- all the way to full depth would otherwise leave the latch set and a LATER outage would be
|
|
//--- swallowed - which is the failure mode this whole function is being fixed for.
|
|
m_indicatorDepthDeadWarned = false;
|
|
return want;
|
|
}
|
|
if(servable <= 0)
|
|
{
|
|
//--- REPORT FIRST, THEN REPAIR - in that order, so the depths in this line are the ones that
|
|
//--- caused it.
|
|
if(!m_indicatorDepthDeadWarned)
|
|
{
|
|
m_indicatorDepthDeadWarned = true;
|
|
PrintFormat("%s: TUNABLE INDICATOR REPORTS NO CALCULATED BARS - %d tunable indicator(s) enabled"
|
|
" and the least-ready answers BarsCalculated()=%d while %s asked for %d. Either way"
|
|
" CopyBuffer fails at EVERY index, the buffer holds nothing, and every feature block"
|
|
" that reads it rejects every bar. -1 means the terminal would not answer for this"
|
|
" handle, which covers BOTH a handle freed out from under this member AND one just"
|
|
" created that has not calculated yet - the handle numbers below separate them, the"
|
|
" depth cannot. This is NOT the depth cap below (that one clamps and trains on what"
|
|
" is servable) - there is nothing to clamp to. Per-indicator depth:%s",
|
|
m_view.Id(), enabled, servable, context, want, IndicatorDepthReport());
|
|
}
|
|
//--- A dead handle answers EMPTY_VALUE at every index, so a 50k-bar pass over it is 50k
|
|
//--- guaranteed rejections followed by a discarded era, forever - the exact loop that froze
|
|
//--- USDJPY and XAUUSD. Rate-limited inside.
|
|
RepairDeadIndicatorHandles();
|
|
return want;
|
|
}
|
|
m_indicatorDepthDeadWarned = false;
|
|
//--- -(m_historyBars + 2): the deepest window slot reads (r + m_historyBars - 1), and the MA
|
|
//--- block one further back again for its bar-over-bar change, so the last usable anchor sits
|
|
//--- that far inside the buffer.
|
|
int capped = servable - (m_view.HistoryBars() + 2);
|
|
if(capped < 0)
|
|
capped = 0;
|
|
//--- Depends only on `servable` and m_historyBars, never on `want`, so it is stable across call
|
|
//--- sites and this logs once per real change rather than once per era per context.
|
|
if(m_indicatorDepthCapBars != capped)
|
|
{
|
|
m_indicatorDepthCapBars = capped;
|
|
PrintFormat("%s: indicator history CAPPED to %d bars (%s asked for %d) - the price series has"
|
|
" that much, but the least-ready tunable indicator has only calculated %d. Past what"
|
|
" an indicator has calculated CopyBuffer does not short-read, it FAILS, so the buffer"
|
|
" holds NOTHING and EVERY index reads EMPTY_VALUE - indistinguishable from a cold"
|
|
" indicator. Per-indicator depth:%s",
|
|
m_view.Id(), capped, context, want, servable, IndicatorDepthReport());
|
|
}
|
|
return capped;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| See the declaration. ServableBars() with the WAIT in front of it. |
|
|
//+------------------------------------------------------------------+
|
|
int CFeatureBuilder::SettledBars(int want, string context)
|
|
{
|
|
if(want <= 0)
|
|
return want;
|
|
int enabled = 0;
|
|
int servable = TunableBarsCalculated(enabled);
|
|
//--- Same three-states-one-branch defect ServableBars() carried (see the long note there):
|
|
//--- `servable < 0` was read as "nothing tunable is on", but it is ALSO what a dead handle
|
|
//--- answers.
|
|
if(enabled == 0 || servable >= want)
|
|
{
|
|
m_depthSettleStart = 0;
|
|
m_depthProbeStable = 0;
|
|
m_depthProbeLast = 0;
|
|
return want;
|
|
}
|
|
if(servable <= 0)
|
|
{
|
|
m_depthSettleStart = 0;
|
|
m_depthProbeStable = 0;
|
|
m_depthProbeLast = 0;
|
|
//--- Routed through ServableBars() rather than answering here, and that detour is the whole
|
|
//--- point: the training sweep - the ONLY caller that reaches the dead-handle state in
|
|
//--- practice - calls SettledBars, not ServableBars.
|
|
ServableBars(want, context);
|
|
//--- 0 = HOLD, and this is the one place the two functions deliberately disagree. Whether the
|
|
//--- handle was just recreated (cold, will climb) or is still dead (repair failed), holding
|
|
//--- is right; Train() reports the hold every minute and the era-barrier liveness escape
|
|
//--- releases the rest of the ensemble if it never resolves.
|
|
return 0;
|
|
}
|
|
uint now = GetTickCount();
|
|
//--- First shortfall: start the clock and let the priming request above do its work. Deliberately
|
|
//--- no sweep this call - a 50k-bar feature sweep is exactly what starves the indicator threads we
|
|
//--- are waiting on, which is how the old loop sustained itself for 40 minutes at a time.
|
|
if(m_depthSettleStart == 0)
|
|
{
|
|
m_depthSettleStart = now;
|
|
m_depthProbeTick = now;
|
|
m_depthProbeLast = servable;
|
|
m_depthProbeStable = 0;
|
|
PrintFormat("%s: PRIMING indicator history for the %s - %d of %d bars calculated so far. Holding"
|
|
" the sweep until the count stops rising (probe every %ds, needs %d steady probes,"
|
|
" gives up after %ds and uses whatever is there). Per-indicator depth:%s",
|
|
m_view.Id(), context, servable, want, DEPTH_SETTLE_PROBE_MS / 1000,
|
|
DEPTH_SETTLE_STABLE_PROBES, DEPTH_SETTLE_TIMEOUT_MS / 1000, IndicatorDepthReport());
|
|
return 0;
|
|
}
|
|
//--- Unsigned subtraction, so this is correct across GetTickCount()'s 49-day wrap (same idiom as
|
|
//--- m_coldSweepTick's backoff).
|
|
if(now - m_depthProbeTick < DEPTH_SETTLE_PROBE_MS)
|
|
return 0;
|
|
m_depthProbeTick = now;
|
|
if(servable != m_depthProbeLast)
|
|
{
|
|
//--- STILL MOVING. Growing is the terminal working through the history; shrinking happens when a
|
|
//--- handle is rebuilt under us and starts over. Either way it is not settled, so the streak
|
|
//--- restarts rather than counting a change as a steady observation.
|
|
PrintFormat("%s: priming %s - %d of %d bars (was %d), still moving", m_view.Id(), context, servable, want,
|
|
m_depthProbeLast);
|
|
m_depthProbeLast = servable;
|
|
m_depthProbeStable = 0;
|
|
return 0;
|
|
}
|
|
m_depthProbeStable++;
|
|
bool steady = (m_depthProbeStable >= DEPTH_SETTLE_STABLE_PROBES);
|
|
bool expired = ((now - m_depthSettleStart) >= DEPTH_SETTLE_TIMEOUT_MS);
|
|
if(!steady && !expired)
|
|
return 0;
|
|
//--- Settled (or waited long enough) BELOW what was asked. This is the real depth, not a snapshot of
|
|
//--- a value still climbing, so it is now safe to clamp to it and get on with training.
|
|
PrintFormat("%s: priming %s DONE - depth settled at %d of %d bars after %ds%s. Training proceeds on"
|
|
" the %d bars the indicators can actually serve.",
|
|
m_view.Id(), context, servable, want, (int)((now - m_depthSettleStart) / 1000),
|
|
expired && !steady ? " (gave up waiting - it never went steady)" : "", servable);
|
|
m_depthSettleStart = 0;
|
|
m_depthProbeStable = 0;
|
|
m_depthProbeLast = 0;
|
|
return ServableBars(want, context);
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| See the declaration. What this configuration would have to FIRE |
|
|
//| before any edge of a given size becomes certifiable. |
|
|
//+------------------------------------------------------------------+
|
|
void CFeatureBuilder::ReportDetectability(int oosBars)
|
|
{
|
|
if(m_detectabilityReported || oosBars <= 0)
|
|
return;
|
|
m_detectabilityReported = true;
|
|
double L = m_view.MeanLabelLifespan();
|
|
//--- CAPACITY, restated against the sample that actually exists. That leaves the first build's
|
|
//--- warning quoting the optimistic figure, so it is restated HERE, at the first moment L is
|
|
//--- real.
|
|
int fanIn = m_view.FirstLayerFanIn();
|
|
double firstLayerW = (double)(fanIn + 1) * (double)m_view.InitialNeuronsCount();
|
|
//--- The MEASURED deflation applied to the raw window, so this line's own arithmetic is the one it
|
|
//--- prints. The topology was sized against an ESTIMATE of L taken from the ZigZag legs at init
|
|
//--- (the label cache does not exist that early - see CTopology::MeasureSwingGeometry); printing
|
|
//--- both is what makes the sizing verifiable rather than asserted.
|
|
double indepRows = m_view.EffectiveSampleSize(m_view.EstimatedInSampleBarsRaw());
|
|
double sizedFor = m_view.SwingLifespanEstimate();
|
|
if(fanIn > 0 && indepRows > 0.0)
|
|
PrintFormat("%s: CAPACITY against the same sample the gate uses - first dense layer is %d x %d"
|
|
" = %.0f weights against ~%.0f independent in-sample observations (%.0f rows / label"
|
|
" overlap %.1f, sized for %.1f) = %.1f weights per observation. One per"
|
|
" observation is already generous for a signal this weak. The two multipliers are the"
|
|
" input window and the feature count (%d bars x %d readings); pooling instruments is"
|
|
" the third lever and the only one that ADDS observations instead of removing"
|
|
" parameters.%s",
|
|
m_view.Id(), fanIn + 1, m_view.InitialNeuronsCount(), firstLayerW, indepRows,
|
|
m_view.EstimatedInSampleBarsRaw(), L, sizedFor, firstLayerW / indepRows,
|
|
m_view.HistoryBars(), m_view.NeuronsCount(),
|
|
(L > 1.0001 && (L > 2.0 * sizedFor || sizedFor > 2.0 * L))
|
|
? " WARNING - the measured lifespan and the one the topology was sized for differ by"
|
|
" more than 2x, which is a full rung of the width ladder: the estimate taken from"
|
|
" the ZigZag legs does not describe this label. Reset the weights from the panel to"
|
|
" re-derive, and if it still diverges the estimator is wrong, not the labels."
|
|
: "");
|
|
double p = m_view.ChanceRatePct() / 100.0;
|
|
if(p <= 0.0 || p >= 1.0)
|
|
return;
|
|
//--- Invert the deploy gate. Everything on the right-hand side is a property of the
|
|
//--- CONFIGURATION (label base rate via p, resolution lag via L, window via oosBars), not of the
|
|
//--- model, which is the whole point: no amount of training moves it.
|
|
string ladder = "";
|
|
double edges[3] = {2.0, 5.0, 10.0};
|
|
for(int i = 0; i < 3; i++)
|
|
{
|
|
double d = edges[i] / 100.0;
|
|
double needEff = BinomialCallsForEdge(p, d, EDGE_MIN_SIGMAS);
|
|
double needRaw = needEff * L;
|
|
double needCoverage = 100.0 * needRaw / (double)oosBars;
|
|
ladder += StringFormat(" %+.0fpp:%.0f indep=%.0f calls=%.0f%% of window%s |",
|
|
edges[i], needEff, needRaw, needCoverage,
|
|
needCoverage > 100.0 ? " IMPOSSIBLE" : "");
|
|
}
|
|
PrintFormat("%s: DETECTABILITY of this configuration (chance precision %.1f%%, mean label"
|
|
" resolution %.1f bars, OOS window %d bars) - to certify an edge of X the gate"
|
|
" needs:%s Read it as a budget, not a target: these are properties of the LABEL and"
|
|
" the WINDOW, so a better model cannot change any of them. Where a rung says"
|
|
" IMPOSSIBLE, no precision this model could ever produce would clear the deploy bar on"
|
|
" this window - the answer there is more instruments or a lower timeframe, never more"
|
|
" eras. Coverage is also not free in the other direction: firing on more bars buys"
|
|
" independent calls at the cost of precision, so the reachable band is bounded at both"
|
|
" ends.",
|
|
m_view.Id(), 100.0 * p, L, oosBars, ladder);
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| RE-CREATE any enabled tunable indicator whose handle has gone |
|
|
//| INVALID underneath us. See the declaration for the evidence. |
|
|
//+------------------------------------------------------------------+
|
|
bool CFeatureBuilder::RepairDeadIndicatorHandles(void)
|
|
{
|
|
CIndicators *indicatorsPtr = m_view.IndicatorsPtr();
|
|
if(indicatorsPtr == NULL)
|
|
return false;
|
|
uint now = GetTickCount();
|
|
//--- Cooldown, because every consumer of ServableBars() can reach this - the training sweep, live
|
|
//--- inference on every tick, online learning - and a repair storm against a terminal that is
|
|
//--- genuinely refusing to create the indicator would be worse than the outage it is fixing.
|
|
if(m_handleRepairTick != 0 && now - m_handleRepairTick < HANDLE_REPAIR_COOLDOWN_MS)
|
|
return false;
|
|
m_handleRepairTick = now;
|
|
//--- NOT released first, deliberately - but NOT because -1 proves the handle is gone.
|
|
int repaired = 0, h = 0;
|
|
string moves = "";
|
|
if(m_view.UseMA() && m_MA.BarsCalculated() < 0)
|
|
{
|
|
h = m_MA.Handle();
|
|
if(InitMA(indicatorsPtr, false))
|
|
repaired += NoteHandleMove("MA", h, m_MA.Handle(), moves);
|
|
}
|
|
if(repaired == 0)
|
|
return false;
|
|
//--- Every cached feature row was computed against the handle that just got replaced. Labels too,
|
|
//--- defensively: only MA is repaired here today and MA feeds no label, but SwingPivotDirectionLabel
|
|
//--- (Labels.mqh) reads m_Close/m_ATR/m_zigZag directly, so the day a repair touches one of those
|
|
//--- this call is already in place rather than being the next silent-staleness incident.
|
|
m_view.FeatureCacheInvalidateAll();
|
|
m_view.LabelCacheInvalidateAll();
|
|
PrintFormat("%s: RECREATED %d indicator handle(s) that answered no calculated bars, so CopyBuffer"
|
|
" failed at every index and every bar of the sweep was rejected.%s A CHANGED number means"
|
|
" the old instance really was gone and this member now holds a new one; SAME means MT5"
|
|
" returned the same refcounted instance, so it was never dead - it had simply not"
|
|
" calculated yet, and this repair was a no-op that cost one reference. Depth is"
|
|
" deliberately NOT re-reported here: a new handle calculates asynchronously and reads -1"
|
|
" until it does, which is the value that triggered the repair. If this line repeats on a"
|
|
" cycle with CHANGING numbers, something is releasing the handle out from under this"
|
|
" member and the recreate is only papering over it.",
|
|
m_view.Id(), repaired, moves);
|
|
return true;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Append " NAME hOLD->hNEW" (or "->hNEW SAME") to a repair report. |
|
|
//| Always counts one repair - the caller only calls it on a Create() |
|
|
//| that succeeded. |
|
|
//+------------------------------------------------------------------+
|
|
int CFeatureBuilder::NoteHandleMove(const string name, const int oldHandle, const int newHandle, string &moves)
|
|
{
|
|
moves += StringFormat(" %s h%d->h%d%s", name, oldHandle, newHandle,
|
|
(oldHandle == newHandle ? " SAME" : ""));
|
|
return 1;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| One " name=depth(hN)" field of the depth report. |
|
|
//+------------------------------------------------------------------+
|
|
string CFeatureBuilder::IndicatorDepthField(const string name, const int depth, const int handle)
|
|
{
|
|
return StringFormat(" %s=%d(h%d)", name, depth, handle);
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| See the declaration. Blocks in the order BufferTempDataCompute |
|
|
//| emits them, widths as Topology.mqh's m_neuronsCount sum declares |
|
|
//| them - those two are the authority and this must track both. |
|
|
//+------------------------------------------------------------------+
|
|
string CFeatureBuilder::FeatureSlotName(const int slot)
|
|
{
|
|
string names[10];
|
|
int widths[10];
|
|
int n = 0;
|
|
names[n] = "candle"; widths[n++] = 4;
|
|
names[n] = "swing"; widths[n++] = (m_view.UseSwingContext() ? 9 : 0);
|
|
names[n] = "volume"; widths[n++] = (m_view.UseVolumes() ? 4 : 0);
|
|
names[n] = "time"; widths[n++] = (m_view.UseTime() ? 6 : 0);
|
|
names[n] = "atr"; widths[n++] = (m_view.UseATR() ? 1 : 0);
|
|
names[n] = "ma"; widths[n++] = (m_view.UseMA() ? 5 : 0);
|
|
names[n] = "news"; widths[n++] = (m_view.UseNews() ? 2 : 0);
|
|
names[n] = "spread"; widths[n++] = (m_view.UseSpreadFeature() ? 2 : 0);
|
|
names[n] = "crossasset"; widths[n++] = (m_view.UseCrossAsset() ? CROSSASSET_FEATURES : 0);
|
|
names[n] = "alt"; widths[n++] = (m_view.UseAltData() ? m_view.AltDataFeatureCount() : 0);
|
|
int total = 0;
|
|
for(int i = 0; i < n; i++)
|
|
total += widths[i];
|
|
if(total != m_view.NeuronsCount())
|
|
return StringFormat("f%d?", slot);
|
|
int from = 0;
|
|
for(int i = 0; i < n; i++)
|
|
{
|
|
if(widths[i] > 0 && slot < from + widths[i])
|
|
return StringFormat("%s[%d]", names[i], slot - from);
|
|
from += widths[i];
|
|
}
|
|
return StringFormat("f%d", slot);
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| See the declaration. Names WHICH indicator is short, so the next |
|
|
//| occurrence is read off the log instead of inferred. |
|
|
//+------------------------------------------------------------------+
|
|
string CFeatureBuilder::IndicatorDepthReport(void)
|
|
{
|
|
string s = StringFormat(" price=%d", Bars(m_view.SymbolName(), PERIOD_CURRENT));
|
|
//--- HANDLE NUMBER beside every depth, not just MA's. A depth alone cannot say whether a handle
|
|
//--- was never created or was created and later released out from under this member; the number
|
|
//--- can.
|
|
if(m_view.UseMA())
|
|
s += IndicatorDepthField("MA", m_MA.BarsCalculated(), m_MA.Handle());
|
|
//--- Not tunable, so absent from TunableBarsCalculated() - but the swing block reads it on every bar
|
|
//--- and neutral-fills when it is short, which is silent. Worth seeing next to the others.
|
|
s += IndicatorDepthField("ZigZag", m_view.ZigZagBarsCalculated(), m_view.ZigZagHandle());
|
|
s += IndicatorDepthField("ATR", m_view.AtrBarsCalculated(), m_view.AtrHandle());
|
|
return s;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Adopt a saved indicator-param set, rebuilding handles only on a |
|
|
//| REAL change. |
|
|
//+------------------------------------------------------------------+
|
|
bool CFeatureBuilder::AdoptIndicatorParams(const double &loaded[], CIndicators *indicators)
|
|
{
|
|
double current[];
|
|
m_view.IndicatorTuner().Flatten(current);
|
|
bool changed = (ArraySize(current) != ArraySize(loaded));
|
|
if(!changed)
|
|
for(int k = 0; k < ArraySize(loaded); k++)
|
|
if(current[k] != loaded[k])
|
|
{
|
|
changed = true;
|
|
break;
|
|
}
|
|
//--- the tuner mirrors the model's params either way - it feeds the .nnw save and the fingerprint
|
|
m_view.IndicatorTuner().Unflatten(loaded);
|
|
if(!changed)
|
|
{
|
|
PrintVerbose(m_view.Id() + ": saved indicator params match the live indicators - keeping the existing"
|
|
" instances (no handle rebuild).");
|
|
return true;
|
|
}
|
|
Print(m_view.Id() + ": saved indicator params differ from the live defaults - rebuilding the tunable"
|
|
" indicator handles to match the model they trained.");
|
|
return ReInitTunableIndicators(indicators);
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
bool CFeatureBuilder::ReInitTunableIndicators(CIndicators *indicators)
|
|
{
|
|
bool result = true;
|
|
//--- The MA feature is the only tunable indicator left - the RSI, MACD, Ichimoku and five
|
|
//--- AD/Wyckoff groups were removed 2026-08-24. The release-AFTER-recreate order below is kept
|
|
//--- exactly as it was: it is not bookkeeping, it is the fix described at the release itself.
|
|
int hMA = m_view.UseMA() ? m_MA.Handle() : INVALID_HANDLE;
|
|
if(m_view.UseMA())
|
|
result = InitMA(indicators, false) && result;
|
|
//--- AFTER the re-create, never before: releasing first can drop the terminal's last reference
|
|
//--- and make it tear the instance down, so an identical-params Create() would then rebuild it
|
|
//--- from scratch instead of re-using the live one - turning a refcount bump into a full
|
|
//--- recalculation over the whole history, 324 times over.
|
|
if(hMA != INVALID_HANDLE)
|
|
IndicatorRelease(hMA);
|
|
//--- Indicator params just changed, so every cached feature row is now stale. The label cache is
|
|
//--- ALSO invalidated here, defensively - only MA is re-created by this path today and MA feeds no
|
|
//--- label, but SwingPivotDirectionLabel (Labels.mqh) reads m_Close/m_ATR/m_zigZag directly, so
|
|
//--- "labels come from ZigZag, not from a tunable indicator" is not the same claim as "labels never
|
|
//--- need invalidating here". Without the feature invalidation, a tuner candidate would silently
|
|
//--- train and be scored on the PREVIOUS candidate's features. Both are cheap: just flags rows for
|
|
//--- lazy recompute/relabel on next read.
|
|
m_view.FeatureCacheInvalidateAll();
|
|
m_view.LabelCacheInvalidateAll();
|
|
return result;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Cache-or-compute wrapper around BufferTempDataCompute(): a given |
|
|
//| now-relative bar index's feature vector is invariant until the |
|
|
//| next candle close (see m_featureCache's declaration comment), so |
|
|
//| a cache hit just replays the m_neuronsCount values already |
|
|
//| computed for this idx straight into TempData instead of re- |
|
|
//| deriving them from price/ATR/AD-indicator buffers again. |
|
|
//+------------------------------------------------------------------+
|
|
bool CFeatureBuilder::BufferTempData(int idx)
|
|
{
|
|
CArrayDouble *td = m_view.TempData();
|
|
int width = m_view.NeuronsCount();
|
|
bool cacheable = (idx >= 0 && idx < m_view.FeatureCacheSize() && width > 0);
|
|
if(cacheable && m_view.FeatureCacheHasValue(idx))
|
|
{
|
|
if(!m_view.FeatureCacheIsValid(idx))
|
|
return false;
|
|
int base = idx * width;
|
|
//--- Bulk read instead of `width` individual FeatureCacheAt() calls (each a CheckPointer plus
|
|
//--- two virtual dispatches for one double) - this is the cache-HIT path, the common one after
|
|
//--- era 0, walked historyBars*neuronsCount times per sample.
|
|
double block[];
|
|
m_view.FeatureCacheBlock(base, width, block);
|
|
if(ArraySize(block) != width || !td.AddArray(block))
|
|
return false;
|
|
return true;
|
|
}
|
|
int startTotal = td.Total();
|
|
bool ok = BufferTempDataCompute(idx);
|
|
//--- WIDTH CONTRACT. Caught here rather than left to surface as BuildFeatureWindow's length
|
|
//--- check, which cannot say which bar or which block was responsible.
|
|
if(ok)
|
|
{
|
|
int produced = td.Total() - startTotal;
|
|
if(produced != width)
|
|
{
|
|
ok = false;
|
|
m_featureFailTransient = false; // a width fault is structural, never "not ready yet"
|
|
if(!m_featureWidthWarned)
|
|
{
|
|
m_featureWidthWarned = true;
|
|
PrintFormat("%s: FEATURE WIDTH MISMATCH at bar %d - the enabled blocks produced %d values"
|
|
" but m_neuronsCount says %d. Every feature after the short block would have"
|
|
" landed in the wrong slot, so the bar is rejected rather than trained on."
|
|
" A block that can be conditionally unavailable must emit neutral values, not"
|
|
" nothing. Check the optional blocks first (cross-asset XA, spread SPR, swing"
|
|
" context) - those are the ones with an availability test.",
|
|
m_view.Id(), idx, produced, width);
|
|
}
|
|
//--- Roll back the partial bar so the caller's window cannot contain half of it.
|
|
while(td.Total() > startTotal)
|
|
td.Delete(td.Total() - 1);
|
|
}
|
|
}
|
|
//--- ONLY SUCCESSES ARE CACHED. A miss is never stored, in any form.
|
|
if(cacheable && ok)
|
|
{
|
|
m_view.FeatureCacheMarkStored(idx);
|
|
int base = idx * width;
|
|
int count = td.Total() - startTotal;
|
|
//--- Bulk write instead of `count` individual FeatureCacheSetAt() calls - same values, same
|
|
//--- order, min(count,width) preserved exactly as the loop it replaces.
|
|
double block[];
|
|
ArrayResize(block, MathMin(count, width));
|
|
for(int f = 0; f < ArraySize(block); f++)
|
|
block[f] = td.At(startTotal + f);
|
|
m_view.FeatureCacheSetBlock(base, ArraySize(block), block);
|
|
}
|
|
return ok;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| THE ONE PLACE a feature WINDOW is assembled. |
|
|
//+------------------------------------------------------------------+
|
|
void CFeatureBuilder::ReportFeatureHealth(int bars)
|
|
{
|
|
int neuronsCount = m_view.NeuronsCount();
|
|
if(m_featureHealthReported || neuronsCount <= 0)
|
|
return;
|
|
m_featureHealthReported = true;
|
|
int per = neuronsCount; // features per BAR
|
|
int lo = MathMax(m_view.HistoryBars() + m_view.LabelResolutionBars() + 2, 2);
|
|
int hi = MathMax(bars - 2, lo);
|
|
if(hi <= lo)
|
|
return;
|
|
//--- Evenly spaced sample across the whole range, so a block that dies only in the deep history
|
|
//--- (the alt-coverage case) is caught as surely as one that is dead everywhere (the cold-indicator
|
|
//--- case). 400 bars is enough to call a feature constant and costs a fraction of one era.
|
|
int want = 400;
|
|
int step = MathMax((hi - lo) / want, 1);
|
|
double vmin[], vmax[];
|
|
int zeroCnt[], seen = 0;
|
|
ArrayResize(vmin, per);
|
|
ArrayResize(vmax, per);
|
|
ArrayResize(zeroCnt, per);
|
|
for(int j = 0; j < per; j++)
|
|
{
|
|
vmin[j] = DBL_MAX;
|
|
vmax[j] = -DBL_MAX;
|
|
zeroCnt[j] = 0;
|
|
}
|
|
CArrayDouble *td = m_view.TempData();
|
|
for(int i = lo; i <= hi; i += step)
|
|
{
|
|
//--- Read ONE bar's block, not a whole window: the per-bar row is what the blocks produce, and
|
|
//--- BuildFeatureWindow would just replicate it historyBars times.
|
|
td.Clear();
|
|
if(!BufferTempData(i))
|
|
continue;
|
|
if(td.Total() < per)
|
|
continue;
|
|
//--- The bar's own row is the LAST `per` values (BufferTempData appends).
|
|
int base = td.Total() - per;
|
|
for(int j = 0; j < per; j++)
|
|
{
|
|
double v = td.At(base + j);
|
|
if(!MathIsValidNumber(v))
|
|
continue;
|
|
if(v < vmin[j]) vmin[j] = v;
|
|
if(v > vmax[j]) vmax[j] = v;
|
|
if(v == 0.0) zeroCnt[j]++;
|
|
}
|
|
seen++;
|
|
}
|
|
if(seen < 20)
|
|
{
|
|
Print(m_view.Id() + StringFormat(": feature health - only %d of %d sampled bars produced a readable row;"
|
|
" too few to judge. This is itself a warning: if it persists the feature"
|
|
" path is rejecting nearly everything.", seen, want));
|
|
return;
|
|
}
|
|
string deadList = "", zeroList = "";
|
|
int dead = 0, mostlyZero = 0;
|
|
for(int j = 0; j < per; j++)
|
|
{
|
|
if(vmin[j] > vmax[j])
|
|
continue; // never read
|
|
bool isConst = (vmax[j] - vmin[j]) <= 1e-12;
|
|
bool isZeroy = (zeroCnt[j] * 2 > seen);
|
|
//--- Named, not numbered. "slot 30 is mostly zero" is a puzzle; "spread[1] is mostly zero" is
|
|
//--- an answer, and here a benign one - a spread CHANGE ratio is exactly 0 whenever the broker
|
|
//--- quotes the same spread two bars running.
|
|
string tag = FeatureSlotName(j);
|
|
if(isConst)
|
|
{
|
|
dead++;
|
|
if(dead <= 12)
|
|
deadList += StringFormat("%s%s=%.4g", (deadList == "" ? "" : " "), tag, vmin[j]);
|
|
}
|
|
else
|
|
if(isZeroy)
|
|
{
|
|
mostlyZero++;
|
|
if(mostlyZero <= 12)
|
|
zeroList += StringFormat("%s%s(%.0f%%)", (zeroList == "" ? "" : " "), tag,
|
|
100.0 * zeroCnt[j] / seen);
|
|
}
|
|
}
|
|
Print(m_view.Id() + StringFormat(": FEATURE HEALTH on %d sampled bars x %d features%s - %d CONSTANT%s%s |"
|
|
" %d mostly-zero (>50%%)%s%s. A constant feature contributes nothing but"
|
|
" still consumes a first-layer column and a BatchNorm slot; a block that is"
|
|
" constant AND zero is usually a source that failed silently rather than a"
|
|
" quiet market.",
|
|
seen, per,
|
|
//--- Every slot below is named by its block now, so the old "alt block =
|
|
//--- slots N..M" hint has nothing left to disambiguate.
|
|
"",
|
|
dead, (deadList == "" ? "" : ": "), deadList,
|
|
mostlyZero, (zeroList == "" ? "" : ": "), zeroList));
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
bool CFeatureBuilder::BuildFeatureWindow(int r)
|
|
{
|
|
int historyBars = m_view.HistoryBars();
|
|
int neuronsCount = m_view.NeuronsCount();
|
|
CArrayDouble *td = m_view.TempData();
|
|
int width = historyBars * neuronsCount;
|
|
td.Clear();
|
|
td.Reserve(width);
|
|
if(r < 0 || historyBars <= 0 || neuronsCount <= 0)
|
|
return false;
|
|
//--- Live-only freshness probe for the external block: two comparisons when quiet, a reload at
|
|
//--- most hourly once the chart outruns the exported data. Never fires in the tester (the newest
|
|
//--- bar is historical there).
|
|
if(m_view.UseAltData())
|
|
m_view.AltDataEnsureFresh(m_view.TimeAt(0));
|
|
//--- b counts bars BACK from r, so (m_historyBars - 1 - b) emits the deepest lookback first and
|
|
//--- lands on r itself on the final iteration. Identical set of bars as before, opposite order.
|
|
for(int b = 0; b < historyBars; b++)
|
|
if(!BufferTempData(r + (historyBars - 1 - b)))
|
|
{
|
|
//--- Which lookback slot rejected, and how much of the window had been assembled. Without
|
|
//--- this the pass-1 stall report can only say "0 of 54681 usable", which is true of a cold
|
|
//--- ATR, a missing optional block and an out-of-range index alike.
|
|
m_view.SetWindowFail(b, td.Total());
|
|
return false;
|
|
}
|
|
if(td.Total() < width)
|
|
{
|
|
//--- Nothing rejected the bar and the window is still short.
|
|
m_view.SetWindowFail(-1, td.Total());
|
|
return false;
|
|
}
|
|
//--- THE ANCHOR BAR'S EXTERNAL READING ENTERS THE WINDOW ONCE, NOT ONCE PER BAR OF ITS DAY
|
|
//--- (2026-08-16).
|
|
if(m_view.UseAltData())
|
|
{
|
|
int an = m_view.AltDataFeatureCount();
|
|
if(an > 0 && an <= neuronsCount && historyBars > 1)
|
|
{
|
|
double anchor[];
|
|
ArrayResize(anchor, an);
|
|
int newest = (historyBars - 1) * neuronsCount + (neuronsCount - an);
|
|
for(int k = 0; k < an; k++)
|
|
anchor[k] = td.At(newest + k);
|
|
for(int b = historyBars - 2; b >= 0; b--)
|
|
{
|
|
int altBase = b * neuronsCount + (neuronsCount - an);
|
|
bool same = true;
|
|
for(int k = 0; k < an && same; k++)
|
|
if(td.At(altBase + k) != anchor[k])
|
|
same = false;
|
|
if(!same)
|
|
break; // a different reading: this bar and everything older keep their values as-is
|
|
for(int k = 0; k < an; k++)
|
|
td.Update(altBase + k, 0.0);
|
|
}
|
|
}
|
|
}
|
|
return true;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| (Re)build the cross-asset panel over `bars` bars. |
|
|
//+------------------------------------------------------------------+
|
|
bool CFeatureBuilder::BuildCrossAssetPanel(int bars)
|
|
{
|
|
if(!m_view.UseCrossAsset())
|
|
return true;
|
|
if(bars <= 0)
|
|
return false;
|
|
CCrossAssetPanel *xa = m_view.CrossAsset();
|
|
//--- Deep enough AND anchored to the current newest bar.
|
|
datetime anchor = m_view.TimeAt(0);
|
|
if(xa.IsReady() && xa.Bars() >= bars && m_crossAssetAnchor == anchor && anchor > 0)
|
|
return true;
|
|
//--- A trained model builds from the pair set it was trained on (adopted from the .cfg), never
|
|
//--- from whatever Market Watch holds today - see m_crossAssetPairsPinned.
|
|
string pinned = m_view.CrossAssetPairsPinned();
|
|
if(pinned != "" && !xa.HasPinnedPairs())
|
|
xa.SetPinnedPairs(pinned);
|
|
if(!xa.Build(m_view.SymbolName(), m_view.Timeframe(), bars))
|
|
{
|
|
m_crossAssetAnchor = 0;
|
|
return false;
|
|
}
|
|
m_crossAssetAnchor = anchor;
|
|
//--- FIRST successful build of a model with no pinned set yet: this pair set is now this model's
|
|
//--- pair set for life.
|
|
if(pinned == "" && xa.UsedPairsCsv() != "")
|
|
{
|
|
pinned = xa.UsedPairsCsv();
|
|
m_view.SetCrossAssetPairsPinned(pinned);
|
|
xa.SetPinnedPairs(pinned);
|
|
if(!m_view.CrossAssetCfgSaved() && m_view.ActiveFileName() != "")
|
|
{
|
|
m_view.SetCrossAssetCfgSaved(true);
|
|
if(m_view.CommitCrossAssetPin())
|
|
Print(m_view.Id() + ": cross-asset pair set PINNED to the .cfg - [" + pinned +
|
|
"]. Restarts and redeploys now build the panel from exactly this set; Market Watch "
|
|
"changes no longer alter what a trained model's features mean.");
|
|
else
|
|
Print(m_view.Id() + ": WARNING - failed to pin the cross-asset pair set to the .cfg; a restart "
|
|
"will re-discover Market Watch instead of adopting the trained set.");
|
|
}
|
|
}
|
|
return true;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Copy the historical spread series onto the current bar grid. |
|
|
//+------------------------------------------------------------------+
|
|
bool CFeatureBuilder::EnsureSpreadSeries(int bars)
|
|
{
|
|
if(!m_view.UseSpreadFeature())
|
|
return true;
|
|
if(bars <= 0)
|
|
return false;
|
|
//--- Length alone is NOT a sufficient cache key - see m_spreadSeriesAnchor's declaration comment.
|
|
datetime anchor = m_view.TimeAt(0);
|
|
if(m_spreadSeriesBars >= bars && m_spreadSeriesAnchor == anchor && anchor > 0)
|
|
return true;
|
|
ArraySetAsSeries(m_spreadSeries, true); // index 0 = newest, matching every other buffer here
|
|
int got = CopySpread(m_view.SymbolName(), m_view.Timeframe(), 0, bars, m_spreadSeries);
|
|
if(got <= 0)
|
|
{
|
|
m_spreadSeriesBars = 0;
|
|
m_spreadSeriesAnchor = 0;
|
|
Print(__FUNCTION__ + ": CopySpread returned " + IntegerToString(got) + " for " + m_view.SymbolName() +
|
|
" - spread features 0-filled this run.");
|
|
return false;
|
|
}
|
|
m_spreadSeriesBars = got;
|
|
m_spreadSeriesAnchor = anchor;
|
|
return true;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
bool CFeatureBuilder::BufferTempDataCompute(int idx)
|
|
{
|
|
CArrayDouble *td = m_view.TempData();
|
|
//--- Where THIS bar's block starts. The function appends m_neuronsCount values below; remembering
|
|
//--- the offset lets the whole vector be validated in one place at the end instead of at each of
|
|
//--- the ~60 Add() call sites.
|
|
int featureStart = td.Total();
|
|
//--- Cleared here, set by the two NOT-READY-YET guards below. See BufferTempData() for what it
|
|
//--- controls: a rejection caused by data that has not arrived yet must not be cached, because the
|
|
//--- cache never re-tries a miss.
|
|
m_featureFailTransient = false;
|
|
//--- Cleared alongside it, and written by every guard below that can return false - see the
|
|
//--- declaration for why a value COUNT was never enough to identify the block.
|
|
m_view.SetFailBlock("");
|
|
m_view.SetFailIdx(idx);
|
|
double open = m_view.OpenAt(idx);
|
|
double close = m_view.CloseAt(idx);
|
|
double high = m_view.HighAt(idx);
|
|
double low = m_view.LowAt(idx);
|
|
MqlDateTime sTime;
|
|
TimeToStruct(m_view.TimeAt(idx), sTime);
|
|
if(open == EMPTY_VALUE)
|
|
{
|
|
m_featureFailTransient = true;
|
|
m_view.SetFailBlock("price/open (m_Open.GetData == EMPTY_VALUE)");
|
|
return false;
|
|
}
|
|
//--- close/high/low go through their OWN CopyClose/CopyHigh/CopyLow underneath CiClose/CiHigh/
|
|
//--- CiLow, so a mid-sync short read can hit one series and not another - open alone being clear
|
|
//--- does not prove these are. Unguarded, a EMPTY_VALUE==DBL_MAX close/high/low here poisons every
|
|
//--- ATR-normalized feature computed from it for the rest of this function, most of which clamp to
|
|
//--- a maximum-magnitude value ([-10,10] etc.) that passes FEATURE_ABS_MAX looking like a real,
|
|
//--- confident reading rather than corrupt data.
|
|
if(close == EMPTY_VALUE || high == EMPTY_VALUE || low == EMPTY_VALUE)
|
|
{
|
|
m_featureFailTransient = true;
|
|
m_view.SetFailBlock(StringFormat("price/close-high-low (close=%s high=%s low=%s)",
|
|
(close == EMPTY_VALUE ? "EMPTY" : "ok"),
|
|
(high == EMPTY_VALUE ? "EMPTY" : "ok"),
|
|
(low == EMPTY_VALUE ? "EMPTY" : "ok")));
|
|
return false;
|
|
}
|
|
//--- ATR-normalize every raw-price-unit feature below instead of feeding e.g. 0.0005 on EURUSD
|
|
//--- vs.
|
|
double atr = m_view.AtrMain(idx);
|
|
if(atr <= 0.0 || atr == EMPTY_VALUE)
|
|
{
|
|
//--- TRANSIENT BY NATURE, and the reason resumed models could never train. A FRESH model
|
|
//--- never saw this: it sits through m_warmupPassesRemaining separately-scheduled Train()
|
|
//--- calls before anything touches a feature, which is exactly what those passes are for.
|
|
m_featureFailTransient = true;
|
|
m_view.SetFailBlock(StringFormat("ATR (m_ATR.Main=%.10g, needs > 0)", atr));
|
|
return false;
|
|
}
|
|
if(!td.Add((close - open) / atr) ||
|
|
!td.Add((high - open) / atr) ||
|
|
!td.Add((low - open) / atr) ||
|
|
// Explicit bullish/bearish flag - (close-open)/atr already encodes direction *and* magnitude
|
|
// together, which asks the network to disentangle "which way" from "how much" out of a single
|
|
// continuous value. Giving direction its own clean +1/-1/0 signal removes that ambiguity.
|
|
!td.Add(close > open ? 1.0 : (close < open ? -1.0 : 0.0)))
|
|
{
|
|
return false;
|
|
}
|
|
if(m_view.UseSwingContext())
|
|
{
|
|
//--- Most recent CONFIRMED swing pivot as of bar idx - "confirmed" meaning at least
|
|
//--- m_swingConfirmationBars MORE bars have closed after it (see m_swingConfirmationBars' and
|
|
//--- m_useSwingContext's declaration comments).
|
|
int pivotIdx = -1;
|
|
double pivotPrice = 0.0;
|
|
bool pivotIsLow = false;
|
|
int swingConfirmationBars = m_view.SwingConfirmationBars();
|
|
if(!m_view.FindConfirmedZigZagPivot(idx + MathMax(swingConfirmationBars, 1), pivotIdx, pivotPrice, pivotIsLow))
|
|
{
|
|
// No confirmed pivot within the scan cap (e.g. right at the start of available history) -
|
|
// this is legitimately "no swing context yet", not bad/missing data, so a neutral 0-fill
|
|
// keeps the bar usable rather than rejecting it outright like the ATR/EMPTY_VALUE guards do.
|
|
if(!td.Add(0.0) || !td.Add(0.0) || !td.Add(0.0) || !td.Add(0.0) || !td.Add(0.0))
|
|
return false;
|
|
}
|
|
else
|
|
{
|
|
// Direction of the CURRENT leg: the last confirmed pivot being a bottom means price has been
|
|
// rising away from it (an up-leg) ever since, and vice versa - same +1/-1 convention as the
|
|
// bullish/bearish flag above, just at swing scale instead of single-bar scale.
|
|
double direction = pivotIsLow ? 1.0 : -1.0;
|
|
// How far price has travelled since that pivot, ATR-normalized and signed (+ve above the
|
|
// pivot price, -ve below) - clamped generously since an extended trending leg has no natural
|
|
// ceiling the way a single bar's range does.
|
|
double distSincePivot = MathMax(-10.0, MathMin(10.0, (close - pivotPrice) / atr));
|
|
//--- Magnitude of the PRIOR completed leg (the pivot immediately before pivotIdx) - a
|
|
//--- scale reference for "is the current move big or small relative to the last full
|
|
//--- swing".
|
|
int priorPivotIdx = -1;
|
|
double priorPivotPrice = 0.0;
|
|
bool priorPivotIsLow = false;
|
|
bool havePrior = m_view.FindConfirmedZigZagPivot(pivotIdx + 1, priorPivotIdx, priorPivotPrice, priorPivotIsLow);
|
|
double priorLegMagnitude = havePrior ? MathMax(0.0, MathMin(10.0, MathAbs(pivotPrice - priorPivotPrice) / atr)) : 0.0;
|
|
//--- Retracement/extension ratio (current distance relative to the prior leg's own size) -
|
|
//--- Fibonacci-style relative position, often more informative than either raw magnitude
|
|
//--- alone since it's comparable across both quiet and volatile regimes.
|
|
double retracementRatio = (havePrior && priorLegMagnitude > 0.0001) ?
|
|
MathMax(-5.0, MathMin(5.0, distSincePivot / priorLegMagnitude)) : 0.0;
|
|
// Swing age (bars since the pivot) - a maturity/exhaustion proxy, same +/- style clamp
|
|
// convention as the volume-ratio feature below.
|
|
double barsSincePivot = MathMax(0.0, MathMin(5.0, (double)(pivotIdx - idx) / 100.0));
|
|
if(!td.Add(direction) ||
|
|
!td.Add(distSincePivot) ||
|
|
!td.Add(priorLegMagnitude) ||
|
|
!td.Add(retracementRatio) ||
|
|
!td.Add(barsSincePivot))
|
|
return false;
|
|
}
|
|
//--- Recent price-action context (4 features), computed from CLOSED bars at idx or older only
|
|
//--- - no ZigZag confirmation, so no repainting and NO embargo, and never stale, unlike the
|
|
//--- five pivot-anchored features above whose confirmed anchor is always >=
|
|
//--- m_swingConfirmationBars (~100) bars old.
|
|
double hi20 = high, lo20 = low, hi50 = high, lo50 = low;
|
|
double sum20 = close, oldestClose20 = close;
|
|
int cnt20 = 1;
|
|
for(int w = 1; w < 50; w++)
|
|
{
|
|
int j = idx + w;
|
|
double jc = m_view.CloseAt(j);
|
|
double jh = m_view.HighAt(j);
|
|
double jl = m_view.LowAt(j);
|
|
// ran off the oldest edge of loaded history (out-of-range reads back as 0/EMPTY_VALUE) -
|
|
// use whatever window we gathered so far rather than rejecting the bar; a shorter early-
|
|
// history window is degraded-but-usable, same spirit as the pivot 0-fill above.
|
|
//--- jc checked too: a bad close alone (jh/jl still fine) would otherwise set
|
|
//--- oldestClose20 = EMPTY_VALUE == DBL_MAX, and recentReturn/smaExtension below clamp that
|
|
//--- to their maximum-magnitude bearish reading rather than rejecting it.
|
|
if(jh == EMPTY_VALUE || jh <= 0.0 || jl <= 0.0 || jc == EMPTY_VALUE)
|
|
break;
|
|
if(jh > hi50)
|
|
hi50 = jh;
|
|
if(jl < lo50)
|
|
lo50 = jl;
|
|
if(w < 20)
|
|
{
|
|
if(jh > hi20)
|
|
hi20 = jh;
|
|
if(jl < lo20)
|
|
lo20 = jl;
|
|
sum20 += jc;
|
|
oldestClose20 = jc;
|
|
cnt20++;
|
|
}
|
|
}
|
|
//--- Donchian position: where close sits inside the recent high/low range, rescaled to
|
|
//--- [-1,+1] (-1 = at the range low / bottom candidate, +1 = at the range high / top
|
|
//--- candidate, 0 = mid- range / mid-trend).
|
|
double range20 = hi20 - lo20;
|
|
double range50 = hi50 - lo50;
|
|
double donchPos20 = (range20 > 0.0) ? ((close - lo20) / range20 - 0.5) * 2.0 : 0.0;
|
|
double donchPos50 = (range50 > 0.0) ? ((close - lo50) / range50 - 0.5) * 2.0 : 0.0;
|
|
// Net directional displacement over the recent window, ATR-normalized and signed - the
|
|
// prevailing-trend strength/direction the counter-trend clusters were ignoring.
|
|
double recentReturn = MathMax(-10.0, MathMin(10.0, (close - oldestClose20) / atr));
|
|
// Distance from the recent mean (SMA), ATR-normalized - a stretch/exhaustion proxy distinct
|
|
// from the net return (a move can be far from its mean with little net displacement, or vice
|
|
// versa); genuine reversals tend to be over-extended from equilibrium.
|
|
double smaExtension = MathMax(-10.0, MathMin(10.0, (close - sum20 / cnt20) / atr));
|
|
if(!td.Add(donchPos20) ||
|
|
!td.Add(donchPos50) ||
|
|
!td.Add(recentReturn) ||
|
|
!td.Add(smaExtension))
|
|
return false;
|
|
}
|
|
if(m_view.UseVolumes())
|
|
{
|
|
//--- FOUR values, not one.
|
|
double vNow = m_Volumes.Main(idx);
|
|
double prevVolume = m_Volumes.Main(idx + 1);
|
|
double volumeDelta = vNow - prevVolume;
|
|
//--- Relative change - trading activity magnitude varies wildly across symbols/timeframes, so
|
|
//--- the previous bar's own volume is the scale reference, same logic as ATR-normalizing
|
|
//--- price above.
|
|
double volumeChangeRatio = prevVolume > 0.0 ? volumeDelta / prevVolume : 0.0;
|
|
// Baseline over the trailing 50 bars, walking toward OLDER bars only (increasing index), so
|
|
// nothing here can see the future. Degraded-but-usable at the oldest edge, same convention as
|
|
// the swing-context window above: a short early-history baseline beats rejecting the bar.
|
|
double volSum = vNow;
|
|
int volCnt = 1;
|
|
for(int w = 1; w < 50; w++)
|
|
{
|
|
double jv = m_Volumes.Main(idx + w);
|
|
if(jv <= 0.0)
|
|
break;
|
|
volSum += jv;
|
|
volCnt++;
|
|
}
|
|
double volBase = volSum / volCnt;
|
|
// LEVEL: is this an active bar or a dead one? The change ratio cannot express this at all -
|
|
// two consecutive dead bars and two consecutive frantic ones both read as ~0 change.
|
|
double volLevel = (volBase > 0.0) ? vNow / volBase : 1.0;
|
|
double rangeAtr = (high - low) / atr;
|
|
// ABSORPTION: range delivered per unit of activity. A low value means heavy participation that
|
|
// went nowhere - supply meeting demand - which is a categorically different bar from heavy
|
|
// participation that travelled. The single change ratio conflates the two.
|
|
double absorption = (volLevel > 0.05) ? rangeAtr / volLevel : 0.0;
|
|
// ...and its converse, effort AND result together, which is the continuation reading.
|
|
double volXrange = volLevel * rangeAtr;
|
|
if(!td.Add(MathMax(-5.0, MathMin(5.0, volumeChangeRatio))) ||
|
|
!td.Add(MathMax(0.0, MathMin(5.0, volLevel))) ||
|
|
!td.Add(MathMax(0.0, MathMin(5.0, absorption))) ||
|
|
!td.Add(MathMax(0.0, MathMin(5.0, volXrange))))
|
|
return false;
|
|
}
|
|
if(m_view.UseTime())
|
|
{
|
|
// Normalize time (cyclical encoding)
|
|
if(!td.Add(sin(2 * M_PI * sTime.hour / 24.0)))
|
|
return false;
|
|
if(!td.Add(cos(2 * M_PI * sTime.hour / 24.0)))
|
|
return false;
|
|
if(!td.Add(sin(2 * M_PI * sTime.day_of_week / 7.0)))
|
|
return false;
|
|
if(!td.Add(cos(2 * M_PI * sTime.day_of_week / 7.0)))
|
|
return false;
|
|
if(!td.Add(sin(2 * M_PI * sTime.mon / 12.0)))
|
|
return false;
|
|
if(!td.Add(cos(2 * M_PI * sTime.mon / 12.0)))
|
|
return false;
|
|
}
|
|
if(m_view.UseATR())
|
|
{
|
|
// ATR/close (volatility as a fraction of price), not raw ATR - the raw absolute value is
|
|
// itself unnormalized (e.g. ~0.0012 on EURUSD vs. ~1.5 on gold, and drifts over time even on
|
|
// one symbol as its price level changes), which is exactly the kind of scale-dependent
|
|
// feature this whole normalization pass is fixing everywhere else.
|
|
if(!td.Add(close != 0.0 ? atr / close : 0.0))
|
|
return false;
|
|
}
|
|
if(m_view.UseMA())
|
|
{
|
|
//--- Same ATR-normalized distance-from-level convention as the base OHLC-from-open features
|
|
//--- above, just measured against the MA instead of the bar's own open - lets the network
|
|
//--- read where price sits relative to the same MA Signals\SignalMA.mqh votes on.
|
|
double maNow = m_MA.GetData(0, idx);
|
|
double maPrev = m_MA.GetData(0, idx + 1);
|
|
if(maNow == EMPTY_VALUE || maPrev == EMPTY_VALUE)
|
|
{
|
|
//--- TRANSIENT, for exactly the reason spelled out at the ATR guard above, and this is the
|
|
//--- guard that proved it: 2026-08-17, six fresh instances on USDJPY and XAUUSD swept
|
|
//--- 33,965-50,162 bars and produced ZERO usable windows, over and over, for 40 minutes.
|
|
m_featureFailTransient = true;
|
|
//--- WHICH of the two reads failed, and whether the indicator is empty EVERYWHERE or only
|
|
//--- here.
|
|
string maNewest = "reads (buffer live; this is a history-edge miss)";
|
|
if(m_MA.GetData(0, 0) == EMPTY_VALUE)
|
|
maNewest = "ALSO EMPTY (whole buffer unreadable - cold or dead handle, NOT a depth shortfall)";
|
|
m_view.SetFailBlock(StringFormat("MA (iMA) - GetData(%d)=%s GetData(%d)=%s,"
|
|
" newest bar %s, BarsCalculated=%d",
|
|
idx, maNow == EMPTY_VALUE ? "EMPTY" : "ok",
|
|
idx + 1, maPrev == EMPTY_VALUE ? "EMPTY" : "ok",
|
|
maNewest, m_MA.BarsCalculated()));
|
|
return false;
|
|
}
|
|
if(!td.Add((open - maNow) / atr) ||
|
|
!td.Add((high - maNow) / atr) ||
|
|
!td.Add((low - maNow) / atr) ||
|
|
!td.Add((close - maNow) / atr) ||
|
|
!td.Add((maNow - maPrev) / atr))
|
|
return false;
|
|
}
|
|
if(m_view.UseNews())
|
|
{
|
|
//--- Event proximity + impact only - see this member's declaration comment and
|
|
//--- System\NewsRelevance.mqh's ImpactWeightedProximity() for why the forward-looking half
|
|
//--- (searchForward=true) isn't lookahead bias despite being computed for a historical bar.
|
|
datetime barTime = m_view.TimeAt(idx);
|
|
int newsWindow = m_view.NewsFeatureWindowMinutes();
|
|
double newsRecency = ImpactWeightedProximity(m_view.SymbolName(), barTime, newsWindow, false);
|
|
double newsProximity = ImpactWeightedProximity(m_view.SymbolName(), barTime, newsWindow, true);
|
|
if(!td.Add(newsRecency) || !td.Add(newsProximity))
|
|
return false;
|
|
}
|
|
if(m_view.UseSpreadFeature())
|
|
{
|
|
//--- TWO values. What this block actually encodes is worth stating precisely, because the raw
|
|
//--- measurement overstates it.
|
|
double sprRatio = 0.0, sprChange = 0.0;
|
|
if(idx + 1 < m_spreadSeriesBars)
|
|
{
|
|
double sNow = (double)m_spreadSeries[idx] * m_view.SymbolPoint();
|
|
double sPrev = (double)m_spreadSeries[idx + 1] * m_view.SymbolPoint();
|
|
sprRatio = sNow / atr;
|
|
if(sPrev > 0.0)
|
|
sprChange = (sNow - sPrev) / sPrev;
|
|
}
|
|
if(!td.Add(MathMax(0.0, MathMin(5.0, sprRatio))) ||
|
|
!td.Add(MathMax(-5.0, MathMin(5.0, sprChange))))
|
|
return false;
|
|
}
|
|
if(m_view.UseCrossAsset())
|
|
{
|
|
//--- What every OTHER instrument was doing at this bar's timestamp - the one feature block
|
|
//--- here that is not a function of this symbol's own series. See System\CrossAsset.mqh.
|
|
double xa[];
|
|
m_view.CrossAsset().Features(idx, xa);
|
|
for(int k = 0; k < CROSSASSET_FEATURES; k++)
|
|
if(!td.Add(xa[k]))
|
|
return false;
|
|
}
|
|
if(m_view.UseAltData())
|
|
{
|
|
//--- External publication-stamped block (COT/VIX/macro) - see System\AltData.mqh and the
|
|
//--- matching m_neuronsCount block in Topology.mqh. As-of lookup by THIS bar's open time, so
|
|
//--- a bar can only read values the live run would have had.
|
|
double av[];
|
|
m_view.AltDataFeatures(m_view.TimeAt(idx), av);
|
|
int an = m_view.AltDataFeatureCount();
|
|
for(int k = 0; k < an; k++)
|
|
if(!td.Add(av[k]))
|
|
return false;
|
|
}
|
|
//--- ONE finiteness/plausibility gate for the whole bar, rather than 60-odd individually guarded
|
|
//--- Add() calls. Most blocks above already clamp their own output; the AD/Wyckoff blocks
|
|
//--- deliberately do not, because those indicators emit plain readings with no natural range.
|
|
int featureEnd = td.Total();
|
|
for(int f = featureStart; f < featureEnd; f++)
|
|
{
|
|
double v = td.At(f);
|
|
if((!MathIsValidNumber(v) || MathAbs(v) > FEATURE_ABS_MAX) && !td.Update(f, 0.0))
|
|
return false;
|
|
}
|
|
return true;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| "Is this AD indicator still calculating?" MT5 fills custom- |
|
|
//| indicator buffers asynchronously after the handle is created, |
|
|
//| and a cold one returns EMPTY_VALUE for EVERY index - including |
|
|
//| the newest bar, which a warm indicator always has. |
|
|
//+------------------------------------------------------------------+
|
|
bool CFeatureBuilder::ADIndicatorCold(CiCustom &ind, string block)
|
|
{
|
|
if(ind.GetData(0, 0) != EMPTY_VALUE)
|
|
return false;
|
|
m_featureFailTransient = true;
|
|
m_view.SetFailBlock(StringFormat("%s - COLD (newest bar EMPTY, whole buffer unreadable),"
|
|
" BarsCalculated=%d", block, ind.BarsCalculated()));
|
|
return true;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Initialize Volumes indicators. |
|
|
//+------------------------------------------------------------------+
|
|
bool CFeatureBuilder::InitVolumes(CIndicators * indicators)
|
|
{
|
|
//--- check pointer
|
|
if(indicators == NULL)
|
|
return (false);
|
|
//--- add object to collection
|
|
if(!indicators.Add(GetPointer(m_Volumes)))
|
|
{
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
return (false);
|
|
}
|
|
//--- initialize object
|
|
if(!m_Volumes.Create(m_view.SymbolName(), m_view.Timeframe(), VolumeData))
|
|
{
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
return (false);
|
|
}
|
|
//--- ok
|
|
return (true);
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Initialize MA indicator (feature use - see m_useMA). Period comes |
|
|
//| from m_indicatorTuner.maPeriod, not the raw PeriodMA input - it |
|
|
//| starts equal to it (see CADIndicatorTuner's constructor) but may |
|
|
//| diverge once AutoTuneIndicators actually searches a trial. The |
|
|
//| Classic Signals MA vote is unaffected - see m_useMA's declaration |
|
|
//| comment. |
|
|
//+------------------------------------------------------------------+
|
|
bool CFeatureBuilder::InitMA(CIndicators * indicators, bool addToCollection)
|
|
{
|
|
if(indicators == NULL)
|
|
return (false);
|
|
if(addToCollection && !indicators.Add(GetPointer(m_MA)))
|
|
{
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
return (false);
|
|
}
|
|
CADIndicatorTuner *tuner = m_view.IndicatorTuner();
|
|
//--- built-in iMA; type AND period are both tuner-driven (m_indicatorTuner.maType/maPeriod). ma_shift
|
|
//--- is 0 - the feature reads a bar index directly, so displacing the average would only skew it.
|
|
if(!m_MA.Create(m_view.SymbolName(), m_view.Timeframe(), tuner.maPeriod, 0,
|
|
(ENUM_MA_METHOD)tuner.maType, PRICE_CLOSE))
|
|
{
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
return (false);
|
|
}
|
|
return (true);
|
|
}
|
|
#endif // WARRIOR_FEATURES_FEATUREBUILDER_MQH
|
|
//+------------------------------------------------------------------+
|