forked from animatedread/Warrior_EA
5e0317f09d
User request: 'move from arrows on lows and highs to small horizontal lines at the actual prices the entry/exit would trigger, just a bit larger than the candles. dark green for buy, dark red for sell.' Every mark is now an OBJ_TREND segment with both anchors at one price and both rays off, spanning 1.3 bar widths, drawn at the bar's CLOSE - the price a market order actually fires at, and the exact entry TripleBarrierLabel assumes. It used to sit on the candle's LOW for a Buy and its HIGH for a Sell: prices the trade never touches, picked so an arrow glyph would clear the candle. The tooltip now carries that price too. COLOUR NOW MEANS DIRECTION AND ONLY DIRECTION on every layer (dark green / dark red). Layer moves to width+style - the traded vote is solid and thick and drawn in front, a single model's raw opinion is thin, dotted and behind the candles - which keeps the distinction the old palette existed to draw (a model's opinion must never read as a trade) while freeing colour to say one thing consistently. Consequences handled, all of them the same 'a typed scan went blind' failure: - SaveChartSignals filtered OBJPROP_TYPE == OBJ_ARROW and read OBJPROP_ARROWCODE. It now filters OBJ_TREND and recovers direction from the colour. The sidecar keeps the old 217/218 numbers as its buy/sell token deliberately, so existing .arrows files still load. - AdvanceChartSignalRestore now rebuilds through the SAME creation point the live path uses, so a restored mark and a fresh one are identical objects. - The rescan-scoped delete enumerated ObjectsTotal(OBJ_ARROW) - retyped, or it silently deletes nothing. - ApplySignalsVisibility enumerated OBJ_ARROW with NO prefix filter. Under the new type that would have hidden and shown THE USER'S OWN trend lines on every Hide/Show click; it is now prefix-scoped. The old type was uncommon enough on a real chart to mask the missing check - trend lines are the most hand-drawn object there is. - DrawObject's high/low parameters are gone (6 call sites pass m_Close instead), so no caller can hand it a price it no longer draws at. - Fixed a pre-existing stale comment that still described the purge sweep as OBJ_ARROW-only three lines above the note explaining it had been widened to every type. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
912 lines
57 KiB
MQL5
912 lines
57 KiB
MQL5
//+------------------------------------------------------------------+
|
|
//| Warrior_EA |
|
|
//| AnimateDread |
|
|
//| |
|
|
//| Read-time signal production: softmax, prior calibration, class p|
|
|
//| |
|
|
//| PARTIAL IMPLEMENTATION FILE - not standalone. |
|
|
//| This holds CExpertSignalAIBase method BODIES only. The class |
|
|
//| declaration lives in Expert\ExpertSignalAIBase.mqh, which |
|
|
//| #includes this file at the bottom, after the declaration. Do not |
|
|
//| include it anywhere else and do not compile it on its own. |
|
|
//| |
|
|
//| Split out purely to make the 8216-line original navigable; the |
|
|
//| code inside was moved verbatim, not rewritten. |
|
|
//+------------------------------------------------------------------+
|
|
#ifndef WARRIOR_AIBASE_INFERENCE_MQH
|
|
#define WARRIOR_AIBASE_INFERENCE_MQH
|
|
//+------------------------------------------------------------------+
|
|
//| Post-convergence "new bar" handler - see ScheduleTrainingIfNeeded()|
|
|
//| for why this exists: once m_trainingComplete is true, a plain new |
|
|
//| bar must NOT re-enter Train()'s full era loop (which resets the |
|
|
//| best-checkpoint/eta-decay tracking and runs real Net.backProp() |
|
|
//| passes again, silently perturbing an already-converged model |
|
|
//| forever, once per bar, with no way to ever actually finish). This |
|
|
//| only refreshes the price/indicator buffers and re-runs inference |
|
|
//| for the newest bar so dPrevSignal/the chart arrow stay current - |
|
|
//| identical cost to what Train() does per-bar, minus every bit of |
|
|
//| training (label caching, backProp, checkpointing). |
|
|
//+------------------------------------------------------------------+
|
|
void CExpertSignalAIBase::RefreshConvergedSignal(void)
|
|
{
|
|
//--- Meta target: the meta head scores PROPOSED TRADES, not a bare bar window - a candidate-less
|
|
//--- forward would also be width-mismatched against its input layer (window + descriptor). Its
|
|
//--- live path is the S3 gate (LiveMetaGate, wired
|
|
//--- 2026-08-19), which builds its own window + descriptor - this vote-refresh path stays closed.
|
|
if(IsMetaTarget())
|
|
return;
|
|
//--- Size the buffers from what the FEATURE BUILDER actually needs, not from a date delta.
|
|
//--- This used to be `Bars(sym, period, dtStudied, TimeCurrent()) + m_historyBars`. dtStudied is a
|
|
//--- training watermark, and in the Strategy Tester it is loaded from a LIVE-chart save whose
|
|
//--- timestamp is AHEAD of the simulated date - so the interval inverts, Bars() returns ~0, and the
|
|
//--- buffer came out at exactly m_historyBars. That is just deep enough for the OHLC window to
|
|
//--- succeed and far too shallow for the swing-context block behind it: the Donchian-50, the 20-bar
|
|
//--- return and the SMA extension all reach further back than m_historyBars, hit the end of the
|
|
//--- loaded series, and take their graceful degraded path. The result was silent - no error, no short
|
|
//--- window, just inference computing DIFFERENT features from the ones training learned on. Live it
|
|
//--- was the same bug with a milder constant (the delta is ~1 bar, giving m_historyBars + 1).
|
|
//--- SWING_SCAN_CAP_BARS is the deepest lookback any feature performs (FindConfirmedZigZagPivot's
|
|
//--- bound); everything else in BufferTempDataCompute reaches less far.
|
|
int need = (int)m_historyBars + SWING_SCAN_CAP_BARS + MathMax(m_barrierHorizonBars, 1) + 2;
|
|
int barsNow = (int)MathMin(need, Bars(m_symbol.Name(), PERIOD_CURRENT));
|
|
//--- HOLD RATHER THAN TRADE ON A SHORT WINDOW. `need` is the depth the feature builder requires for
|
|
//--- inference features to match the ones training learned on; a shallower buffer does not fail, it
|
|
//--- makes the swing block take its graceful degraded path - which is precisely the silent
|
|
//--- feature-mismatch this whole `need` calculation was introduced to end (see above). If the
|
|
//--- indicators cannot serve `need`, the only safe output is no output: a signal computed from a
|
|
//--- different feature distribution than the model was fitted on is worse than no signal, and this
|
|
//--- EA sizes real positions off it.
|
|
//---
|
|
//--- In practice this cannot fire on a sane terminal - `need` tops out around 1,152 bars (16 + 750 +
|
|
//--- 384 + 2) and the SMALLEST "Max bars in chart" MT5 offers is 5,000. It is insurance against the
|
|
//--- failure mode being reachable at all, not a case expected in the field.
|
|
int servable = ServableBars(need, "live inference");
|
|
if(servable < need)
|
|
{
|
|
if(!m_inferenceDepthRefusalWarned)
|
|
{
|
|
m_inferenceDepthRefusalWarned = true;
|
|
PrintFormat("%s: LIVE INFERENCE HELD - the feature window needs %d bars and the indicators can"
|
|
" only serve %d. Computing a signal here would silently use the swing block's"
|
|
" degraded path, i.e. different features from the ones this model was trained on,"
|
|
" so no signal is emitted until the depth is available. See the indicator-cap line"
|
|
" above for how to raise it.", ID, need, servable);
|
|
}
|
|
return;
|
|
}
|
|
m_inferenceDepthRefusalWarned = false;
|
|
if(!ResizeBuffers(barsNow) || !RefreshData())
|
|
return;
|
|
//--- INVALIDATE THE NOW-RELATIVE BAR CACHES. Non-obvious and load-bearing: the feature cache is keyed
|
|
//--- by MQL5 series index, and index 0 means "newest bar", so every closed candle shifts what every
|
|
//--- cached row stands for. Train() is the only other caller of this, and once m_trainingComplete is
|
|
//--- set ScheduleTrainingIfNeeded() routes every subsequent bar HERE instead - Train() is never
|
|
//--- re-entered, so without this call nothing ever clears the cache again for the rest of the process.
|
|
//--- A chart that trained to convergence (or was deployed via DeployNow()) would then keep replaying
|
|
//--- the rows computed for the last training era's bar grid: BufferTempData(0..m_historyBars-1) all hit
|
|
//--- the cache, the feature window never changes, and dPrevSignal freezes at its convergence-time value
|
|
//--- forever - silently, since every buffer above refreshed correctly and the vector is the right SHAPE.
|
|
//--- OnlineLearnStep() below would compound it by backpropping those stale features against freshly
|
|
//--- resolved labels, i.e. actively training the deployed model on mismatched pairs.
|
|
//--- A freshly started inference-only process (a backtest, or a buyer loading a deployed .nnw) was
|
|
//--- never affected: it never allocates these arrays, so BufferTempData()'s `cacheable` test is false
|
|
//--- and it always recomputes. This is a live/forward-chart fix, not a backtest one.
|
|
EnsureBarCachesCapacity(barsNow);
|
|
//--- Same bar grid, same panel. A deployed model never enters Train(), so this is the only place
|
|
//--- its cross-asset panel gets built - and it must be built from the SAME reference set training
|
|
//--- used, or inference reads a different feature vector than the weights were fitted to.
|
|
//--- Only as deep as inference actually reads. RefreshLatestSignal() touches bars 1..m_historyBars
|
|
//--- (window ends on the newest CLOSED bar - the +2 slack below covers the extra bar of depth)
|
|
//--- and the panel's own slow window reaches CROSSASSET_SLOW_BARS further back - nothing else. Asking
|
|
//--- for the full `barsNow` here would rebuild a training-depth panel on EVERY bar, which in the
|
|
//--- tester means one full multi-symbol resample per simulated bar. The cache check in
|
|
//--- BuildCrossAssetPanel is >=, so a deeper panel left over from training still satisfies this.
|
|
BuildCrossAssetPanel((int)m_historyBars + CROSSASSET_SLOW_BARS + 2);
|
|
EnsureSpreadSeries(barsNow);
|
|
//--- A deployed model never enters Train(), so this is the only place its barrier horizon gets
|
|
//--- measured - and OnlineLearnStep() below depends on it being right. First call sizes buffers
|
|
//--- against the fallback, which is harmless: `need` is dominated by SWING_SCAN_CAP_BARS either way.
|
|
EnsureBarrierHorizon(barsNow);
|
|
bool refreshed = RefreshLatestSignal();
|
|
//--- Continual learning: on a LIVE chart (never the tester/optimizer - OnlineLearnStep() self-guards
|
|
//--- on m_inferenceOnly) a deployed model keeps adapting to newly-confirmed structure. Runs AFTER the
|
|
//--- live signal is drawn (so the arrow uses the shadow as it was for THIS bar's decision) and BEFORE
|
|
//--- dtStudied advances (OnlineLearnStep keeps its own time watermark, independent of dtStudied).
|
|
OnlineLearnStep();
|
|
//--- Advance the live new-bar watermark ONLY on success. Advancing it unconditionally meant a
|
|
//--- transient window failure (indicator hole, history hiccup) closed the gate for the rest of the
|
|
//--- bar with the PREVIOUS bar's dPrevSignal still voting - the tester path (m_lastBarTime) already
|
|
//--- advanced only on success and self-healed; this is the live path catching up. On failure the
|
|
//--- gate stays open, so the next tick retries.
|
|
if(refreshed)
|
|
dtStudied = m_Time.GetData(0);
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| |
|
|
//+------------------------------------------------------------------+
|
|
bool CExpertSignalAIBase::RefreshLatestSignal(void)
|
|
{
|
|
//--- Meta target: the live path is the S3 gate (LiveMetaGate), not the per-bar vote - see
|
|
//--- RefreshConvergedSignal's meta guard.
|
|
if(IsMetaTarget())
|
|
return false;
|
|
//--- Bar 1: the newest CLOSED bar, NOT the forming bar. This runs at the first tick after a bar
|
|
//--- opens, when series index 0 is a bar with one tick of data: (close-open)/atr ~ 0, high ~ low,
|
|
//--- a degenerate volume block, indicators computed on a 1-tick candle. Train() never produces
|
|
//--- such a window - every labeled bar is fully closed, and its label assumes entry at that bar's
|
|
//--- CLOSE (see TripleBarrierLabel's header). The training-parity query at this instant (fixed
|
|
//--- 2026-08-11) is therefore the window ending on bar 1, whose close IS the current price - the
|
|
//--- exact instant the label's hypothetical entry happens. The old i = 0 fed the deployed model an
|
|
//--- out-of-distribution final timestep - the timestep the LSTM/HYBRID output is keyed to - and
|
|
//--- semantically asked for the label of a bar whose close was still an hour away, so the deploy
|
|
//--- gate's OOS scores (closed bars, pass 3) measured a different query than live executed. Both
|
|
//--- paths go through BuildFeatureWindow(), which guarantees identical construction; this index is
|
|
//--- what makes them the same QUESTION.
|
|
int i = 1;
|
|
int r = i;
|
|
if(!BuildFeatureWindow(r))
|
|
{
|
|
//--- One combined failure now (partial window OR short total) where there used to be two counters.
|
|
//--- Kept distinct in the tally by testing what actually landed: a window that built every bar but
|
|
//--- came up short is the "short" case, anything else is a feature-build failure.
|
|
if(TempData.Total() > 0 && TempData.Total() < (int)m_historyBars * m_neuronsCount)
|
|
m_refreshFailShort++;
|
|
else
|
|
m_refreshFailFeatures++; // see PrintInferenceTally()
|
|
//--- No opinion this bar rather than a stale one: dPrevSignal still holds the PREVIOUS bar's
|
|
//--- decision, and LongCondition()/ShortCondition() would keep voting that stale direction all
|
|
//--- bar. The caller retries (RefreshConvergedSignal only advances dtStudied on success), so a
|
|
//--- transient failure costs ticks, not the bar.
|
|
dPrevSignal = 0.0;
|
|
return false;
|
|
}
|
|
//--- Live trading/inference reads from the EMA shadow net, not Net directly - see m_shadowNet's
|
|
//--- declaration comment. Falls back to Net if the shadow isn't bootstrapped yet (should only be
|
|
//--- momentarily, on a genuinely fresh start before EnsureShadowNet() has run).
|
|
EnsureShadowNet();
|
|
CNet *deployNet = (CheckPointer(m_shadowNet) != POINTER_INVALID) ? m_shadowNet : Net;
|
|
deployNet.feedForward(TempData);
|
|
deployNet.getResults(TempData);
|
|
if(m_outputNeuronsCount == 1)
|
|
dPrevSignal = TempData[0];
|
|
else
|
|
if(m_outputNeuronsCount == 3)
|
|
{
|
|
//--- Live decision. ApplyClassificationSoftmax() computes the softmax INTO TempData and returns
|
|
//--- the decision; AdjustedSignalFromSoftmax() re-reads that same TempData and applies the same
|
|
//--- strict-majority/ties-to-Neutral rule, so since the read-time prior correction was removed
|
|
//--- (2026-07-31) the two provably agree. The call is kept because a dozen sites name it as
|
|
//--- "the live decision rule" and that is still exactly what it is - the correction now lives
|
|
//--- in the trained weights instead of here.
|
|
//--- The "raw softmax was neutralized by prior correction" diagnostic that used to sit here went
|
|
//--- with it: with nothing between the two values it could never fire again.
|
|
ApplyClassificationSoftmax();
|
|
dPrevSignal = AdjustedSignalFromSoftmax();
|
|
}
|
|
m_refreshOk++;
|
|
//--- bt anchors the DECISION bar (bar 1, the closed bar the window ends on) - it keys the arrow,
|
|
//--- its High/Low placement and NMS declustering, and now matches the rescan path, which draws
|
|
//--- each historical arrow at the bar its window ends on.
|
|
datetime bt = m_Time.GetData(i);
|
|
//--- Keep a pure inference-side watermark of the newest bar FRAME this model has already evaluated.
|
|
//--- This must be the FORMING bar's open time (index 0), not bt: the new-bar gate compares it
|
|
//--- against SERIES_LASTBAR_DATE (also the forming bar's open), so anchoring it at bt (bar 1)
|
|
//--- would compare one bar behind and re-fire the refresh on every tick forever. The tester may
|
|
//--- load dtStudied from a live-chart save whose timestamp is AHEAD of the simulated backtest date
|
|
//--- range; using that training watermark to decide whether a "new bar" exists then freezes
|
|
//--- dPrevSignal at its init-bar value for the whole run. m_lastBarTime is this runtime's own
|
|
//--- latest evaluated frame instead, so it stays aligned to whichever history the current process
|
|
//--- is actually traversing.
|
|
m_lastBarTime = m_Time.GetData(0);
|
|
//--- LIVE NMS, AND IT NOW GATES THE TRADE, NOT JUST THE ARROW.
|
|
//---
|
|
//--- It used to sit at the bottom of this function wrapped around DrawObject() alone, so a suppressed
|
|
//--- bar lost its arrow and still traded: dPrevSignal was never touched, and dPrevSignal is what
|
|
//--- LongCondition()/ShortCondition()/SignedAIConfidence() read. The chart therefore showed roughly one
|
|
//--- arrow per EIGHT positions the EA would open - measured on SP500 H1 2026-08-09, where CONV called a
|
|
//--- direction on 64% of bars while ~40 arrows appeared across the ~500 visible ones. Worse, the arrows
|
|
//--- that survived were not a random eighth: rule 2 below keeps the HIGHER-CONFIDENCE side of a
|
|
//--- cluster, so the visible set was systematically the best member of each run. A chart that shows the
|
|
//--- best of every eight decisions and hides the rest reads far better than the model is, which is the
|
|
//--- same best-of-N selection error this codebase has now corrected in four other places - this time on
|
|
//--- the display layer, where it is most likely to mislead the person deciding whether to trade it.
|
|
//---
|
|
//--- Neutralising dPrevSignal (rather than adding a separate "may trade" flag consulted at each of the
|
|
//--- half-dozen read sites) is deliberate: it leaves exactly ONE definition of what this model decided
|
|
//--- this bar, so the arrow, the panel's "Current signal", the confidence handed to sizing/SL/TP/
|
|
//--- trailing, the refresh tally below and the order itself cannot drift apart again. One arrow is now
|
|
//--- one trade, which is what makes the chart an honest record.
|
|
//---
|
|
//--- NOTE the scoring consequence, deliberately NOT papered over: the era line's dir-precision still
|
|
//--- counts EVERY directional call, so it now describes a larger population than the one that trades.
|
|
//--- The era line carries a separate declustered figure alongside it (see m_oosNmsFired) so both are
|
|
//--- visible; the selection metric is not switched over until those numbers show what the coverage
|
|
//--- floor should be, because a blind switch is how the minRR and recall-floor catch-22s happened.
|
|
ENUM_SIGNAL lsig = DoubleToSignal(dPrevSignal);
|
|
bool nmsAccept = (lsig != Neutral) && NmsLiveAccept(bt, lsig, MathAbs(dPrevSignal));
|
|
if(lsig != Neutral && !nmsAccept)
|
|
dPrevSignal = 0.0; // declustered away: no arrow, no vote, no position
|
|
switch(DoubleToSignal(dPrevSignal))
|
|
{
|
|
case Buy:
|
|
m_refreshBuy++;
|
|
break;
|
|
case Sell:
|
|
m_refreshSell++;
|
|
break;
|
|
default:
|
|
m_refreshNeutral++;
|
|
break;
|
|
}
|
|
if(nmsAccept)
|
|
DrawObject(bt, dPrevSignal, m_Close.GetData(i));
|
|
else
|
|
DeleteObject(bt);
|
|
return true;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| |
|
|
//+------------------------------------------------------------------+
|
|
double CExpertSignalAIBase::ApplyClassificationSoftmax(void)
|
|
{
|
|
// A non-finite logit poisons everything downstream: maxLogit, every exp(), the sum, and all three
|
|
// probabilities become NaN, and since NaN fails every comparison the two directional tests below
|
|
// are both false - so a NaN'd net returns Neutral on every bar forever and looks EXACTLY like a
|
|
// model that has simply gone quiet. That is the failure mode this project has chased repeatedly
|
|
// from the outside (panel says "no directional calls", nobody can tell whether the model is
|
|
// cautious or dead). Detect it here, at the one place the raw logits are first read, and say so.
|
|
if(!MathIsValidNumber(TempData.At(0)) || !MathIsValidNumber(TempData.At(1)) || !MathIsValidNumber(TempData.At(2)))
|
|
{
|
|
static int nanLogitReports = 0;
|
|
// Bounded: this cannot heal on its own (the weights are already corrupt), so unlimited logging
|
|
// would fill the journal for as long as the chart stays attached. Three is enough to prove it.
|
|
if(nanLogitReports < 3)
|
|
{
|
|
nanLogitReports++;
|
|
PrintFormat("%s: %s NON-FINITE network output (%g / %g / %g) - forcing Neutral. The weights are "
|
|
"corrupt; reload the last good .nnw or reset and retrain. Report %d of 3.",
|
|
__FUNCTION__, ID, TempData.At(0), TempData.At(1), TempData.At(2), nanLogitReports);
|
|
}
|
|
return 0;
|
|
}
|
|
// CLASS_LOGIT_SCALE (AI\Network.mqh) must match the training-gradient softmax in
|
|
// backProp/backPropOCL exactly - this is the same normalization the loss was trained against.
|
|
double maxLogit = CLASS_LOGIT_SCALE * MathMax(TempData.At(0), MathMax(TempData.At(1), TempData.At(2)));
|
|
double sum = 0;
|
|
for(int res = 0; res < 3; res++)
|
|
{
|
|
double temp = exp(CLASS_LOGIT_SCALE * TempData.At(res) - maxLogit);
|
|
sum += temp;
|
|
TempData.Update(res, temp);
|
|
}
|
|
for(int res = 0; res < 3; res++)
|
|
TempData.Update(res, TempData.At(res) / sum);
|
|
double pBuy = TempData.At(0);
|
|
double pSell = TempData.At(1);
|
|
double pNeutral = TempData.At(2);
|
|
// TempData.Maximum(0,3) scans left-to-right and keeps the FIRST index on a tie, so any tie
|
|
// (including the degenerate all-equal 0.3333/0.3333/0.3333 case from a collapsed/untrained net)
|
|
// always resolved to Buy (index 0) - silently turning "the model has no idea" into a directional
|
|
// trade. Buy/Sell now only win with a strict majority over BOTH other classes; every tie,
|
|
// 2-way or 3-way, falls through to Neutral.
|
|
if(pBuy > pSell && pBuy > pNeutral)
|
|
return pBuy; // Buy signal
|
|
if(pSell > pBuy && pSell > pNeutral)
|
|
return -pSell; // Sell signal
|
|
return 0; // Neutral signal (also the fallback on any tie)
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Post-hoc logit adjustment (prior correction) of the 3-class |
|
|
//| decision. Reads the raw softmax probabilities ApplyClassification|
|
|
//| Softmax() left in TempData[0..2] and returns the prior-corrected |
|
|
//| signed decision (+P'(buy)/-P'(sell)/0-neutral), the exact rule |
|
|
//| live trading fires on and the live-fired precision metric scores. |
|
|
//| |
|
|
//| RAW ARGMAX, deliberately. The prior correction this function used |
|
|
//| to apply at read time (Saerens et al. 2002) was REMOVED |
|
|
//| 2026-07-31 along with the AILogitPriorStrength input. |
|
|
//| |
|
|
//| Why there is nothing to correct here: the logit-adjusted loss |
|
|
//| adds tau*log(prior_c) to each class logit inside the TRAINING |
|
|
//| gradient, so the network learns to absorb the offset and its raw |
|
|
//| argmax is ALREADY the balanced-error-optimal decision. Applying a |
|
|
//| second correction at inference would account for the same base |
|
|
//| rate twice and push the decision back toward Neutral - undoing |
|
|
//| exactly what the loss bought. The old code knew this: the whole |
|
|
//| adjustment sat behind an `if(m_useLogitAdjustedLoss) return raw` |
|
|
//| guard and had been unreachable for the entire shipped default |
|
|
//| configuration. Kept as a named function rather than inlined |
|
|
//| because a dozen call sites document themselves by calling "the |
|
|
//| live decision rule" - and that is exactly what this is. |
|
|
//+------------------------------------------------------------------+
|
|
double CExpertSignalAIBase::AdjustedSignalFromSoftmax(void)
|
|
{
|
|
if(TempData.Total() < 3)
|
|
return 0.0;
|
|
double pBuy = TempData.At(0), pSell = TempData.At(1), pNeutral = TempData.At(2);
|
|
//--- Strict majority, ties to Neutral - the same rule as ApplyClassificationSoftmax(). The returned
|
|
//--- magnitude is a genuine probability, which the confidence floor and ConfidenceTier() read.
|
|
bool wantBuy = (pBuy > pSell && pBuy > pNeutral);
|
|
bool wantSell = (pSell > pBuy && pSell > pNeutral);
|
|
if(!wantBuy && !wantSell)
|
|
return 0.0;
|
|
//--- OPERATING POINT (2026-08-09). Argmax alone answers "which class is most likely"; it does not
|
|
//--- answer "is this worth trading", and those are different questions whenever the top two classes
|
|
//--- are nearly tied. A marginal directional win over Neutral used to become a trade, which is the
|
|
//--- mechanical source of the model calling a direction on ~90% of bars. Below the fitted margin
|
|
//--- this abstains instead - and abstaining is not a loss of information, it is the model declining
|
|
//--- to act on a distinction it cannot make. See DIR_CONF_THRESHOLD_BINS for how the value is chosen.
|
|
//---
|
|
//--- Returning Neutral rather than exposing a separate "tradeable" flag is deliberate, and matches
|
|
//--- the same decision made for live NMS (see RefreshLatestSignal): one definition of what this model
|
|
//--- decided this bar, so the arrow, the panel, the confidence handed to sizing/SL/TP, the OOS score
|
|
//--- and the order itself cannot drift apart.
|
|
if(m_dirConfThreshold > 0.0)
|
|
{
|
|
double win = wantBuy ? pBuy : pSell;
|
|
double rival = wantBuy ? MathMax(pSell, pNeutral) : MathMax(pBuy, pNeutral);
|
|
if((win - rival) < m_dirConfThreshold)
|
|
return 0.0;
|
|
}
|
|
return wantBuy ? pBuy : -pSell;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| The statistic the operating point is expressed in - see the |
|
|
//| declaration. Reads the softmax ALREADY in TempData, so callers |
|
|
//| must have run ApplyClassificationSoftmax() first. |
|
|
//+------------------------------------------------------------------+
|
|
double CExpertSignalAIBase::DirectionalMargin(void)
|
|
{
|
|
if(TempData.Total() < 3)
|
|
return -1.0;
|
|
double pBuy = TempData.At(0), pSell = TempData.At(1), pNeutral = TempData.At(2);
|
|
if(pBuy > pSell && pBuy > pNeutral)
|
|
return pBuy - MathMax(pSell, pNeutral);
|
|
if(pSell > pBuy && pSell > pNeutral)
|
|
return pSell - MathMax(pBuy, pNeutral);
|
|
return -1.0; // Neutral won: no directional call, so no operating point applies
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Clear the margin histogram at the start of the calibration walk. |
|
|
//+------------------------------------------------------------------+
|
|
void CExpertSignalAIBase::ResetDirConfHistogram(void)
|
|
{
|
|
ArrayInitialize(m_dirConfBinCalls, 0);
|
|
ArrayInitialize(m_dirConfBinHits, 0);
|
|
m_dirConfPrimaryBars = 0;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| One calibration sample. isPrimaryBar survives from when this was |
|
|
//| harvested inside pass 2's oversampled replay queue, where counting |
|
|
//| duplicated minority bars would have fitted the operating point to |
|
|
//| a class balance the live model never sees (the same correction |
|
|
//| m_cumIsTotal makes - see its note in Training.mqh). The calibration |
|
|
//| walk visits each bar exactly once and passes true; the parameter |
|
|
//| stays so any future caller must state which it is. |
|
|
//+------------------------------------------------------------------+
|
|
void CExpertSignalAIBase::AccumulateDirConfSample(double margin, bool wasCorrect, bool isPrimaryBar)
|
|
{
|
|
if(!isPrimaryBar)
|
|
return;
|
|
//--- Counted BEFORE the directional test: this is the coverage denominator, so it has to be every
|
|
//--- primary bar the model scored, including the ones it called Neutral. Using only directional
|
|
//--- bars would make coverage 100% by construction at every threshold.
|
|
m_dirConfPrimaryBars++;
|
|
if(margin < 0.0)
|
|
return; // Neutral won - not a directional call
|
|
int bin = (int)(margin * DIR_CONF_THRESHOLD_BINS);
|
|
if(bin < 0)
|
|
bin = 0;
|
|
if(bin >= DIR_CONF_THRESHOLD_BINS)
|
|
bin = DIR_CONF_THRESHOLD_BINS - 1; // margin can reach exactly 1.0
|
|
m_dirConfBinCalls[bin]++;
|
|
if(wasCorrect)
|
|
m_dirConfBinHits[bin]++;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Choose the operating point: the margin that maximises EXPECTANCY |
|
|
//| on the held-out calibration slice while still calling a direction |
|
|
//| often enough to clear the SAME coverage floor the deploy gate |
|
|
//| uses. Held-out matters as much as the objective does - see |
|
|
//| DIR_CONF_CALIB_PCT_OF_IS for what fitting it on the training |
|
|
//| bars did to the sign of (p - break-even). |
|
|
//| |
|
|
//| Swept from the top down so the running totals are "calls at or |
|
|
//| above this bin", which is exactly the set a threshold there would |
|
|
//| admit - one pass, no nested loop over candidate thresholds. |
|
|
//| |
|
|
//| TIES GO TO THE LOWER THRESHOLD. Precision is a ratio of counts |
|
|
//| and plateaus over ranges of margin; taking the highest threshold |
|
|
//| on a plateau would buy identical precision for strictly less |
|
|
//| coverage, and coverage is what keeps the model tradeable. |
|
|
//+------------------------------------------------------------------+
|
|
void CExpertSignalAIBase::FitDirConfThreshold(void)
|
|
{
|
|
long totalCalls = 0;
|
|
for(int b = 0; b < DIR_CONF_THRESHOLD_BINS; b++)
|
|
totalCalls += m_dirConfBinCalls[b];
|
|
if(totalCalls < DIR_CONF_MIN_FIT_CALLS || m_dirConfPrimaryBars <= 0)
|
|
{
|
|
//--- Not enough evidence to place an operating point. KEEP THE PREVIOUS ONE - the old behaviour
|
|
//--- here was to reset to 0.0, which is "call a direction on every bar", the single most exposed
|
|
//--- setting in the range. A failed measurement must never decay to the most aggressive value it
|
|
//--- could have returned; the last threshold that WAS fitted is a strictly better estimate than
|
|
//--- the one setting we know maximises exposure. At era 0 the previous value is 0.0 regardless,
|
|
//--- so the cold-start path is unchanged.
|
|
if(!m_dirConfSparseWarned)
|
|
{
|
|
m_dirConfSparseWarned = true;
|
|
Print(ID + StringFormat(": directional confidence threshold NOT refitted - only %d directional "
|
|
"calls in the held-out calibration slice this era (need %d). Keeping "
|
|
"the previous operating point %.2f; this is normal for the first eras "
|
|
"and self-corrects as the model starts calling directions.",
|
|
(int)totalCalls, DIR_CONF_MIN_FIT_CALLS, m_dirConfThreshold));
|
|
}
|
|
return;
|
|
}
|
|
//--- The floor is the true directional base rate x MIN_COVERAGE_FRACTION_OF_BASE_RATE, matching
|
|
//--- Train()'s minCoveragePct exactly. Derived from THIS era's own IS labels rather than passed in,
|
|
//--- so the two cannot fall out of step when one of them is edited.
|
|
long trueDir = m_trueBuyCount + m_trueSellCount;
|
|
long trueTot = trueDir + m_trueNeutralCount;
|
|
double baseRatePct = (trueTot > 0) ? 100.0 * (double)trueDir / trueTot : 0.0;
|
|
double minCoveragePct = baseRatePct * MIN_COVERAGE_FRACTION_OF_BASE_RATE;
|
|
//--- EXPECTANCY, NOT PRECISION. Maximising the win rate alone has no answer for a PLATEAU, and the
|
|
//--- previous `precPct >= bestPrec` resolved one by walking to ever more coverage. That is a
|
|
//--- catastrophe on exactly the models that need a threshold most: a net with no edge scores its base
|
|
//--- rate at EVERY threshold, which is a perfect plateau, so the walk ran to bin 0 and returned
|
|
//--- threshold 0.0 - fire on every bar. Observed 2026-08-10 as PAI "overshooting signals" while the
|
|
//--- other three stayed selective; PAI has the most degenerate margin distribution (OOS outputs
|
|
//--- spanning the full 0.000..1.000 where CONV sits at 0.214..0.814), so its plateau is the flattest.
|
|
//---
|
|
//--- The money quantity is expectancy per BAR, and for a k:m barrier
|
|
//--- EV = (p - p0) * (k + m) with p0 = m/(m+k),
|
|
//--- so EV per bar = coverage * (p - p0) * (k + m). (k+m) is constant across thresholds, which
|
|
//--- leaves coverage * (p - p0) as the objective. It behaves correctly in all three regimes and
|
|
//--- needs no tie-break rule:
|
|
//--- p > p0 everywhere -> more coverage is more money, so it takes the coverage (the old
|
|
//--- behaviour, but for a reason rather than as a plateau artifact)
|
|
//--- p flat AT p0 -> every point scores 0 and the floor decides; no runaway
|
|
//--- p < p0 everywhere -> the LEAST coverage loses the least, so it becomes MORE selective
|
|
//--- instead of trading everything, which is the current reality for all
|
|
//--- four models and the opposite of what the old rule did.
|
|
//--- COST-ADJUSTED, 2026-08-17. This is the reference the operating-point objective subtracts, so
|
|
//--- using the frictionless SL/(SL+TP) here made every candidate threshold look better than it was by
|
|
//--- the width of the spread - on SP500 H4 that was 2.2pp against a measured edge of 2.3pp, i.e. very
|
|
//--- nearly all of it. See CostAdjustedBreakEvenPct.
|
|
double breakEvenPct = CostAdjustedBreakEvenPct();
|
|
//--- Per-bin curve, cached so the second pass does not re-accumulate. Values are the running
|
|
//--- "at or above this bin" totals, which is exactly the population a threshold there admits.
|
|
double binCov[DIR_CONF_THRESHOLD_BINS];
|
|
double binPrec[DIR_CONF_THRESHOLD_BINS];
|
|
double binScore[DIR_CONF_THRESHOLD_BINS];
|
|
double binSe[DIR_CONF_THRESHOLD_BINS];
|
|
bool binOk[DIR_CONF_THRESHOLD_BINS];
|
|
long runCalls = 0, runHits = 0;
|
|
double bestScore = -DBL_MAX, bestSe = 0.0;
|
|
int bestBin = -1, floorBin = -1, eligibleBins = 0;
|
|
for(int b = DIR_CONF_THRESHOLD_BINS - 1; b >= 0; b--)
|
|
{
|
|
binOk[b] = false;
|
|
binCov[b] = 0.0;
|
|
binPrec[b] = 0.0;
|
|
binScore[b] = 0.0;
|
|
binSe[b] = 0.0;
|
|
runCalls += m_dirConfBinCalls[b];
|
|
runHits += m_dirConfBinHits[b];
|
|
if(runCalls <= 0)
|
|
continue;
|
|
double coveragePct = 100.0 * (double)runCalls / m_dirConfPrimaryBars;
|
|
if(coveragePct < minCoveragePct)
|
|
continue; // too selective to be deployable
|
|
double precPct = 100.0 * (double)runHits / runCalls;
|
|
double p = precPct / 100.0;
|
|
binOk[b] = true;
|
|
binCov[b] = coveragePct;
|
|
binPrec[b] = precPct;
|
|
binScore[b] = coveragePct * (precPct - breakEvenPct);
|
|
//--- Binomial standard error of the win rate at this operating point, carried into the score's
|
|
//--- own units. Coverage is measured against a FIXED denominator every bin, so it is far better
|
|
//--- determined than the win rate; the score's error is dominated by the precision term.
|
|
//--- ON THE EFFECTIVE SAMPLE, not the raw call count (2026-08-17). These calls are triple-barrier
|
|
//--- outcomes on consecutive bars, so they overlap: at a 384-bar horizon, neighbouring labels share
|
|
//--- almost their entire outcome window and are nothing like independent draws, and runCalls
|
|
//--- understates the error by up to ~sqrt(mean lifespan).
|
|
//--- NOT because the gate below was observed to misfire - measured over the full 6,930-era run it
|
|
//--- fires on 1.5-8.2% of eras, at or under the ~5% a family-wise test should. See
|
|
//--- EffectiveSampleSize(); this is a formula correction, not a bug fix, and it makes the bar
|
|
//--- higher rather than lower.
|
|
binSe[b] = coveragePct * 100.0 * MathSqrt(MathMax(p * (1.0 - p), 0.0)
|
|
/ EffectiveSampleSize((double)runCalls));
|
|
//--- The sweep runs top-down, so the FIRST eligible bin is the most selective one that still
|
|
//--- clears the coverage floor. That point is the deterministic fallback below.
|
|
if(floorBin < 0)
|
|
floorBin = b;
|
|
eligibleBins++;
|
|
//--- Strict >, so a genuine tie keeps the MORE selective point (the loop reaches it first). The
|
|
//--- old >= did the reverse and that is what made the plateau run away.
|
|
if(binScore[b] > bestScore)
|
|
{
|
|
bestScore = binScore[b];
|
|
bestSe = binSe[b];
|
|
bestBin = b;
|
|
}
|
|
}
|
|
if(bestBin < 0)
|
|
{
|
|
//--- Even calling on every directional argmax does not reach the coverage floor, so there is no
|
|
//--- room to be MORE selective. Unthresholded is then the only setting that can clear the gate.
|
|
m_dirConfThreshold = 0.0;
|
|
return;
|
|
}
|
|
//--- SELECTION UNDER A NULL OF THE MAXIMUM, with a parsimony fallback in the spirit of the
|
|
//--- one-standard-error rule (Breiman et al. 1984, CART 3.4.3; Hastie/Tibshirani/Friedman, ESL 2ed
|
|
//--- 7.10 - prefer the simpler model when the score difference is inside the noise). The bare
|
|
//--- argmax above is the right ANSWER only if the curve it maximises is measured well enough to
|
|
//--- rank its own candidates, and on this data it is not. Measured over 98 consecutive fits of the
|
|
//--- shipped SP500 H4 model:
|
|
//---
|
|
//--- correlation(chosen threshold, win rate at it) = -0.056 over the full 0.00..0.74 range
|
|
//--- win rate stdev across fits = 1.32pp
|
|
//--- binomial SE of that win rate at ~1430 calls = 1.25pp
|
|
//---
|
|
//--- The correlation is zero - the margin does not rank trades at all - and the era-to-era spread
|
|
//--- IS its own sampling error to within 0.07pp. So `coverage x (precision - breakEven)` was
|
|
//--- `coverage x (3.4 +/- 1.3)`, and taking the argmax over ~37 eligible bins returned whichever
|
|
//--- bin drew the luckiest sample. The threshold then teleported 0.42 -> 0.04 -> 0.74 in three
|
|
//--- eras, swinging OOS coverage 0% -> 39%, which left the era win rate measured on 1-5 calls and
|
|
//--- swinging 0% <-> 100%. That is the whole of the "training is highly unstable" report, and none
|
|
//--- of it was the optimizer.
|
|
//---
|
|
//--- This is the same defect the family-wise gate rule already governs elsewhere in this file - a
|
|
//--- best-of-N adopted without a null of the maximum - applied here to the operating point rather
|
|
//--- than the deploy decision.
|
|
//---
|
|
//--- THE RULE. The argmax is adopted only if it beats the DETERMINISTIC fallback by more than a
|
|
//--- best-of-N maximum could manage on noise alone; otherwise the fallback is taken.
|
|
//---
|
|
//--- Fallback = the most selective bin that still clears the coverage floor. That point is chosen
|
|
//--- from the MARGIN DISTRIBUTION only - it never consults a win rate - so it carries none of the
|
|
//--- outcome noise that was driving the thrash, and it moves era to era only when the model's own
|
|
//--- confidence distribution genuinely moves. It is also the conservative end of the sweep: the
|
|
//--- fewest bars called that still leaves a deployable model, which is the right default on a
|
|
//--- funded account when no operating point has been shown to be better than another.
|
|
//---
|
|
//--- A plain one-standard-error band was the first thing tried here and it is NOT sufficient: the
|
|
//--- band edge is bestScore - bestSE, and with an edge of 2.3pp against a 1.25pp standard error
|
|
//--- bestScore is itself +/-50%, so the admitted set - and the coverage it implies - would still
|
|
//--- wander by half its own width every era. The fallback has to be independent of the noisy
|
|
//--- quantity, not merely a wider window around it.
|
|
//---
|
|
//--- Significance uses the null of the MAXIMUM, not a per-candidate test: the argmax is the best of
|
|
//--- `eligibleBins` draws, and the expected maximum of N standard normals grows like sqrt(2 ln N),
|
|
//--- so that is the bar it has to clear. Same correction the deploy gate already applies to
|
|
//--- best-of-N model selection, applied here to the operating point.
|
|
double refScore = binScore[floorBin];
|
|
double refSe = binSe[floorBin];
|
|
//--- Conservative: the two points are NESTED samples, so their difference is better determined than
|
|
//--- this independent-errors sum implies. Erring toward "not significant" is the safe direction.
|
|
double seDiff = MathSqrt(bestSe * bestSe + refSe * refSe);
|
|
double zMax = MathSqrt(2.0 * MathLog(MathMax((double)eligibleBins, 2.0)));
|
|
bool separates = ((bestScore - refScore) > zMax * seDiff);
|
|
int selBin = (separates ? bestBin : floorBin);
|
|
double bestThresh = (double)selBin / DIR_CONF_THRESHOLD_BINS;
|
|
double bestPrec = binPrec[selBin];
|
|
double bestCov = binCov[selBin];
|
|
double prevThresh = m_dirConfThreshold;
|
|
m_dirConfThreshold = bestThresh;
|
|
//--- Built as a local rather than inlined into the ternary: the two branches are long enough that
|
|
//--- keeping them out of the argument list is what makes the call readable.
|
|
string ruleNote = "which CLEARS that bar, so the margin genuinely separates these operating points"
|
|
" and the argmax was adopted";
|
|
if(!separates)
|
|
ruleNote = "which it does NOT clear - the margin does not rank these trades, so the operating"
|
|
" point fell back to the most selective bin that still clears the coverage floor."
|
|
" That fallback reads the margin DISTRIBUTION only, never a win rate, so it cannot"
|
|
" thrash on outcome noise the way the argmax did";
|
|
//--- Logged when it moves a bin AND the era cadence is due (2026-08-19). "Stays quiet when stable"
|
|
//--- had stopped being a filter: the operating point is measured noise-dominated (project memory:
|
|
//--- "ratchet, then noise"), so it moved a bin nearly every era - ~400 prints per member per day.
|
|
//--- The threshold itself keeps updating every era regardless; only the announcement is throttled.
|
|
if(TrainLogDue() && MathAbs(m_dirConfThreshold - prevThresh) >= 1.0 / DIR_CONF_THRESHOLD_BINS)
|
|
Print(ID + StringFormat(": directional confidence threshold %.2f -> %.2f (fitted on %d HELD-OUT "
|
|
"calibration bars: %.1f%% coverage at %.1f%% WIN RATE vs " + DoubleToString(breakEvenPct, 1) +
|
|
"%% break-even, edge " + DoubleToString(bestPrec - breakEvenPct, 1) +
|
|
"pp, coverage floor %.1f%%). Below "
|
|
"this winner-vs-rival margin the model abstains instead of trading. The "
|
|
"rate is wins - target before stop on the side actually called - not "
|
|
"agreement with the collapsed 3-class label; see m_oosBuyPredictedWins."
|
|
" | best-of-N gate over %d eligible bins: argmax %.2f scored %.0f vs the"
|
|
" %.2f fallback's %.0f, a gap of %.0f against a null-of-the-maximum bar"
|
|
" of %.0f (z_max %.2f x SE %.0f, sized on the EFFECTIVE sample - labels"
|
|
" overlap by a mean lifespan of %.0f bars, so n is deflated %.0fx), %s",
|
|
prevThresh, m_dirConfThreshold, (int)m_dirConfPrimaryBars, bestCov,
|
|
bestPrec, minCoveragePct, eligibleBins,
|
|
(double)bestBin / DIR_CONF_THRESHOLD_BINS, bestScore,
|
|
(double)floorBin / DIR_CONF_THRESHOLD_BINS, refScore,
|
|
bestScore - refScore, zMax * seDiff, zMax, seDiff,
|
|
MeanLabelLifespan(), MeanLabelLifespan(), ruleNote));
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| EMA-updates the persisted true class base rates from a finished |
|
|
//| era's true class counts. First real measurement seeds directly; |
|
|
//| thereafter blended with the same smoothing as the accuracy/ |
|
|
//| confidence EMAs so one noisy era can't swing the live decision. |
|
|
//+------------------------------------------------------------------+
|
|
void CExpertSignalAIBase::UpdateClassPriors(long buyCnt, long sellCnt, long neutralCnt)
|
|
{
|
|
long tot = buyCnt + sellCnt + neutralCnt;
|
|
if(tot <= 0)
|
|
return;
|
|
double pb = (double)buyCnt / tot, ps = (double)sellCnt / tot, pn = (double)neutralCnt / tot;
|
|
if(m_priorNeutral <= 0.0) // first real measurement
|
|
{
|
|
m_priorBuy = pb;
|
|
m_priorSell = ps;
|
|
m_priorNeutral = pn;
|
|
return;
|
|
}
|
|
//--- (Was `m_useStaticPrior || m_freezePriorCalibration`. Those were two separate user-facing inputs
|
|
//--- whose only effect anywhere in the codebase was this one OR - two controls for one decision.
|
|
//--- UseStaticPrior was removed 2026-07-31; see the class-imbalance audit in Variables\Inputs.mqh.)
|
|
if(m_freezePriorCalibration)
|
|
return;
|
|
double k = Net.recentAverageSmoothingFactor;
|
|
if(k < 1.0)
|
|
k = 1.0;
|
|
m_priorBuy += (pb - m_priorBuy) / k;
|
|
m_priorSell += (ps - m_priorSell) / k;
|
|
m_priorNeutral += (pn - m_priorNeutral) / k;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Installs the training-time logit offsets - see the declaration. |
|
|
//+------------------------------------------------------------------+
|
|
void CExpertSignalAIBase::ApplyLogitAdjustment(void)
|
|
{
|
|
if(CheckPointer(Net) == POINTER_INVALID)
|
|
return;
|
|
if(m_logitAdjustTau <= 0.0)
|
|
{
|
|
//--- Clear rather than merely skip: the input can be turned off on a chart that already installed
|
|
//--- offsets this session, and a stale adjustment would keep biasing the gradient silently.
|
|
Net.ClearLogitAdjustment();
|
|
return;
|
|
}
|
|
//--- Priors not measured yet (era 0 before the first tally, or a model with no .stats): leave the
|
|
//--- gradient unadjusted rather than guessing a distribution. The next era installs them.
|
|
//--- THIS USED TO BE SILENT, and that silence hid a whole-run failure: while the auto-tune search ran,
|
|
//--- UpdateClassPriors() was skipped in eval mode, so this branch was taken on EVERY era and the
|
|
//--- imbalance correction never once ran - with nothing in the log to say so. A mechanism that
|
|
//--- declines to act must announce it; the alternative is indistinguishable from working. Third time
|
|
//--- this codebase has been bitten by a quiet no-op, so it now warns every time it is not merely the
|
|
//--- expected era-0 case.
|
|
if(m_priorBuy <= 0.0 || m_priorSell <= 0.0 || m_priorNeutral <= 0.0)
|
|
{
|
|
if(m_eraCount > 0 && !m_logitAdjustSkipWarned)
|
|
{
|
|
m_logitAdjustSkipWarned = true;
|
|
Print(ID + ": WARNING - class-imbalance correction is NOT running at era " +
|
|
IntegerToString(m_eraCount) + ": the class priors have never been measured (Buy " +
|
|
DoubleToString(m_priorBuy, 4) + " Sell " + DoubleToString(m_priorSell, 4) + " Neutral " +
|
|
DoubleToString(m_priorNeutral, 4) + "). Training is falling back to plain cross-entropy, "
|
|
"which on a skewed label set collapses to the majority class.");
|
|
}
|
|
Net.ClearLogitAdjustment();
|
|
return;
|
|
}
|
|
//--- Effective tau, capped so the offsets cannot swamp the head's usable logit range - see
|
|
//--- LOGIT_ADJUST_MAX_RANGE_FRACTION. The binding quantity is the SPREAD between the largest and
|
|
//--- smallest offset, not their absolute size: softmax is shift-invariant, so a constant added to
|
|
//--- all three classes changes nothing and only their differences move the decision.
|
|
double lb = MathLog(m_priorBuy), ls = MathLog(m_priorSell), lnn = MathLog(m_priorNeutral);
|
|
//--- THE CORRECTION SPANS THE DECIDABLE CLASSES ONLY - Buy against Sell. Neutral is excluded, and
|
|
//--- that exclusion is the whole point of this block (2026-08-16).
|
|
//---
|
|
//--- Logit adjustment (Menon et al. 2020) makes the classifier Bayes-optimal for BALANCED error by
|
|
//--- subsidising rare classes. It was wired here when Neutral was the DOMINANT class - the era of
|
|
//--- "big move up / big move down / nothing much", where the majority outcome was no move and the
|
|
//--- correction pulled the model off it. The triple-barrier relabel (b4a704d) inverted that: the
|
|
//--- barriers are now the EA's own SL/TP, so ~89% of bars RESOLVE and only the timeouts are Neutral.
|
|
//--- Measured on SP500 H4: Buy 48.26% Sell 41.13% Neutral 10.61%. Neutral became the RAREST class,
|
|
//--- and the correction dutifully started subsidising it - by tau*(log pB - log pN) = 1.20 logits at
|
|
//--- the capped tau of 0.79. With no directional edge to overcome that (direction is closed at
|
|
//--- best-of-999, p=1.0000), the model took the free lunch: OOS recall Buy:1% Sell:0% Neutral:100%,
|
|
//--- softmax saturated at spread 0.9993, and the first-layer weight block froze at 0.000% dW/W while
|
|
//--- the head kept twitching. The anti-collapse mechanism WAS the collapse.
|
|
//---
|
|
//--- Neutral is not a class worth predicting here - it is the ABSTAIN outcome, and abstention is
|
|
//--- already owned by a better mechanism: m_dirConfThreshold, refitted every era on the held-out
|
|
//--- calibration band against a coverage floor and the measured break-even. Subsidising the abstain
|
|
//--- class does the same job twice and spends the entire correction suppressing the only decisions
|
|
//--- that can make money. What DOES deserve correcting is Buy vs Sell: a trending symbol resolves
|
|
//--- more long barriers than short ones, and left uncorrected the model inherits that drift as a
|
|
//--- standing directional bias. Here that is log(0.4826) - log(0.4113) = 0.16, so the offsets are
|
|
//--- tiny - which is the correct answer, not a broken one. The two classes were already balanced;
|
|
//--- all the old spread of 1.52 ever described was how rare a timeout is.
|
|
//---
|
|
//--- Centred on the midpoint of the two so the pair is corrected against EACH OTHER and Neutral sits
|
|
//--- at zero. Softmax is shift-invariant, so only the differences matter: Neutral now sits within
|
|
//--- tau*0.08 of both trading classes instead of 1.20 above them.
|
|
double mid = 0.5 * (lb + ls);
|
|
double spread = MathAbs(lb - ls);
|
|
double tauEff = m_logitAdjustTau;
|
|
if(spread > 0.0)
|
|
{
|
|
double cap = LOGIT_ADJUST_MAX_RANGE_FRACTION * CLASS_LOGIT_SCALE / spread;
|
|
if(tauEff > cap)
|
|
tauEff = cap;
|
|
}
|
|
if(!m_logitAdjustLogged)
|
|
{
|
|
m_logitAdjustLogged = true;
|
|
//--- Reports BOTH spreads on purpose. The Buy-vs-Sell one is what is actually applied; the
|
|
//--- all-three one is what the old code applied, and printing them side by side is what makes it
|
|
//--- visible when a label set has drifted so far that the abstain class is the rare one.
|
|
Print(ID + ": logit adjustment - measured priors Buy " + DoubleToString(m_priorBuy * 100.0, 2) +
|
|
"% Sell " + DoubleToString(m_priorSell * 100.0, 2) + "% Neutral " +
|
|
DoubleToString(m_priorNeutral * 100.0, 2) + "% | APPLIED across Buy/Sell only, log-prior"
|
|
" spread " + DoubleToString(spread, 2) + " (all three would be " +
|
|
DoubleToString(MathMax(lb, MathMax(ls, lnn)) - MathMin(lb, MathMin(ls, lnn)), 2) +
|
|
"; Neutral is the ABSTAIN outcome and is never subsidised - m_dirConfThreshold owns"
|
|
" abstention)" +
|
|
(m_priorNeutral < m_priorBuy && m_priorNeutral < m_priorSell
|
|
? " | note: Neutral is the RAREST class here, so the pre-2026-08-16 all-three form would"
|
|
" have BOOSTED it by " +
|
|
DoubleToString(tauEff * (MathMax(lb, ls) - lnn), 2) + " logits"
|
|
: "") +
|
|
" against a logit range of " +
|
|
DoubleToString(CLASS_LOGIT_SCALE, 1) + " | tau " + DoubleToString(m_logitAdjustTau, 2) +
|
|
(tauEff < m_logitAdjustTau
|
|
? " CAPPED to " + DoubleToString(tauEff, 2) + " (uncapped it would consume " +
|
|
DoubleToString(100.0 * spread * m_logitAdjustTau / CLASS_LOGIT_SCALE, 0) +
|
|
"% of the range and saturate the head)"
|
|
: " (uncapped - within budget)"));
|
|
}
|
|
//--- ORDERED to match the output layer: [0]=Buy, [1]=Sell, [2]=Neutral - the order
|
|
//--- BuildFreshTopology emits and the order the softmax gradient reads (AI\Network.mqh).
|
|
double offsets[3];
|
|
offsets[0] = tauEff * (lb - mid);
|
|
offsets[1] = tauEff * (ls - mid);
|
|
offsets[2] = 0.0; // ABSTAIN class - never subsidised; see the block above
|
|
Net.SetLogitAdjustment(offsets);
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Converts a double to ENUM_SIGNAL. |
|
|
//| 3-output (softmax classification) case: dPrevSignal's *sign* |
|
|
//| alone already encodes the argmax-selected class (+prob for Buy, |
|
|
//| -prob for Sell, exactly 0.0 for Neutral - see Train()/ |
|
|
//| RefreshLatestSignal()), so classification here is pure argmax: |
|
|
//| whichever class the network actually picked, full stop. No |
|
|
//| magnitude threshold is applied - confidence magnitude is a |
|
|
//| separate concern, already exposed via AIConfidence()/ |
|
|
//| SignedAIConfidence() (MathAbs(dPrevSignal)/dPrevSignal) for the |
|
|
//| signal engine's own confidence-weighted filters/lot sizing/SLTP, |
|
|
//| so this keeps "which class" and "how confident" decoupled. |
|
|
//| 1-output (tanh regression) case: unrelated network shape, keeps |
|
|
//| the original 0.50 magnitude cutoff as a genuine confidence gate. |
|
|
//+------------------------------------------------------------------+
|
|
ENUM_SIGNAL CExpertSignalAIBase::DoubleToSignal(double value)
|
|
{
|
|
value = NormalizeDouble(value, 2); // Round 'value' to two decimal places
|
|
if(value < -1.0 || value > 1.0)
|
|
return Undefine; // out of range, e.g. the -2 "not yet studied" sentinel
|
|
if(m_outputNeuronsCount == 3)
|
|
{
|
|
if(value > 0.0)
|
|
return Buy;
|
|
if(value < 0.0)
|
|
return Sell;
|
|
return Neutral;
|
|
}
|
|
if(value > 0.50)
|
|
return Buy;
|
|
if(value < -0.50)
|
|
return Sell;
|
|
return Neutral;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Throttled, SIDE-EFFECT-FREE forward of the current decision bar, |
|
|
//| for display only (the HUD member lines and the prospective vote). |
|
|
//| |
|
|
//| The reference library kept its training label honest by simply |
|
|
//| printing the last training sample's outputs - but a shuffled pass-2|
|
|
//| sample is a random historical bar, and what the user tracks is the|
|
|
//| model's opinion of NOW under the weights of NOW. So this asks the |
|
|
//| exact question the live path asks (the window ending on bar 1, the|
|
|
//| newest CLOSED bar - see RefreshLatestSignal for why not bar 0) and |
|
|
//| touches NOTHING the trading or training paths read: |
|
|
//| - dPrevSignal, the NMS state, the refresh tallies, dtStudied and |
|
|
//| m_lastBarTime all stay untouched - RefreshLatestSignal is NOT |
|
|
//| reusable here precisely because it writes all of them; |
|
|
//| - batch-norm running statistics are bracketed frozen/restored |
|
|
//| (GetBatchNormFrozen), because an unfrozen forward ADVANCES them|
|
|
//| - hundreds of display reads per era would otherwise retrain the|
|
|
//| normalization on one bar's window; restore-not-unfreeze because|
|
|
//| pass 3 holds them frozen across its whole scan and a display |
|
|
//| tick landing between its chunks must not unfreeze mid-scan; |
|
|
//| - the LSTM is safe by construction: h_{-1}/c_{-1} are zeroed per |
|
|
//| forward (see AI\Impl\NeuronOCLLSTM.mqh), nothing leaks between |
|
|
//| samples; |
|
|
//| - TempData is the shared scratch every consumer rebuilds before |
|
|
//| use, and this builds/forwards/reads it atomically. |
|
|
//| |
|
|
//| It forwards Net - the LEARNER - not the shadow: the shadow is what|
|
|
//| trades, but EnsureShadowNet()'s first call clones a full net |
|
|
//| through a temp file, a side effect a display routine must never |
|
|
//| trigger, and during training (the whole use case) the shadow lags |
|
|
//| the learner by construction. Post-convergence Net holds the |
|
|
//| converged weights and online learning keeps updating it, so the |
|
|
//| line stays honest there too. |
|
|
//| |
|
|
//| Throttle: a real forward at most every DISPLAY_FWD_MIN_MS, or |
|
|
//| DISPLAY_FWD_ERA_MS after an era boundary (weights AND tier money |
|
|
//| just moved, the cached read is priced in a dead regime). Between |
|
|
//| refreshes the cached m_dispProbs/m_dispSignal serve every caller, |
|
|
//| so the 500ms timer costs nothing extra. Failures keep the last |
|
|
//| good read on display (stale-by-seconds beats blank) but stamp the |
|
|
//| attempt, so a broken window retries at throttle pace, not 2/sec. |
|
|
//+------------------------------------------------------------------+
|
|
//--- (uint) so the throttle comparison is unsigned-vs-unsigned: age is a uint tick delta, and
|
|
//--- the ternary picking between these is a runtime expression the compiler cannot constant-
|
|
//--- fold, so bare int literals here drew a sign-mismatch warning (reported 2026-08-19).
|
|
#define DISPLAY_FWD_MIN_MS ((uint)4000)
|
|
#define DISPLAY_FWD_ERA_MS ((uint)1000)
|
|
bool CExpertSignalAIBase::DisplayInference(void)
|
|
{
|
|
//--- Meta head consumes fired candidates, not a bare bar window - a candidate-less forward is
|
|
//--- width-mismatched against its input layer. Same guard as RefreshConvergedSignal.
|
|
if(IsMetaTarget())
|
|
return false;
|
|
if(CheckPointer(Net) == POINTER_INVALID)
|
|
return false;
|
|
if(m_outputNeuronsCount != 1 && m_outputNeuronsCount != 3)
|
|
return false;
|
|
uint now = GetTickCount();
|
|
uint age = now - m_dispStamp; // unsigned subtraction survives the 49-day wrap
|
|
bool eraMoved = ((long)m_eraCount != m_dispEra);
|
|
if(m_dispStamp != 0 && age < (eraMoved ? DISPLAY_FWD_ERA_MS : DISPLAY_FWD_MIN_MS))
|
|
return m_dispValid; // serve the cache (or keep failing quietly) until the throttle opens
|
|
m_dispStamp = now;
|
|
if(!BuildFeatureWindow(1))
|
|
return m_dispValid; // window not buildable (warm-up, indicator hole): keep the last read
|
|
//--- Save/restore, NOT set/clear - see the header. Frozen, this forward is a pure function.
|
|
bool bnWasFrozen = Net.GetBatchNormFrozen();
|
|
if(!bnWasFrozen)
|
|
Net.SetBatchNormFrozen(true);
|
|
bool fwdOk = Net.feedForward(TempData);
|
|
if(fwdOk)
|
|
Net.getResults(TempData);
|
|
if(!bnWasFrozen)
|
|
Net.SetBatchNormFrozen(false);
|
|
if(!fwdOk)
|
|
return m_dispValid;
|
|
if(m_outputNeuronsCount == 1)
|
|
{
|
|
double v = TempData.At(0);
|
|
if(!MathIsValidNumber(v))
|
|
return m_dispValid; // NaN net: keep the last finite read, the NaN latch reports elsewhere
|
|
m_dispProbs[0] = v;
|
|
m_dispProbs[1] = 0.0;
|
|
m_dispProbs[2] = 0.0;
|
|
m_dispSignal = v;
|
|
}
|
|
else
|
|
{
|
|
//--- Same two calls, same order, as the live decision in RefreshLatestSignal: softmax INTO
|
|
//--- TempData, then the strict-majority read. On non-finite logits the softmax returns 0
|
|
//--- WITHOUT normalizing TempData - the finiteness check below is what keeps raw NaN logits
|
|
//--- from being displayed as probabilities.
|
|
ApplyClassificationSoftmax();
|
|
double p0 = TempData.At(0), p1 = TempData.At(1), p2 = TempData.At(2);
|
|
if(!MathIsValidNumber(p0) || !MathIsValidNumber(p1) || !MathIsValidNumber(p2))
|
|
return m_dispValid;
|
|
m_dispProbs[0] = p0; // Buy
|
|
m_dispProbs[1] = p1; // Sell
|
|
m_dispProbs[2] = p2; // Neutral
|
|
m_dispSignal = AdjustedSignalFromSoftmax();
|
|
}
|
|
m_dispEra = (long)m_eraCount;
|
|
m_dispValid = true;
|
|
return true;
|
|
}
|
|
#endif // WARRIOR_AIBASE_INFERENCE_MQH
|