Warrior_EA/System/BinomialStats.mqh

95 lines
5 KiB
MQL5
Raw Permalink Normal View History

refactor(dry): one binomial arithmetic for every "is this edge real" test The formula p(1-p)/n was transcribed nine times across six files - the two deploy gates, the two edge floors, the collapse recall floor, the barrier rung ladder, the inference bin SE, the pooled inverse-variance weights and both detectability reports. System\BinomialStats.mqh now holds it once, as free functions with no class dependency, so the god-class declaration does not grow to host pure math. BinomialVar(p, n) p(1-p)/n BinomialSEPct(p, n) 100*sqrt(p(1-p)/n) BinomialCallsForEdge(p, edge, sigmas) the same, solved for n NormalUpperTailQ(z) Q(z), via Math\Stat\Normal.mqh SidakFamilyP(z, N) 1-(1-Q(z))^N Value-preserving by construction: rates go in as probabilities so no call site gained a *100/100 round-trip, and BinomialSEPct is written through BinomialVar so the multiply order is the one it replaced. Every degenerate guard each site carried (p<=0, p>=1, n<=0) now lives in one place and returns the 0 those sites already treated as "no bar to clear". CExpertSignalAIBase::NormalUpperTail is gone; NormalUpperTailQ replaces it. What consolidating SURFACED, and is deliberately NOT changed here: the two Sidak selection gates compute their SE on the RAW call count, while every other SE in the project deflates by EffectiveSampleSize() for triple- barrier label overlap. That makes them the most permissive test in the codebase, by ~sqrt(mean label lifespan). Correcting it tightens a live deploy bar, which is a policy decision, not a refactor - flagged in the code at both sites. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:50:24 -04:00
//+------------------------------------------------------------------+
//| BinomialStats.mqh |
//| AnimateDread |
//| project. Free functions, no state, so the deploy gate, the two |
//| edge floors, the barrier ladder and the detectability reports |
//| all read the same formula instead of nine transcriptions of it. |
refactor(dry): one binomial arithmetic for every "is this edge real" test The formula p(1-p)/n was transcribed nine times across six files - the two deploy gates, the two edge floors, the collapse recall floor, the barrier rung ladder, the inference bin SE, the pooled inverse-variance weights and both detectability reports. System\BinomialStats.mqh now holds it once, as free functions with no class dependency, so the god-class declaration does not grow to host pure math. BinomialVar(p, n) p(1-p)/n BinomialSEPct(p, n) 100*sqrt(p(1-p)/n) BinomialCallsForEdge(p, edge, sigmas) the same, solved for n NormalUpperTailQ(z) Q(z), via Math\Stat\Normal.mqh SidakFamilyP(z, N) 1-(1-Q(z))^N Value-preserving by construction: rates go in as probabilities so no call site gained a *100/100 round-trip, and BinomialSEPct is written through BinomialVar so the multiply order is the one it replaced. Every degenerate guard each site carried (p<=0, p>=1, n<=0) now lives in one place and returns the 0 those sites already treated as "no bar to clear". CExpertSignalAIBase::NormalUpperTail is gone; NormalUpperTailQ replaces it. What consolidating SURFACED, and is deliberately NOT changed here: the two Sidak selection gates compute their SE on the RAW call count, while every other SE in the project deflates by EffectiveSampleSize() for triple- barrier label overlap. That makes them the most permissive test in the codebase, by ~sqrt(mean label lifespan). Correcting it tightens a live deploy bar, which is a policy decision, not a refactor - flagged in the code at both sites. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:50:24 -04:00
//+------------------------------------------------------------------+
#ifndef WARRIOR_SYSTEM_BINOMIALSTATS_MQH
#define WARRIOR_SYSTEM_BINOMIALSTATS_MQH
#include <Math\Stat\Normal.mqh>
//+------------------------------------------------------------------+
//| Upper-tail standard normal, Q(z) = P(Z >= z). |
//| |
//| Wraps the library so the NaN policy lives in one place: an |
//| unusable z reads as "not significant" rather than propagating a |
//| NaN into a gate decision. tail=false asks for the UPPER tail, and |
//| the clamp keeps a -1e-17 round-off out of the Sidak power. |
//+------------------------------------------------------------------+
double NormalUpperTailQ(const double z)
{
if(!MathIsValidNumber(z))
return(1.0);
int err=0;
double q=MathCumulativeDistributionNormal(z,0.0,1.0,false,false,err);
if(err!=ERR_OK || !MathIsValidNumber(q))
return(1.0);
return(MathMax(0.0,MathMin(1.0,q)));
}
//+------------------------------------------------------------------+
//| Sampling variance of a binomial rate p over n observations: |
//| p(1-p)/n, in fraction^2. |
//| |
//| Returns 0 for a degenerate rate or an empty sample, which every |
//| caller already treats as "no bar to clear". Callers that combine |
//| symbols by inverse variance want this rather than the SE. |
//+------------------------------------------------------------------+
double BinomialVar(const double p,const double n)
{
if(!MathIsValidNumber(p) || !MathIsValidNumber(n))
return(0.0);
if(n<=0.0 || p<=0.0 || p>=1.0)
return(0.0);
return(p*(1.0-p)/n);
}
//+------------------------------------------------------------------+
//| Standard error of a binomial rate p over n observations, in |
//| percentage points: 100 * sqrt(p(1-p)/n). |
//+------------------------------------------------------------------+
double BinomialSEPct(const double p,const double n)
{
return(100.0*MathSqrt(BinomialVar(p,n)));
}
//+------------------------------------------------------------------+
//| Independent observations needed to certify an edge of `edge` over |
//| a base rate p, at `sigmas` standard errors: |
//| n = sigmas^2 * p(1-p) / edge^2 |
//| |
//| BinomialSEPct solved for n. Answers "could this configuration |
//| EVER prove an edge this size" - a property of the geometry, the |
//| horizon and the window, which no amount of training moves. |
//+------------------------------------------------------------------+
double BinomialCallsForEdge(const double p,const double edge,const double sigmas)
{
if(edge<=0.0 || p<=0.0 || p>=1.0)
return(0.0);
return(sigmas*sigmas*p*(1.0-p)/(edge*edge));
}
//+------------------------------------------------------------------+
//| A rate shrunk toward a prior - the estimator, where everything |
//| above is the test. |
refactor(dry): one shrinkage estimator for classic ladders and AI tiers The Beta-prior arithmetic that turns counts into a ranking weight was written twice, term for term: WinRateFromCounts() for the classic pattern ladders and RankTiersFromOos() for the AI confidence tiers. Same formula, two transcriptions, and the same class of duplication the binomial SE consolidation removed a few commits ago. ShrunkRatePct() in System\BinomialStats.mqh is now the only copy. The two call sites keep what genuinely differs - the classic path passes RAW trade counts with a prior of MIN_TRADES_FOR_WIN_RATE, the AI path passes OVERLAP-CORRECTED effective counts with TIER_PRIOR_EFF_N, which is far smaller precisely because effective counts are - and that contract is now stated once, in the function, instead of being implied by two comments that could drift apart. Also fixes a difference the consolidation exposed: with an empty sample and a prior present, the posterior mean IS the prior, and returning 0 there would have handed a tier a vote weight of zero on no evidence. The AI path could reach that (effN can round to 0 when labels overlap heavily); the classic path cannot, since it returns NO_DATA_WIN_RATE first. Corrects a stale note of my own in passing: this ranking was recorded as a "raw win rate behind a MIN_TRADES cutoff heuristic". It is not, and has not been for some time - it is already a proper empirical-Bayes estimator with a per-filter pooled prior. Replacing it with a significance test, as that note implied, would have swapped the estimator the weight needs for a gate answering a different question. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 23:47:06 -04:00
//+------------------------------------------------------------------+
double ShrunkRatePct(const double hits,const double n,const double priorPct,const double priorN)
{
bool havePrior=(priorN>0.0 && MathIsValidNumber(priorPct) && priorPct>=0.0);
if(!MathIsValidNumber(hits) || !MathIsValidNumber(n) || n<=0.0)
return(havePrior ? priorPct : 0.0); // no evidence => the prior IS the estimate
if(!havePrior)
return(100.0*hits/n);
return((hits+priorN*(priorPct/100.0))*100.0/(n+priorN));
}
//+------------------------------------------------------------------+
refactor(dry): one binomial arithmetic for every "is this edge real" test The formula p(1-p)/n was transcribed nine times across six files - the two deploy gates, the two edge floors, the collapse recall floor, the barrier rung ladder, the inference bin SE, the pooled inverse-variance weights and both detectability reports. System\BinomialStats.mqh now holds it once, as free functions with no class dependency, so the god-class declaration does not grow to host pure math. BinomialVar(p, n) p(1-p)/n BinomialSEPct(p, n) 100*sqrt(p(1-p)/n) BinomialCallsForEdge(p, edge, sigmas) the same, solved for n NormalUpperTailQ(z) Q(z), via Math\Stat\Normal.mqh SidakFamilyP(z, N) 1-(1-Q(z))^N Value-preserving by construction: rates go in as probabilities so no call site gained a *100/100 round-trip, and BinomialSEPct is written through BinomialVar so the multiply order is the one it replaced. Every degenerate guard each site carried (p<=0, p>=1, n<=0) now lives in one place and returns the 0 those sites already treated as "no bar to clear". CExpertSignalAIBase::NormalUpperTail is gone; NormalUpperTailQ replaces it. What consolidating SURFACED, and is deliberately NOT changed here: the two Sidak selection gates compute their SE on the RAW call count, while every other SE in the project deflates by EffectiveSampleSize() for triple- barrier label overlap. That makes them the most permissive test in the codebase, by ~sqrt(mean label lifespan). Correcting it tightens a live deploy bar, which is a policy decision, not a refactor - flagged in the code at both sites. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:50:24 -04:00
//| Sidak family-wise p for the best of nTried candidates: |
//| 1 - (1 - p_single)^N. |
//| |
//| The null of the MAXIMUM, not of a single draw. At the magnitudes |
//| in play (p ~ 1e-4..1e-2, N ~ 10..1000) plain double precision is |
//| ample - no need for the log1p/expm1 form MQL5 would not give us |
//| anyway. |
//+------------------------------------------------------------------+
double SidakFamilyP(const double zObs,const int nTried)
{
return(1.0-MathPow(1.0-NormalUpperTailQ(zObs),(double)MathMax(nTried,1)));
}
//+------------------------------------------------------------------+
#endif