refactor(dry): one binomial arithmetic for every "is this edge real" test
The formula p(1-p)/n was transcribed nine times across six files - the two
deploy gates, the two edge floors, the collapse recall floor, the barrier
rung ladder, the inference bin SE, the pooled inverse-variance weights and
both detectability reports. System\BinomialStats.mqh now holds it once, as
free functions with no class dependency, so the god-class declaration does
not grow to host pure math.
BinomialVar(p, n) p(1-p)/n
BinomialSEPct(p, n) 100*sqrt(p(1-p)/n)
BinomialCallsForEdge(p, edge, sigmas) the same, solved for n
NormalUpperTailQ(z) Q(z), via Math\Stat\Normal.mqh
SidakFamilyP(z, N) 1-(1-Q(z))^N
Value-preserving by construction: rates go in as probabilities so no call
site gained a *100/100 round-trip, and BinomialSEPct is written through
BinomialVar so the multiply order is the one it replaced. Every degenerate
guard each site carried (p<=0, p>=1, n<=0) now lives in one place and
returns the 0 those sites already treated as "no bar to clear".
CExpertSignalAIBase::NormalUpperTail is gone; NormalUpperTailQ replaces it.
What consolidating SURFACED, and is deliberately NOT changed here: the two
Sidak selection gates compute their SE on the RAW call count, while every
other SE in the project deflates by EffectiveSampleSize() for triple-
barrier label overlap. That makes them the most permissive test in the
codebase, by ~sqrt(mean label lifespan). Correcting it tightens a live
deploy bar, which is a policy decision, not a refactor - flagged in the
code at both sites.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:50:24 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| BinomialStats.mqh |
|
|
|
|
|
//| AnimateDread |
|
2026-08-22 00:30:14 -04:00
|
|
|
//| project. Free functions, no state, so the deploy gate, the two |
|
|
|
|
|
//| edge floors, the barrier ladder and the detectability reports |
|
|
|
|
|
//| all read the same formula instead of nine transcriptions of it. |
|
refactor(dry): one binomial arithmetic for every "is this edge real" test
The formula p(1-p)/n was transcribed nine times across six files - the two
deploy gates, the two edge floors, the collapse recall floor, the barrier
rung ladder, the inference bin SE, the pooled inverse-variance weights and
both detectability reports. System\BinomialStats.mqh now holds it once, as
free functions with no class dependency, so the god-class declaration does
not grow to host pure math.
BinomialVar(p, n) p(1-p)/n
BinomialSEPct(p, n) 100*sqrt(p(1-p)/n)
BinomialCallsForEdge(p, edge, sigmas) the same, solved for n
NormalUpperTailQ(z) Q(z), via Math\Stat\Normal.mqh
SidakFamilyP(z, N) 1-(1-Q(z))^N
Value-preserving by construction: rates go in as probabilities so no call
site gained a *100/100 round-trip, and BinomialSEPct is written through
BinomialVar so the multiply order is the one it replaced. Every degenerate
guard each site carried (p<=0, p>=1, n<=0) now lives in one place and
returns the 0 those sites already treated as "no bar to clear".
CExpertSignalAIBase::NormalUpperTail is gone; NormalUpperTailQ replaces it.
What consolidating SURFACED, and is deliberately NOT changed here: the two
Sidak selection gates compute their SE on the RAW call count, while every
other SE in the project deflates by EffectiveSampleSize() for triple-
barrier label overlap. That makes them the most permissive test in the
codebase, by ~sqrt(mean label lifespan). Correcting it tightens a live
deploy bar, which is a policy decision, not a refactor - flagged in the
code at both sites.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:50:24 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
#ifndef WARRIOR_SYSTEM_BINOMIALSTATS_MQH
|
|
|
|
|
#define WARRIOR_SYSTEM_BINOMIALSTATS_MQH
|
|
|
|
|
#include <Math\Stat\Normal.mqh>
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Upper-tail standard normal, Q(z) = P(Z >= z). |
|
|
|
|
|
//| |
|
|
|
|
|
//| Wraps the library so the NaN policy lives in one place: an |
|
|
|
|
|
//| unusable z reads as "not significant" rather than propagating a |
|
|
|
|
|
//| NaN into a gate decision. tail=false asks for the UPPER tail, and |
|
|
|
|
|
//| the clamp keeps a -1e-17 round-off out of the Sidak power. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
double NormalUpperTailQ(const double z)
|
|
|
|
|
{
|
|
|
|
|
if(!MathIsValidNumber(z))
|
|
|
|
|
return(1.0);
|
|
|
|
|
int err=0;
|
|
|
|
|
double q=MathCumulativeDistributionNormal(z,0.0,1.0,false,false,err);
|
|
|
|
|
if(err!=ERR_OK || !MathIsValidNumber(q))
|
|
|
|
|
return(1.0);
|
|
|
|
|
return(MathMax(0.0,MathMin(1.0,q)));
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Sampling variance of a binomial rate p over n observations: |
|
|
|
|
|
//| p(1-p)/n, in fraction^2. |
|
|
|
|
|
//| |
|
|
|
|
|
//| Returns 0 for a degenerate rate or an empty sample, which every |
|
|
|
|
|
//| caller already treats as "no bar to clear". Callers that combine |
|
|
|
|
|
//| symbols by inverse variance want this rather than the SE. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
double BinomialVar(const double p,const double n)
|
|
|
|
|
{
|
|
|
|
|
if(!MathIsValidNumber(p) || !MathIsValidNumber(n))
|
|
|
|
|
return(0.0);
|
|
|
|
|
if(n<=0.0 || p<=0.0 || p>=1.0)
|
|
|
|
|
return(0.0);
|
|
|
|
|
return(p*(1.0-p)/n);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Standard error of a binomial rate p over n observations, in |
|
|
|
|
|
//| percentage points: 100 * sqrt(p(1-p)/n). |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
double BinomialSEPct(const double p,const double n)
|
|
|
|
|
{
|
|
|
|
|
return(100.0*MathSqrt(BinomialVar(p,n)));
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Independent observations needed to certify an edge of `edge` over |
|
|
|
|
|
//| a base rate p, at `sigmas` standard errors: |
|
|
|
|
|
//| n = sigmas^2 * p(1-p) / edge^2 |
|
|
|
|
|
//| |
|
|
|
|
|
//| BinomialSEPct solved for n. Answers "could this configuration |
|
|
|
|
|
//| EVER prove an edge this size" - a property of the geometry, the |
|
|
|
|
|
//| horizon and the window, which no amount of training moves. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
double BinomialCallsForEdge(const double p,const double edge,const double sigmas)
|
|
|
|
|
{
|
|
|
|
|
if(edge<=0.0 || p<=0.0 || p>=1.0)
|
|
|
|
|
return(0.0);
|
|
|
|
|
return(sigmas*sigmas*p*(1.0-p)/(edge*edge));
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
2026-08-22 00:30:14 -04:00
|
|
|
//| A rate shrunk toward a prior - the estimator, where everything |
|
|
|
|
|
//| above is the test. |
|
refactor(dry): one shrinkage estimator for classic ladders and AI tiers
The Beta-prior arithmetic that turns counts into a ranking weight was
written twice, term for term: WinRateFromCounts() for the classic
pattern ladders and RankTiersFromOos() for the AI confidence tiers.
Same formula, two transcriptions, and the same class of duplication the
binomial SE consolidation removed a few commits ago.
ShrunkRatePct() in System\BinomialStats.mqh is now the only copy. The
two call sites keep what genuinely differs - the classic path passes RAW
trade counts with a prior of MIN_TRADES_FOR_WIN_RATE, the AI path passes
OVERLAP-CORRECTED effective counts with TIER_PRIOR_EFF_N, which is far
smaller precisely because effective counts are - and that contract is
now stated once, in the function, instead of being implied by two
comments that could drift apart.
Also fixes a difference the consolidation exposed: with an empty sample
and a prior present, the posterior mean IS the prior, and returning 0
there would have handed a tier a vote weight of zero on no evidence.
The AI path could reach that (effN can round to 0 when labels overlap
heavily); the classic path cannot, since it returns NO_DATA_WIN_RATE
first.
Corrects a stale note of my own in passing: this ranking was recorded as
a "raw win rate behind a MIN_TRADES cutoff heuristic". It is not, and
has not been for some time - it is already a proper empirical-Bayes
estimator with a per-filter pooled prior. Replacing it with a
significance test, as that note implied, would have swapped the
estimator the weight needs for a gate answering a different question.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 23:47:06 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
double ShrunkRatePct(const double hits,const double n,const double priorPct,const double priorN)
|
|
|
|
|
{
|
|
|
|
|
bool havePrior=(priorN>0.0 && MathIsValidNumber(priorPct) && priorPct>=0.0);
|
|
|
|
|
if(!MathIsValidNumber(hits) || !MathIsValidNumber(n) || n<=0.0)
|
|
|
|
|
return(havePrior ? priorPct : 0.0); // no evidence => the prior IS the estimate
|
|
|
|
|
if(!havePrior)
|
|
|
|
|
return(100.0*hits/n);
|
|
|
|
|
return((hits+priorN*(priorPct/100.0))*100.0/(n+priorN));
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
refactor(dry): one binomial arithmetic for every "is this edge real" test
The formula p(1-p)/n was transcribed nine times across six files - the two
deploy gates, the two edge floors, the collapse recall floor, the barrier
rung ladder, the inference bin SE, the pooled inverse-variance weights and
both detectability reports. System\BinomialStats.mqh now holds it once, as
free functions with no class dependency, so the god-class declaration does
not grow to host pure math.
BinomialVar(p, n) p(1-p)/n
BinomialSEPct(p, n) 100*sqrt(p(1-p)/n)
BinomialCallsForEdge(p, edge, sigmas) the same, solved for n
NormalUpperTailQ(z) Q(z), via Math\Stat\Normal.mqh
SidakFamilyP(z, N) 1-(1-Q(z))^N
Value-preserving by construction: rates go in as probabilities so no call
site gained a *100/100 round-trip, and BinomialSEPct is written through
BinomialVar so the multiply order is the one it replaced. Every degenerate
guard each site carried (p<=0, p>=1, n<=0) now lives in one place and
returns the 0 those sites already treated as "no bar to clear".
CExpertSignalAIBase::NormalUpperTail is gone; NormalUpperTailQ replaces it.
What consolidating SURFACED, and is deliberately NOT changed here: the two
Sidak selection gates compute their SE on the RAW call count, while every
other SE in the project deflates by EffectiveSampleSize() for triple-
barrier label overlap. That makes them the most permissive test in the
codebase, by ~sqrt(mean label lifespan). Correcting it tightens a live
deploy bar, which is a policy decision, not a refactor - flagged in the
code at both sites.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:50:24 -04:00
|
|
|
//| Sidak family-wise p for the best of nTried candidates: |
|
|
|
|
|
//| 1 - (1 - p_single)^N. |
|
|
|
|
|
//| |
|
|
|
|
|
//| The null of the MAXIMUM, not of a single draw. At the magnitudes |
|
|
|
|
|
//| in play (p ~ 1e-4..1e-2, N ~ 10..1000) plain double precision is |
|
|
|
|
|
//| ample - no need for the log1p/expm1 form MQL5 would not give us |
|
|
|
|
|
//| anyway. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
double SidakFamilyP(const double zObs,const int nTried)
|
|
|
|
|
{
|
|
|
|
|
return(1.0-MathPow(1.0-NormalUpperTailQ(zObs),(double)MathMax(nTried,1)));
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
#endif
|