OnlineFDR

Online false discovery rate control in native MQL5: LORD3, LORD++ and SAFFRON deciding, one signal at a time, which firings are worth acting on, and a measured false discovery fraction among the ones that are.

Companion code for the MQL5 article: https://www.mql5.com/en/articles/24577

What it does

Point a scanner at 28 symbols and 7 timeframes, test each cell at 5% on data with no edge anywhere, and about ten cells light up every sweep. Run it again next week and ten more do. Nothing is wrong with the test; the arithmetic of testing 196 hypotheses at 5% says so. What is missing is any accounting for how many questions were asked, and without that accounting the word "significant" carries no information about the signals you end up trading.

The offline answer is Benjamini-Hochberg, which sorts a finished batch of p-values and picks a threshold after seeing all of them. A live scanner does not have a finished batch. Hypotheses arrive one at a time, each must be answered before the next one shows up, the total is never known, and a decision once taken is not revisited. That is the online problem, and it needs different machinery.

The machinery is an error budget called alpha-wealth. The procedure starts with w0, pays the test level it assigns out of that budget for every hypothesis it examines, and is refunded a payout each time it rejects. Discoveries finance further testing; a long run of nulls slowly starves the procedure, which is correct behaviour rather than a defect and is the single most important practical fact about running these on a fast stream. Five procedures ship: two baselines that control nothing (uncorrected, Bonferroni) so there is something to compare against, and LORD3, LORD++ and SAFFRON. On the synthetic control LORD++ holds realised FDR at 0.027 against a nominal 0.10, and roughly half of that conservatism is the gamma sequence rather than the procedure: the LORD form sums to 0.504, not 1.

Getting a valid p-value out of a trading rule turned out to be harder than implementing the procedures, and two findings came out of it that are not visible from reading the code.

A rank-based bootstrap p-value has a resolution floor of 1/(B+1), and an online procedure walks straight underneath it. With B = 200 that floor sits at 0.005, while LORD++ is handing out levels of 1e-4 by the twentieth hypothesis. The smallest p-value the source could physically produce was larger than the threshold it had to clear, so zero rejections were guaranteed by arithmetic no matter what the data said, and it looked exactly like a well-behaved conservative filter. Raising B does not rescue it: at B = 2000 the floor is still five times too high, and the level keeps falling faster than brute force can chase it. FDR_P_BOOT_NORMAL keeps the block-bootstrap null but reads its tail off a fitted normal, so the p-value is continuous and has no floor. The cost is an explicit modelling assumption in the far tail, and it is a real one: the measured p < 0.01 rate is 0.018, so the tail is the weak point and it is exactly where the levels live.

The verdict on a p-value source has to be one-sided. What the procedures require is super-uniformity, P(p <= x) <= x, not uniformity. A source returning too few small p-values is conservative, costs power and nothing else, and every guarantee still holds. A two-sided KS test cannot tell that apart from the opposite failure and fails both, which condemned the correct block bootstrap alongside the genuinely broken t-test. The statistic that answers the right question is the one-sided D+.

Two more results worth stating. Online FDR is path-dependent: on a 4000-bar window the identical configuration made zero discoveries and sat 94% starved, while on 20000 bars it found 105, because the first discovery is what refills the wealth and an unlucky start starves the procedure permanently. And a failed exit silently breaks the experiment: the Expert holds h bars to match the tested horizon, PositionClose can be refused, and resetting the hold counter regardless held positions past h while reporting h. The buggy version flattered the gate, reporting an expected payoff of -0.90 against a true -1.49, so it would have produced a wrong table.

On 20000 EURUSD H1 bars the five rules produce 3428 testable hypotheses. SAFFRON ends with wealth 7.658 after 107 discoveries, which reconciles exactly as 0.05 - 3.04 + 10.65 and is worth about 76 discoveries of remaining budget. The Expert then runs the gate against its own control arm on the same stream.

Layout

Include/OnlineFDR/FdrCore.mqh            shared enums, structs, gamma sequences, deterministic RNG, normal CDF/quantile
Include/OnlineFDR/FdrProcedures.mqh      CFdrBase and the five procedures: uncorrected, Bonferroni, LORD3, LORD++, SAFFRON
Include/OnlineFDR/PValueSource.mqh       stationary block bootstrap, the three p-value modes, forward returns
Include/OnlineFDR/SignalBank.mqh         five deliberately different rules, each firing on transition rather than state
Include/OnlineFDR/FdrStream.mqh          walks a finished price array and emits candidates in time order
Include/OnlineFDR/FdrSynthetic.mqh       synthetic streams with known ground truth, one-sided KS, dependence scenarios
Include/OnlineFDR/FdrMonitor.mqh         the live side: one bar in, one decision out, append or full replay

Scripts/OnlineFDR/CalibrationBench.mq5   p-value validity on a null market, then FDR and power for all five procedures
Scripts/OnlineFDR/DependenceBench.mq5    the dependence failure envelope and the throttle mitigation
Scripts/OnlineFDR/StreamBench.mq5        the whole pipeline on real bars, with per-procedure and per-candidate dumps

Indicators/OnlineFDR/AlphaWealth.mq5     level, wealth, p-values and discoveries drawn in a subwindow
Experts/OnlineFDR/FdrFilterEA.mq5        one Expert, one switch: GATE_OFF acts on every firing, GATE_ON only on discoveries

Run CalibrationBench.mq5 first. It decides whether the p-value source is valid at all, and if the one-sided D+ fails there then nothing downstream is worth reading. DependenceBench.mq5 maps where the guarantee breaks. StreamBench.mq5 is the clean signal-level comparison on real bars, and FdrFilterEA.mq5 is the execution-realism check.

Substituting your own rules is a single edit: SignalBank.mqh is the only place that turns a price series into firings, and nothing downstream knows where a candidate came from. Keep the rules genuinely different from each other; five variations of one idea produce a stream of duplicates, and duplicates break the independence the procedures lean on in the direction that makes realised FDR exceed nominal.

Disclaimer

Educational code. This is a measurement tool, not a profit tool, and the distinction is the whole point. On the EURUSD H1 A/B both arms lose money: the control takes 938 trades for -1774 at a 19.1% drawdown, the gated arm 89 trades for -133 at 3.0%, and the win rate is unchanged at 0.481 against 0.483. The gate did not find an edge because these five rules do not have one and the Expert pays spread. What it delivers is a number you did not have before, namely the expected share of the signals you acted on that were noise. Note also that the control arm is throttled by the one-position book, which is why StreamBench rather than the Expert is the clean comparison. Test on your own data and broker conditions before drawing conclusions.