Warrior_EA/research/sim.py

56 lines
2.6 KiB
Python
Raw Permalink Normal View History

research: offline validation kit, and the answer it produced Keeps the Python that turned a per-hypothesis cost of minutes into seconds, so the next person (or the next me) can re-run any of this without MetaTrader in the loop. kit.py vectorised triple-barrier labeller + scale-free feature set. The labeller is a faithful port - stop tested before target within a bar, so a bar spanning both scores as the loss - and reproduces the EA's own distribution to 0.05pp (24.93/22.01/53.06 vs 24.9/22.0/53.1) at 12x the speed. wf.py purged, embargoed walk-forward gradient boosting. sim.py sequential NON-OVERLAPPING trade simulation. sweep.py the cell x geometry sweep. detail.py full threshold profile for one configuration. WHAT IT FOUND, and the order matters because the first answer was wrong: Naive walk-forward looked like an edge - precision rising monotonically with model confidence, 24.4/24.6/25.3/26.2/27.1%, topping out at +2.66pp and 2.9 sigma. All of it pseudo-replication: a 128-bar barrier means adjacent bars share almost their whole outcome window, so one trade was being counted up to 128 times. Counting each trade ONCE (sim.py) the ordering collapses to 25.9/31.1/26.8/28.1/25.8 and nothing is significant. Same error family as the MI null that assumed independence. That exposed a structural problem bigger than the result: at a 128-bar horizon, 18 years of SP500 H1 yields at most ~300 independent trades, which can only resolve a +6pp edge at 2 sigma. Real edges are 1-3pp. The shipped configuration is not merely unproven - it is statistically UNFALSIFIABLE on the available history. The cost/power screen then showed SP500 is among the worst cells available: 54k bars and spread/ATR 0.34, against EURUSD/USDJPY at 178k bars and 0.05. Weeks of training went into the hardest instrument on the list, 3x less data and 7x the relative cost. Final sweep - 4 instruments x 3 geometries, purged walk-forward, independent trades: nothing clears +2 sigma. The one survivor (EURUSD 2:3 h48, +2.76pp at +1.57 sigma) dissolves under its full threshold profile: non-monotone across thresholds, and its per-fold win rate decays monotonically through time (52.9 -> 45.9 -> 41.7 -> 35.2 -> 26.2). Conclusion: with price/volume-derived technical features there is no tradeable entry-direction edge on these instruments - now tested with a model class that finds interactions, on 3x the data, with honest statistics. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 16:21:44 -04:00
import numpy as np, sys, datetime as dt
sys.stdout.reconfigure(encoding='utf-8',errors='replace')
from sklearn.ensemble import HistGradientBoostingClassifier
F=np.load("F.npy"); meta=np.load("meta.npy"); y=np.load("lab.npy"); valid=np.load("valid.npy")
t=meta[:,0]; hrs=np.load("hrs.npy"); dow=np.load("dow.npy")
X=np.column_stack([F,np.sin(2*np.pi*hrs/24),np.cos(2*np.pi*hrs/24),dow])
n=len(y); idx=np.arange(n); H=128
# ---- out-of-sample probabilities via purged walk-forward (never fit on data touching the test window)
prob=np.full((n,3),np.nan)
bounds=np.linspace(int(n*0.3),n,7).astype(int)
fold_of=np.full(n,-1)
for f in range(6):
te0,te1=bounds[f],bounds[f+1]
te=(idx>=te0)&(idx<te1)&valid; tr=(idx<te0-H)&valid
if tr.sum()<5000: continue
m=HistGradientBoostingClassifier(max_iter=300,learning_rate=0.06,max_depth=6,
l2_regularization=1.0,random_state=0).fit(X[tr],y[tr])
prob[te]=m.predict_proba(X[te]); fold_of[te]=f
np.save("prob.npy",prob); np.save("fold.npy",fold_of)
def simulate(thr, verbose=True):
"""Sequential, NON-OVERLAPPING trades - the only honest simulation. You cannot hold 128
overlapping positions, so a signal is ignored while a trade is open. Every resulting trade
is then statistically independent, which is what makes the confidence interval mean anything."""
trades=[]; j=0
while j<n:
if fold_of[j]<0 or not np.isfinite(prob[j,0]): j+=1; continue
p=prob[j]; pred=int(np.argmax(p)); conf=float(np.max(p))
if pred==2 or conf<thr: j+=1; continue
win = (y[j]==pred)
trades.append((j,pred,win,fold_of[j]))
j+=H # hold to the barrier before looking again
if not trades:
print(f" thr {thr:.2f}: no trades"); return None
arr=np.array([(a,b,c,d) for a,b,c,d in trades])
wins=arr[:,2].astype(bool); nT=len(arr)
wr=wins.mean()
# 1:3 payoff, spread already charged inside the label
exp_atr = wr*6 - (1-wr)*2
se=np.sqrt(0.25*0.75/nT)
if verbose:
nb=int((arr[:,1]==0).sum()); ns=int((arr[:,1]==1).sum())
print(f" thr {thr:.2f}: {nT:4d} independent trades ({nb} long / {ns} short) "
f"win {100*wr:5.2f}% vs 25.00% edge {100*(wr-0.25):+5.2f}pp ({(wr-0.25)/se:+.2f} sigma) "
f"expectancy {exp_atr:+.3f} ATR/trade")
for f in range(6):
fm=arr[:,3]==f
if fm.sum()>=10:
print(f" fold {f}: {int(fm.sum()):4d} trades win {100*wins[fm].mean():5.2f}%")
return wr,nT
print("=== SEQUENTIAL NON-OVERLAPPING TRADES (26 features + hour/dow, 2:6, h=128) ===")
for thr in (0.40,0.45,0.50,0.55,0.60):
simulate(thr)