forked from animatedread/Warrior_EA
3b77500726
Screened yields (DGS2/DGS10), curve slope, inflation breakevens, Fed policy,
the Fed-ECB policy differential, and monthly US unemployment and CPI - all on
ALFRED first prints, 499 permutations, against forward 5-day range.
NOT ONE macro feature clears the family-wise bar on any symbol. The only thing
that clears anywhere is the trailing-range positive control, which is what it
is there to do. Best a-priori candidate, the Fed-ECB differential on EURUSD,
came in at MI 0.00170 p=0.088 - nothing. The two features flagged INCREMENTAL
(dgs2_chg5 on SP500) have null marginal MI and are isolated conditional cells
at the expected false-positive rate, not findings.
The `distinct` column quantifies the power argument instead of asserting it:
unemployment takes 51-66 distinct values across 3,745-6,159 bars, CPI 174-277,
against 6,159 for a continuous feature. A monthly series pasted onto daily bars
carries about 1% of the resolution, and it showed - the monthly features were
among the weakest in every table.
The contrast with the implied-vol screen is the useful part: the options
market FORWARD-LOOKING view of an instrument (gvz_chg5 on gold, MI|vol 0.0197)
carries real information about its range, while the economy BACKWARD-LOOKING
state carries none. Mismatched timescales - rate levels move over months,
5-day range moves daily.
Also makes load_bars fall back to htf/{SYM}_D1_mid.npz when the tick-derived
build is absent (the 2026-08-16 disk cleanup removed bars/ but htf/ survived),
with need_ticks=True turning that fallback into a loud failure for the
order-flow screen rather than silently testing flow features on OHLC data.
No EA change: nothing survived to wire.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
86 lines
4 KiB
Python
86 lines
4 KiB
Python
"""Order-flow 'toxicity' screen: can VPIN-style flow features predict forward RANGE?
|
|
|
|
Motivated by the microstructure literature (Easley/Lopez de Prado VPIN: toxic
|
|
flow predicts liquidity evaporation = range expansion). Our earlier tick-flow
|
|
verdict killed flow->DIRECTION (real but decays with spread); flow->RANGE was
|
|
never tested, and range is the channel that pays for geometry/sizing.
|
|
|
|
Features are built from the tick-derived D1 bars already on disk (upticks/
|
|
downticks/ticks/rvol/spread columns) - zero collection cost. The conditional
|
|
MI|vol column is the decisive one: every flow feature must beat what trailing
|
|
realized range already knows.
|
|
|
|
Usage:
|
|
python -m altdata.screen_flow [--perms 499]
|
|
"""
|
|
import argparse
|
|
|
|
import numpy as np
|
|
import pandas as pd
|
|
|
|
from .screen import _tercile, cmi_3x3x3, load_bars, mi_3x3, true_range
|
|
|
|
|
|
def main() -> None:
|
|
ap = argparse.ArgumentParser()
|
|
ap.add_argument("--sym", nargs="*", default=["SP500", "EURUSD", "USDJPY", "XAUUSD"])
|
|
ap.add_argument("--perms", type=int, default=499)
|
|
ap.add_argument("--seed", type=int, default=7)
|
|
args = ap.parse_args()
|
|
rng = np.random.default_rng(args.seed)
|
|
|
|
for sym in args.sym:
|
|
bars = load_bars(sym, need_ticks=True) # flow features need the tick-derived build
|
|
tr = true_range(bars)
|
|
atr = tr.ewm(alpha=1 / 14, min_periods=14).mean()
|
|
trailing = tr.rolling(5).sum() / atr
|
|
|
|
ticks = bars["ticks"].replace(0, np.nan)
|
|
imb = (bars["upticks"] - bars["downticks"]) / ticks
|
|
feats = pd.DataFrame(index=bars.index)
|
|
feats["flow_imb"] = imb # signed flow
|
|
feats["flow_tox"] = imb.abs() # VPIN-ish one-sidedness
|
|
feats["flow_tox_5d"] = imb.abs().rolling(5).mean() # persistent toxicity
|
|
feats["activity"] = ticks / ticks.rolling(20).mean() # tick-count surge
|
|
feats["rvol_gap"] = bars["rvol"] / atr # intrabar vol vs ATR estimate
|
|
feats["spread_stress"] = bars["spread_max"] / bars["spread_mean"].replace(0, np.nan)
|
|
feats["CTRL_trailing_range"] = trailing
|
|
feats["CTRL_noise"] = rng.standard_normal(len(bars))
|
|
|
|
for target in ("range", "dir"):
|
|
if target == "range":
|
|
y_raw = tr.shift(-1).rolling(5).sum().shift(-4) / atr
|
|
else:
|
|
y_raw = (bars["close"].shift(-5) - bars["close"]) / atr
|
|
valid = y_raw.notna() & trailing.notna()
|
|
y_bin = _tercile(y_raw[valid].to_numpy())
|
|
z_bin = _tercile(trailing[valid].to_numpy())
|
|
print(f"\n=== {sym}: {int(valid.sum())} D1 bars, target={target}, {args.perms} perms ===")
|
|
results, null_max = [], np.zeros(args.perms)
|
|
for name in feats.columns:
|
|
x = feats.loc[valid, name].to_numpy(dtype=float)
|
|
ok = ~np.isnan(x)
|
|
if ok.sum() < 500:
|
|
continue
|
|
xo, yo, zo = x[ok], y_bin[ok], z_bin[ok]
|
|
n = len(xo)
|
|
obs, obs_c = mi_3x3(xo, yo), cmi_3x3x3(xo, yo, zo)
|
|
null = np.empty(args.perms)
|
|
null_c = np.empty(args.perms)
|
|
for k in range(args.perms):
|
|
xs = np.roll(xo, rng.integers(63, n - 63))
|
|
null[k] = mi_3x3(xs, yo)
|
|
null_c[k] = cmi_3x3x3(xs, yo, zo)
|
|
p = (np.sum(null >= obs) + 1) / (args.perms + 1)
|
|
p_c = (np.sum(null_c >= obs_c) + 1) / (args.perms + 1)
|
|
results.append((name, obs, p, obs_c, p_c))
|
|
null_max = np.maximum(null_max, null)
|
|
fam = np.quantile(null_max, 0.95)
|
|
print(f" family-wise 5% bar over {len(results)} features: {fam:.5f}")
|
|
for name, obs, p, obs_c, p_c in sorted(results, key=lambda r: -r[1]):
|
|
mark = "FAMILY" if obs > fam else " "
|
|
print(f" {name:20s} MI {obs:.5f} p={p:.4f} {mark} | MI|vol {obs_c:.5f} p={p_c:.4f}")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|