Commit graph Warrior_EA/research/test_temporal.py
Author SHA1 Message Date
AnimateDread
fcb69d8efb research: test whether across-time structure exists at all
The one hypothesis the shipped diagnostics do not cover. The EA prints it
itself: the MI measure is "marginal (one feature at a time) and per-bar",
so a floor reading "cannot rule out one that only exists in combination or
across time". The instrument for across-time structure is the sequence
model, and until bfc1da9 it read its window backwards - so the hypothesis
has never been tested, not once.

Asks the DATA rather than retraining the LSTM. A topology comparison in
MT5 confounds "can order be exploited" with learning rate, init, capacity,
class weighting, batch-norm state and a single OOS slice, and costs hours
per run on a CPU-only box.

Three configurations, identical model / CV / null:
  A  entry bar only         joint across FEATURES, one bar
  B  window, chronological  joint across features AND time
  C  window, lag order shuffled INDEPENDENTLY PER SAMPLE

B-A: does history help at all.  B-C: does temporal POSITION carry
information - the actual hypothesis.

C is the design. A single global shuffle tests nothing, because any model
relearns a permuted column layout; per-sample shuffling kills the
lag<->column correspondence while preserving the exact multiset of values
in the window. It is the time-order analogue of the shuffled-label null.

VALIDATED WITH BOTH CONTROLS, because an untested research script is worse
than none:
  planted order-only label (monotone up/down):
      A -4.13pp   B +85.68pp   C -4.13pp   B-C +89.81pp
      C collapses to exactly A - an order-destroyed window carries
      precisely as much as one bar. Correct.
  planted order-free label (window sum):
      A +22.23pp  B +65.62pp   C +65.44pp  B-C +0.18pp
      history helps (+43pp over one bar) and the test attributes none of
      it to order. Correct.

Metric is directional precision against max(P(Buy),P(Sell)) - the same
metric and zero-skill benchmark the EA's deployability gate uses, so a
result reads directly against an era line.

Two p-values, because the analytic one lies: barrier labels are massively
autocorrelated, so the binomial p is optimistic and is reported only for
comparability. The BLOCK-permutation p (block = horizon, Phipson & Smyth)
is the one to believe - same correction test_volume.py already applies.

Purged, embargoed walk-forward CV (H bars each side), because without the
embargo the barrier horizon leaks across the split - the mistake that
scored +0.53 R on an "honest" holdout once already.

normal_upper() mirrors NormalUpperTail() in Training.mqh so the two agree
to 6dp.

Needs <SYM>_<TF>_rates.csv from a WARRIOR_EXPORT_FEATURES build.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 20:42:49 -04:00