forked from animatedread/Warrior_EA
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
AnimateDread
|
932c94e890 |
feat(research): offline meta-pool pipeline - loader, EA-mirrored splits, MLP, cov x (p-BE) eval
Loads the EA's MetaExport .f32 datasets (UTF-16 sidecars), applies the EA's own discipline offline: chronological 55/15/30 split with horizon-length purges, operating point fitted on the calibration slice only via coverage x (precision - BE) with the 25% floor, test slice touched once, deployability at the 2-sigma edge floor. Small leaky-ReLU MLP + Adam in numpy; `stats` / `eval <tag>` / `pool` commands. First run on XAUUSD_16388 validated the plumbing and exposed the data: the 2.5h gold tester run only covered 2004-07..2006-10 (3,214 candidates) because gold tick volume is huge - and corpus builds do not need ticks at all (journaling is bar-open-keyed, labels come from bar history later), so "Open prices only" modeling builds the same corpus ~100x faster. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
b06fdd2f0e |
research: point the seasonal screen at the five breadth instruments
seasonal.py gets a frame-based entry (analyse_frame) and a reusable report() so the identical statistics - circular-rotation family-wise null, max-|t| bar, split-half - can run on instruments whose book is synthesised from M1 bars. breadth_seasonal.py runs it on the five SQX-decoded instruments (FTSE100, UK100, WTI x2 feeds, USDCAD) that share no data path with the four originals; the duplicate-market pairs (FTSE100/UK100, WTI_d/WTI_5) double as replication checks. Caveats stated in the module docstring: synthesised flat spread (move/spread is approximate, no intraday spread shape) and file-time clock labels; drift/t columns are spread-free and unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
371f8aaecd |
fix: the Adam second moment was never Adam - all four tiers
Root cause of the B=32 regression, and it predates F4 entirely. Every Adam
kernel stored v already square-rooted and then fed that stored value back in
as if it were the variance:
v_new = sqrt(b2 * v_old + (1 - b2) * g^2)
That recursion has a fixed point at v ~= b2 = 0.999 for ANY gradient below
unit scale, so the denominator stops tracking the gradient and Adam degrades
into plain SGD with lr = lt. Measured against the shipped WarriorCPU.dll
(batch_accum_check.cpp, TestOptimizerScaleInvariance), 4000 steps of a
constant gradient: 3285x less displacement at |g|=1e-5 than at |g|=1, where
a scale-invariant optimizer gives the same distance for both. After the fix
all six magnitudes read 1.199 and v tracks |g| exactly.
It hit conv/LSTM specifically because they sit behind a batch-norm with
running variance ~2.6e+05, so their gradients arrive divided by ~500 - deep
in the degraded regime - while the dense stack near the loss stayed in the
working one. In situ on SP500 H1: lstm1 dW/W 2.62/10.0/7.14% -> 0.024/0.022/
0.003%, conv1 decaying to 0.000% by era 30. NeuronBatchNorm.mqh already
squared v back for gamma/beta and its comment named the kernels as wrong,
which is exactly why gamma/beta kept training while the stages behind froze.
Persisted .nnw needs no migration - v keeps its std-dev meaning.
Also, the two ways F4 exposed it, both mine:
- No LR compensation for B fewer steps per era. sqrt(B) for adaptive methods
(Krizhevsky 2014; Granziol et al. 2022), applied once in
InitialEtaForOptimizer(). Linear scaling (Goyal et al. 2017) is for SGD.
- Plateau patience denominated in eras, so raising B made the ladder 32x more
impatient in its only unit. PAI converged at era 41 on ~49k updates where
the same config had been finding new bests at era 1028.
TrainPlateauPatienceEras() stretches it by the same sqrt(B).
TRAIN_BATCH_SIZE 32 -> 8 so the patience stretch stays affordable (8 -> 23
eras per stage, not 8 -> 45). Both helpers are identities at B=1.
Deploy gate: DEPLOY_MIN_SIDE_RECALL_PCT (10%) folded into tradeableOK. The
perceptron reported Sell:0% recall in all 41 eras, cleared the floor on Buy
alone at 36.6% vs 34% chance, deployed, and sprayed buy arrows. Folded into
the ranking key rather than checked at deploy time so a one-sided era cannot
become best-so-far in the first place.
Deinit: the arrow purge now runs BEFORE ExtPanel.Destroy(), an unbounded
CAppDialog teardown that sat ahead of it - the same ordering inversion the
rule there exists to prevent. CONV was force-terminated 4.8 s into OnDeinit
(vs ~1.1 s for the three that finished) having reached none of its cleanup,
so its arrows stayed on the chart. Steps are now timed in the log.
PurgeChart's verification rescan filtered on OBJ_ARROW, the same blind spot
as the bulk delete, so "persisted 10 ... cleared 0" passed silently. It now
walks every object type and reports the object counts when both are zero.
Both build variants compile 0 errors / 0 warnings; both DLLs rebuilt.
FORCES A RETRAIN (already forced by N1) and both DLLs must ship with the .ex5.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
0c01dc279b |
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1)
Completes the 2026-08-09 training audit. FORCES A RETRAIN of every
Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be
redeployed alongside the .ex5 - they carry new exports.
F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online
SGD (one weight update per bar), which is the mechanical source of the
era-to-era whipsaw every downstream guard was built to cope with. The O(n^2)
outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv /
AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the
optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so
there is one Adam/SGD implementation instead of four that can drift.
- the LSTM needs no outer-product kernel (WeightsGradient already holds the
sample's full dW) but could NOT simply be left un-zeroed between samples:
CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a
separate accumulator plus an elementwise add.
- batch-norm gamma/beta accumulate in host arrays, not new BatchOptions
slots - BN_OPT_STRIDE is baked into every persisted .nnw.
- scoped to pass 2; online learning keeps immediate updates. Every save /
checkpoint / scoring boundary flushes, scaling by the real sample count.
- degrades to per-sample updates (one log line) on a tier that cannot
accumulate, so old devices and DLL-free builds are unaffected.
- verified offline: DirectML/batch_accum_check.cpp drives the real exports
against an independent reference; at B=1 the accumulator matches the
shipped unbatched kernel's own gradient to 1.1e-16. Math only - the
in-situ check remains the per-layer dW/W report on a real era.
F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a
conv/LSTM front end had already reduced it, so an LSTM's dense stack was
charged for 1,280 inputs when it receives 64. Confirmed from the deployed
.cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now
budgeted against the front-end output and capped at it (never fan out), with
the derivation reordered so both stages settle first.
N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing
direction and Wyckoff stage into one scalar across a sign discontinuity. Split
into direction + [0,1] magnitude, the same convention the base OHLC block uses.
Information-preserving; 13 readings now occupy 16 inputs.
Compiled clean (0 errors, 0 warnings); both DLLs rebuilt.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
274630f802 |
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric
Four of the six findings from research/training_pipeline_audit_2026-08-09.md (F4 mini-batching and F6 feature re-encode deliberately deferred - see the report's implementation-status section for why): - F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%, which is 15-bit - provably non-uniform on every full-history era over 32,768 queued samples. New 30-bit ShuffleRandomIndex(). - F2: plateau warm restarts were a no-op whenever eta already sat at its ceiling (the normal state of a non-regressing plateau) - the ladder was just a 24-era countdown. Restarts now overshoot to 5x the ceiling (PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has real range. - F3: checkpoint restores put weights back but kept the rejected trajectory's Adam moments, so the optimizer immediately pushed back toward the rolled-back state (the restore->regress->restore oscillation). CNet::ResetOptimizerState() zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta untouched) on every mid-run restore, every boosted restart, and the deploy-time restore that online learning continues from. - F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk, so the selection metric the checkpoint ranking and deploy gate read is a pure function of the checkpoint instead of partly measuring BN drift. Defensive unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and the OOS continual-learning simulation stay adaptive by design. Compiled clean (0 errors, 0 warnings) via the staged-tree recipe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
fcb69d8efb |
research: test whether across-time structure exists at all
The one hypothesis the shipped diagnostics do not cover. The EA prints it
itself: the MI measure is "marginal (one feature at a time) and per-bar",
so a floor reading "cannot rule out one that only exists in combination or
across time". The instrument for across-time structure is the sequence
model, and until
|
||
|
AnimateDread
|
8ccbddb051 |
Add new research scripts for trading strategy analysis
- Implemented sqx_audit.py to audit StrategyQuant X trade lists, focusing on performance metrics and cost analysis. - Created sqx_portfolio.py to evaluate portfolio performance based on uncorrelated components and their impact on risk and return. - Developed swing.py to analyze cost ratios across different holding periods and assess swing trading structures. - Introduced test_management.py to investigate the effectiveness of exit rules on random entries and their impact on expectancy. |
||
|
AnimateDread
|
f1b7dcf7f3 |
fix: correct MI sample alignment and improve BN weight diagnostic report
The MI sample builder used `MathAbs(labelBarOffset)` as a padding, causing rows from offset and non-offset builds to be paired with a double shift. This broke the positive control, failed the 5× gate, and voided all reported mutual‑information figures. Replace with the fixed `MiShiftPad` constant to ensure builds enumerate the same set of bars and row-k alignment is preserved. Add `BatchOptionsTotal()` to `CNeuronBatchNormOCL` and split the packed BN weight array in the learning report into separate norms for the outgoing dense matrix, gamma, beta, running statistics, and Adam moment buffers. This turns an ambiguous single‑norm reading into precise diagnostics that distinguish weight divergence from scaling issues. |
||
|
AnimateDread
|
3459db887b |
infra: bid/ask M1 bars and an order engine that makes the fill bug unavailable
Three results today were invalidated by the same family of error: a price level used as an entry while the outcome was measured from somewhere else. The defence is not vigilance - it is an interface where the mistake cannot be expressed. bidask.py M1 bars carrying SEPARATE BID and ASK OHLC, built from the tick stream in one parallel pass. Every existing bar file stores MID, which is fine for measuring returns and useless for simulating orders, because no order ever executes at the mid. Mid bars force the spread to be bolted on afterwards as an average - the approximation that let today's artifacts through. With both sides carried, the spread is whatever it actually was, including the overnight and news blowouts an average hides. Fails loudly if ask < bid. fills.py owns the ENTIRE trade lifecycle; no test may open a position any other way. Longs enter at the ask and exit at the bid, shorts the reverse. A buy stop triggers on the ask and fills at the trigger price OR the bar's open if the bar gapped past it, which is where real slippage comes from. Limits fill on the opposite side and a gap is capped in their favour. The invariant that was violated: THE OUTCOME CLOCK STARTS AT THE FILL BAR. The fill index IS the start index - they are the same variable and cannot diverge. That is what went wrong before, and it is now unrepresentable rather than merely discouraged. Same-bar ambiguity is REPORTED, not assumed away: every result carries the fraction of trades decided by a bar containing both barriers, alongside fill rate and unresolved rate. A resolution-limited result now says so itself. M1 rather than raw ticks is a deliberate, stated bound: 513M ticks per symbol is ~8 GB packed and four symbols will not fit in memory, while M1 keeps it at ~500 MB and cuts the residual ambiguity 60x versus H1. It is an approximation with a visible error bar, not an exact simulator. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
82cdbb40df |
research: Wyckoff context REPLICATES on a second, independent trigger
Applies the same five-trace context score from
|
||
|
AnimateDread
|
c998d655ee |
research: Wyckoff CONTEXT has real predictive content - 8/8 positive slopes
Earlier tests fired on the shakeout alone, which is not the method. Book 2 2.3
treats it as the third of four cumulative traces and reads the structure's own
history first. This scores all of them, oriented to the shakeout's direction:
1 Phase A test location (upper vs lower half of the structure)
2 Phase B test location
2b STRUCTURAL FAILURE - after the Phase B test, did price fail to reach the
opposite extreme
4 effort/result on the shakeout bar (close position + volume vs range average)
7.1 higher-timeframe context - is the larger move in the shakeout's favour
Conditioning on agreement shrinks the sample and multiplies the ways to slice
it, so the test is NOT 'find the combination that works'. It is the one
pre-specified prediction the books make and mining does not: expR must rise
MONOTONICALLY with the number of agreeing traces. One slope, no threshold to
tune, no best cell to pick.
POOLED (16,234 non-overlapping trades):
0 traces -0.332 3 traces -0.114
1 trace -0.214 4 traces -0.117
2 traces -0.184
slope +0.0464 R per agreeing trace, t +2.16
per-symbol slopes POSITIVE IN ALL 8 CELLS (p ~ 0.004 on sign alone)
So the context logic is real and measurable - it is not folklore. But the base
trade is in too deep a hole for it to matter: full confluence still returns
-0.117, and reaching break-even would need ~7 agreeing traces when only 5 exist.
The useful reading is that context is a MODIFIER worth about +0.05 R per trace,
which is only interesting when bolted to a trigger whose base expectancy is
already near zero. The shakeout's is not, because its structural target sits
4-6R away and is rarely reached.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
9a4ae635e0 |
research: the two canonical Wyckoff trades tested whole - both negative
Completes the programme on both books. Entries are MARKET ORDERS at a bar's
open throughout, so the fill-timing artifact that invalidated the last round is
designed out rather than remembered. Benchmark is analytic: entry, stop and
target fixed at entry means a driftless market gives expR = 0 exactly.
1. SPRING / UPTHRUST (book 1 ch.18, the event 'all Wyckoff operators wait for').
Pierce of a COMPRESSION-QUALIFIED range edge, close back inside, stop beyond
the shakeout extreme, target the far side of the range.
30 cells across 4 symbols x M15/H1/H4. Reward-to-risk averages 4-6:1, so the
break-even win rate is only 15-20%, and it still loses nearly everywhere:
M15 all four symbols -0.15 to -0.24 with 0/4 folds positive. Best cell is
EURUSD H1 climactic-volume +0.302 at t +2.27, which over 30 cells is inside
the family-wise band.
The books' volume requirement was applied - climactic (>1.5x range average)
and quiet (<0.8x) shakeouts scored separately. Neither rescues it.
2. LPS / LPSY, the test-after-breakout, and book 2's A/B (5.7.1, 5.8.3): it
claims the retest should be awaited at the VOLUME PROFILE level, not the
price edge. Same breakout, same stop, same 2R target, only the location
differs:
retest at typical expR
A price edge -0.041 .. -0.315
B value-area edge -0.128 .. -0.413
C range VPOC -0.129 .. -0.506
24/24 cells negative, and A > B > C in ALL EIGHT symbol/timeframe
combinations. That monotone ordering is not noise, and it inverts the book's
recommendation. Mechanism is adverse selection: the VPOC sits deep inside the
old range, so a retest that reaches it is disproportionately a breakout that
has already failed. The deeper the level you wait at, the more your fills are
selected against you.
Practical consequence: the volume profile is real (levels beat distance-matched
placebos at z +3 to +7.8) but using it to LOCATE ENTRIES makes this trade
worse, not better.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
79f108ab84 |
research: RETRACTION - the retail-fade edge was a fill-timing artifact
The +0.097 R EURUSD result in |
||
|
AnimateDread
|
821f16df8b |
research: model-selected retail fades hold up out of sample (after killing a big leak)
Reframes what the net is asked. Direction is the one thing the data says is barely predictable; but the fade of retail pin/inside setups has a MEASURED gross edge of ~0.139 R and a cost of spread/stop that varies 5x across instruments, sessions and vol regimes. So the answerable question is selection: spend the edge only where it survives. label realised R of the fade (what actually pays) features 28, all knowable at entry, incl. the volume-profile levels split CHRONOLOGICAL train/val/test; threshold picked on val, frozen for test TEST BLOCK 2022-08 .. 2026-07, 63,901 candidate trades, keep top 10%: mean R +0.1005 vs -0.0616 for taking everything by year +0.140 +0.075 +0.098 +0.105 +0.105 -> 5/5 positive with the COST FEATURE REMOVED: +0.0800, still 5/5 positive That last control matters: the model is not merely learning 'skip wide spreads'. Something in the setup geometry, session and level structure carries signal beyond the cost. HONEST SIGNIFICANCE. The naive t of +8.06 is not believable - with an 8-day horizon these trades overlap heavily and thousands share one price path. On a strictly non-overlapping subset (163 independent trades) it is +0.123 R at t +1.61; without the cost feature, +0.257 at t +3.45. The overlap filter applies the H1 horizon to M15 trades too, so 163 is a conservative floor and the true independent count is higher. Suggestive, not settled. THE LEAK THIS RUN NEARLY SHIPPED. First version scored +0.53 R on the held-out block, t +54. Bar-derived features were read at the FILL bar i2, but the order fills intrabar and the outcome race starts at the first M5 bar inside i2 - so i2's close, tick count and realised volatility are not knowable at entry. The model was seeing how the bar it entered on turned out. Second lookahead of this hunt (the first was worth +0.15 R in the sweep test). RULE: a clean chronological split does NOT protect against lookahead. The split was honest and the features were not. Any feature indexed at the entry bar must be re-derived from the bar before it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
c9b489e5d8 |
research: the retail fade DOES clear cost on EURUSD - correcting the earlier verdict
The previous commit pooled four instruments with very different spread-to-stop ratios and concluded the edge never beats the cost. That was too broad. Per cell (48 cells, M5 path, k=1), 8 clear - and they are not scattered: EURUSD H1 pin spread/stop 0.042 edge +0.160 cost 0.063 -> +0.097 R EURUSD H1 pin 0.045 +0.127 0.067 +0.060 EURUSD H1 inside 0.047 +0.123 0.060 +0.063 EURUSD H1 inside 0.050 +0.102 0.062 +0.040 EURUSD M15 pin 0.074 +0.141 0.100 +0.042 EURUSD M15 pin 0.072 +0.140 0.099 +0.041 Every clearing cell is on the tightest-spread instrument. XAUUSD carries the same gross edge (+0.09 to +0.135) and never clears, because its cost is 3x. That is the mechanism predicting where the effect should survive and being right - the opposite of the stop-run case, which inverted. WALK-FORWARD, 4 chronological folds: 6 of 8 hold at >=3/4. EURUSD H1 pin short side is +0.116 / +0.061 / +0.143 / +0.067 across 23 years, 4/4. WIDENING THE STOP still says what it said: EURUSD H1 pin goes +0.078 (m=1) -> +0.028 -> +0.009 -> +0.017 -> -0.003 (m=5). The gross edge collapses ~15x while the stop widens 5x, so this is NOT drift - it is reversion inside roughly one setup-risk of a stop order filled at a local extreme. It is only tradeable at the tight stop, which is exactly where cost bites hardest. WHAT IS NOT MODELLED, and it decides this: commission and stop slippage. Gross edge is ~0.139 R = ~2.4 pips on a 17.3-pip stop, against 0.75 pips of spread. That leaves ~1.6 pips of headroom for commission plus slippage before it is gone. A demo forward test measuring both is the next step, not more history. Also fixes a LOOKAHEAD found in the sweep-entry test: the protective stop was anchored to the low of the very bar that filled the limit order, which is not known until that bar closes. It was worth ~+0.15 R - larger than any real effect here - and it inflated the placebo equally, which is how it was caught. With it removed, buying at retail stop levels is no better than buying at an arbitrary level the same distance away: the 'stops are a farmable magnet' claim fails its own control. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
c2dd9eb6aa |
research: retail setups ARE anti-predictive - and the edge dies with the cost
Tests the user's thesis directly: if price is unpredictable, trade against the
people predicting it badly. Implements the three mechanical setups from 'How To
Day Trade Forex For Profit' ch.5 with their DOCUMENTED stop rules, so retail
stops are located exactly rather than by proxy.
THE MIRROR TEST. Retail's trade and its exact mirror, priced under identical
rules. Both sides pay the same spread and suffer the same same-bar tie
convention, so those cancel in the difference and double in the sum:
edge = (mirror - retail)/2 cost = -(mirror + retail)/2
pin EDGE +0.108 R COST +0.143 R
inside EDGE +0.068 R COST +0.140 R
engulf EDGE -0.001 R COST +0.095 R
So pin-bar and inside-bar setups really are anti-predictive - the first
confirmed directional edge in this project. Engulfing is a pure coin flip whose
loss is entirely the spread, i.e. money already gone to the broker.
Stable across three conventions: H1 bars pessimistic ties, M5 path pessimistic,
M5 path optimistic. Re-walking the barriers on M5 CUT the cost (0.195 -> 0.143)
and RAISED the edge (0.078 -> 0.108), so the coarse-bar convention was masking
the effect, not manufacturing it.
THEN THE TEST THAT KILLS IT. Cost in R is spread/stop-distance, so widening the
stop divides it. If the edge is directional drift it survives. Fade expR by stop
multiple (pin, k=1, 122k trades):
m=1.0 cost 0.146 expR -0.045 implied edge +0.101
m=1.5 cost 0.097 expR -0.067 +0.030
m=2.0 cost 0.073 expR -0.065 +0.008
m=3.0 cost 0.049 expR -0.051 -0.002
m=5.0 cost 0.029 expR -0.040 -0.011
The edge decays exactly as fast as the cost, then inverts. It was never drift:
it is reversion against a stop order filled AT a local extreme, and it lives
within one bar-range of the entry - the same short-horizon reversal the tick-flow
work already measured, meeting the same fate.
Also in this commit, the volume-profile claims from Wyckoff 2.0:
MAGNET all 10 tests positive vs a distance-matched placebo, z +3.0 to +7.8,
family-wise bar 2.79 - but the effect is +0.15 to +0.59pp on a ~74%
base rate, i.e. ~0.01 R.
REACTION naked VPOC and VPOC reject +0.71 to +0.86pp (z to +5.65); value-area
edges null or negative; HVN/LVN marginal.
80% RULE dead. 27.8% traversal against a 29.0% martingale benchmark. Acceptance
nearly DOUBLES the raw rate (14.1% -> 27.8%) and the benchmark doubles
with it - every bit of the apparent improvement is geometry.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
8e6525d819 |
research: Wyckoff's law of cause and effect is real but SUBLINEAR
The 1:1 range projection is the target rule both books recommend (book 1 ch.8 discards point-and-figure counting as too subjective and keeps the vertical projection). Tested as a complete trade on 4 symbols x M15/H1: enter on the range breakout, stop at the far side of the range, target k x risk. A driftless market gives P(win) = 1/(1+k) and expR = 0 at EVERY k, so the benchmark here is analytic - no permutation null needed. Result: expR sits on that benchmark at every k on every symbol. Target placement does not move expectancy, which is what a martingale already said. But the law itself is measurable, and it is not 1:1. Regressing log(MFE) on log(range height) with log(ATR) as a FREE regressor (a shared ATR denominator correlates the errors and biases the exponent towards the hypothesis, so it cannot be used to argue against it): b = 0.10 .. 0.92, centred ~0.6 b = 1 rejected in 5 of 8 at >2sd, never significantly above 1 b = 0 rejected in 7 of 8 So a bigger cause does produce a bigger effect - sub-proportionally. The 1:1 projection systematically over-reaches after a large consolidation and under-reaches after a small one. Median travel in risk-multiples falls monotonically across height quartiles in 8 of 8 runs. Also adds the volume-profile machinery the second book is built on and which nothing in the EA has: tick-level volume-at-price on a fixed absolute grid, per-session VPOC/value area by the standard Market Profile walk, naked VPOCs, and HVN/LVN from a rolling causal composite. Two biases are left in deliberately, both against the hypothesis: a bar spanning stop and target books the loss, and spread is charged on entry and both barriers. Unresolved trades are marked to market at the horizon rather than discarded - discarding them deletes slow winners and manufactures a false deficit at large k. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
6f7aa4f1e8 |
research: the stop-run edge fails its own mechanism test - disconfirmed
The statistics had cleared a family-wise bar (z to +6.56) and a split-half. Both necessary, neither sufficient: thorough enough mining passes both. What mining cannot do is obey a mechanism it was never fitted to - so the decisive test is whether the effect appears WHERE THE THEORY SAYS IT MUST. Osler's stop-clustering predicts the edge concentrates where the stop reservoir is deepest and fresh forced orders arrive: London open and the London/NY overlap. Measured, by session in real UTC (broker is UTC+2): EURUSD Sydney/Tokyo +0.197* London open -0.063 London -0.156 USDJPY Rollover +0.233 London open -0.223 XAUUSD Sydney/Tokyo +0.070 London open -0.345 LDN/NY -0.185 SP500 Rollover +0.206 NY afternoon -0.098 Exactly inverted. The liquid sessions where stops actually cluster are the worst on every instrument; what remains lives in Sydney/Tokyo and rollover - the THINNEST hours, where fewest stops sit. That is not the mechanism, and thin hours are also where spreads are widest and fills worst, so even the surviving fragment points away from tradeability rather than toward it. Walk-forward by quarter agrees, and shows what the two-way split was hiding: EURUSD 2/4 positive (Q1 -0.029, Q2 +0.062, Q3 -0.010, Q4 +0.089) USDJPY 1/4 XAUUSD 0/4 SP500 2/4 No instrument reaches 3/4. The split-half HOLDS was Q2+Q4 carrying Q1+Q3. Verdict: not an edge. Recording it as disconfirmed rather than leaving an encouraging half-result in the log, because the next person to read this - me - would otherwise build on it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
9ca0ffcc5c |
research: stop-run/liquidity sweep - the first setup to survive everything
Osler's currency-order-flow work is the published mechanism: stop-loss orders cluster just beyond recent swing extremes and cascade price; take-profits cluster and reverse it. So the claim is not "a pattern repeats" but "there is a reservoir of forced orders at a location computable in advance" - which is falsifiable in a way chart patterns are not. Tested as a COMPLETE trade rather than a signal with barriers bolted on: entry, stop and target all come from one structure. Price takes out the N-bar extreme MARGINALLY (<= over*ATR), closes back inside, enter the opposite way next open, stop just beyond the sweep extreme (where the liquidity actually was), target a multiple of that risk. 19 of 24 configs clear a family-wise max-statistic bar on EURUSD/USDJPY H1, all in the predicted direction, z to +6.56. R=1 configs are negative and R=2/R=3 turn positive, which is coherent: the edge is directional and a tight stop pays the spread as a large fraction of risk, so it needs a big R to clear. SPLIT-HALF then kills most of it, as it should: EURUSD N=50 ov=0.5 R=3 +0.013 / +0.042 HOLDS EURUSD N=20 ov=0.5 R=3 +0.014 / +0.034 HOLDS every R=2 config one half negative USDJPY one half negative Surviving configs are STRONGER in the second half, the opposite of a mined artifact decaying out of sample. But N=20 and N=50 overlap heavily and are not independent, so this is one instrument and one R - a lead, not a system. dukas.py: direct Dukascopy datafeed client. SQX mirrors through its own CDN (CdnCache/CdnDownloadJob) so there is nothing reusable there. Dukascopy publishes the raw feed - bi5, raw LZMA, 20-byte big-endian records, ZERO-BASED MONTH in the URL (fails silently into the wrong month otherwise). Cached, resumable, bounded concurrency. Two corrections it forced, per the user: SQX conforms Dukascopy data to the5ers' broker profile AND timestamps. Measured empirically, broker time = UTC+2 (EET), clean minimum. So (1) previously reported "hours" are BROKER time - gold's hour 1 is 23:00 UTC, the daily rollover and COMEX Globex reopen, a real mechanism; and (2) Dukascopy's raw 0.2-pip ECN spread must NOT be used for cost - the5ers' ~0.47 is what is actually paid, so the existing cost analysis was right and Dukascopy would have made every result look falsely tradeable. Its value is the bid/ask VOLUMES, which SQX lacks entirely - true signed flow instead of the event-count proxy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
05c6a5c484 |
research: four more hypothesis families - drift is real, timing still is not
Everything tested before this asked ONE question - can recent price or flow
predict the next bar's direction - and answered no four ways. These are different
families, each with a published prior rather than a hunch.
1 TIME-SERIES MOMENTUM (Moskowitz/Ooi/Pedersen). 34 configurations across 4
symbols x D1/H4 x 6 lookbacks. Nothing. The one rule that looks strong -
XAUUSD H4 250-bar, p=0.0038, t+3.30, +10.32%/yr - returns essentially exactly
buy-and-hold's +10.36%. It is not timing gold, it is being long gold. Hence the
vs-B&H column: on a drifting asset a rule that is merely long most of the time
looks skilful and is not.
2 SEASONALITY. The first thing in this project to survive a properly controlled
test: 5 of 8 clear a family-wise max-statistic bar, two at p=0.0002. Split-half
kills two of them (USDJPY dow-6 n=116 and SP500 hour-0 n=533 are thin
off-session buckets). Two HOLD with near-identical halves:
XAUUSD hour 1 +2.29 bp (t+6.44) / +2.43 bp (t+5.53)
EURUSD hour 13 -1.47 bp (t-6.80) / -0.64 bp (t-3.56)
Gold's hour 1 alone carries more than half the +4.22 bp/day drift.
And it is still not tradeable. Widening the window to amortise the 4.92 bp
round trip: the best of 144 windows (hour 1, 8h) nets +0.14 bp/day, t +0.23,
and splits +1.29 / -1.00 - the sign flips between halves. Every other window is
negative. Real, stable, well measured, and about 2x too small to cross its own
spread. Same shape as the flow result.
3 OVERNIGHT/INTRADAY - folded into the hour analysis above.
4 VOLATILITY-MANAGED DRIFT (Moreira/Muir) - the one needing no directional edge.
Does NOT reproduce here: flat on gold (-0.01), and it HURTS SP500 (0.71 -> 0.41
D1, 0.77 -> 0.54 H4). Honest negative against a strong prior.
What survives all of it is drift, which is large and significant while every
timing rule is noise: XAUUSD +10.24%/yr (t 2.88), SP500 +12.25%/yr (t 2.84),
against USDJPY +1.16%/yr (t 0.59) and EURUSD ~0.
resample.py gains D1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
50e1c7ef84 |
research: flow effect and spread cost decay together and never cross
resample.py composes M5 bars into M15/H1/H4 exactly - every column this pipeline produces is composable (sums sum, maxes max, OHLC nests, means re-weight by tick count), so this costs seconds instead of another 37-minute decode per timeframe. Asserts tick conservation and extreme preservation on every output. Motivation: ATR grows ~sqrt(time) while the spread does not, so spread/ATR should fall with timeframe and make a small edge affordable. It does, monotonically, and the measurement is clean (EURUSD, 1:1 barriers): M5 spread 0.099 ATR random wins 36.8% cost 13.2pp M15 0.057 39.7% 10.3pp H1 0.029 43.6% 6.4pp H4 0.015 47.6% 2.4pp But the signal decays at the same rate. Rows clearing the family-wise bar: M5 many, z to -10.1 M15 many, z to -5.8 H1 2 of 9, one POSITIVE and one negative - the shape of noise, not signal H4 none So the effect lives where the cost is fatal and is gone where the cost is affordable. They never cross. Also added --cheap=Q, which trades only the lowest-Q quantile of spread/ATR. This is the one honest use of an unsigned feature: it cannot point a direction but it can decline to trade, and both terms are known before entry. It does cut cost (EURUSD M15 10.3 -> 7.6pp, USDJPY 12.7 -> 6.4pp) and the effect does not survive there either - nothing clears the bar. test_flow.py gained --tf= and keeps the z-score window at ~1 day on every timeframe rather than a fixed 288 bars. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
f83425aad8 |
research: tick-flow verdict across 4 instruments - real reversal, untradeable
1.93 BILLION ticks -> 5.5M M5 bars (EURUSD/USDJPY/XAUUSD 2003-2026, SP500 2011-2026). Sequential non-overlapping trades, triple barriers, per-bar spread, direction-permutation null with a family-wise max-statistic bar. Order flow is genuinely ANTI-predictive at M5 - price mildly reverses the prior bar's flow. Same sign on all four instruments, clearing the family-wise bar on three: USDJPY z -10.10 -1.73pp vs null EURUSD z -7.95 -1.41pp XAUUSD z -5.34 -0.68pp SP500 z -2.60 -0.87pp (does not clear; half the sample) Agrees with the -0.0151 next-bar correlation (vs +0.4961 same-bar, which is the contemporaneous Cont/Kukanov/Stoikov effect and is not edge). And the cost dwarfs it. Random entry at 1 ATR barriers after spread: EURUSD spread 0.099 ATR -> wins 36.8% (13.2pp below the costless 50%) USDJPY 0.154 34.9% (15.1pp) SP500 0.292 26.5% (23.5pp) XAUUSD 0.450 20.0% (30.0pp) Cost rises monotonically with spread/ATR, which is an internal consistency check on the apparatus. A ~1pp effect against 13-30pp of cost is 10-100x short. Reversing does not rescue it - expR_rev is negative in every row of every geometry. Widening the barriers does not either: at 4-8 ATR nothing clears the bar (max |z| 2.51 vs 2.97). The effect lives exactly where the spread is fatal and vanishes where the spread would be affordable, which is what a seconds-to-minutes phenomenon predicts. Adds --narrow/--wide geometry sets so both regimes are reproducible. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
eeacb609b6 |
fix(research): standardise flow test against the EMPIRICAL null, not a costless coin
The first run reported -23pp edges at -75 sigma, which is not a market effect - it is the tell this project has been burned by before (a lookahead, or here a wrong reference, inflates whatever sign it lands on). The give-away was in the output itself: a family-wise 5% bar of |z| > 71.67 where a centred null over 16 tests should sit near 2.5. Random entry was losing almost as badly as the signal. Cause: z and "edge pp" were measured against be = sl/(sl+tp), the break-even of a COSTLESS coin. These barriers charge the spread and book a loss when a single bar spans both levels, so random entry at 1 ATR on M5 wins ~36.8%, not 50%. The table was reporting the fixed cost of trading as if it were signal. Now every row shows the empirical null win rate, the gap against it, and z standardised by the null's own spread. Family-wise bar drops to 2.95 and the result becomes legible: order flow is genuinely ANTI-predictive at M5, about 1pp below random at z -5 to -8, clearing the bar in 12 of 16 tests and reproducing across three geometries and two independent signal families. It agrees with the -0.0151 next-bar correlation. It is also untradeable, which the table now says out loud: the spread is 0.099 ATR and costs 13pp of win rate against a 1pp effect. Reversing does not rescue it - expR_rev is reported per row and stays negative everywhere. Added a footer stating that beating the null is necessary but NOT sufficient; only exp R > 0 makes money. Also: - permutation null was allocating a single (nperm x nT) array, ~2 GB at these trade counts. Now batched. - null permutes the OBSERVED directions instead of flipping a fair coin, so a directionally skewed rule on a trending instrument cannot pass on drift alone. - timeouts reported separately rather than silently booked as stop-outs. - calibration falls back to a midpoint sample when the tick history predates the MT5 reference series (XAUUSD ticks start 2003-05-05, its H1 export 2004-06-11, so the head sample overlapped by nothing). 2M ticks, because the sample must span >=50 reference HOURS - 200k ticks of modern gold is nine. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
91d67db737 |
perf(research): parallel tick decode, 12x, plus two correctness fixes
The SQX decoder is a per-record Python loop and cannot be vectorised - record LENGTH depends on the config nibbles, so record k+1's offset is unknowable without parsing record k. It therefore saturated exactly one core: 10% CPU on a 12-core box, ~3h for the four files. But the format is randomly seekable. Every BLOCK_LENGTH records SQX restates all four fields as absolute int64s, so byte ranges beginning at block headers decode with no shared history. split_offsets() cuts a file on those boundaries and decode_iter() gained start/stop. EURUSD: 55 min -> 9.4 min, 94% CPU. find_block() will not trust a bare MAGIC match: 0x00..0x0e is a byte run that delta payloads produce by coincidence, so a candidate is accepted only when the next header downstream carries the next sequential block index. Verified equal, not assumed equal: the same 315MB span decoded serially and in 6 chunks gives identical tick counts (31,056,000), identical bar counts (206,316) and identical OHLC. The only divergence is the documented seam artifact - the first tick of a chunk has no predecessor so its delta counts as zero, bounded at workers-1 ticks in 513M (~2e-8). Two fixes this shook out: - The feed is not perfectly time-ordered. EURUSD carries 2 backward steps in 513,494,303 ticks, both under an hour, both in 2003-2006. Bucketing is by absolute timestamp so every tick still lands in its true bar; the symptom is a bucket emitted twice out of order. finalise() now stable-sorts before the duplicate merge. The ordering assert is kept but keyed to MAGNITUDE, since a real chunking bug displaces a large fraction of rows and feed noise displaces a handful - only one of those is safe to continue past. - Chunk workers return undivided sums; means are divided once globally. Dividing per chunk would weight a straddling bar's mean-of-means wrong. test_flow.py: charge the PER-BAR spread instead of a single median across 2003-2026 - FX spreads narrowed by roughly an order of magnitude over that span, so one median charges modern cost to the 2000s and vice versa. Timeouts are now reported separately rather than silently booked as stop-outs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
fac9f620b0 |
research: harness for the M5 order-flow edge test
Ready ahead of the bar build so the analysis runs the moment EURUSD lands. Tests the two SIGNED features the tick pipeline can produce - tick-rule imbalance and event-count OFI - as entry triggers at 1:1/12, 1:2/24 and 2:3/48 bar geometries. Everything else the pipeline computes is unsigned and cannot point a trade however well it measures. Same discipline as every other test here: signal on bar i, entry at the OPEN of i+1, barriers scanned forward only, sequential NON-OVERLAPPING trades so each is independent (skipping that is what produced a fake +2.66pp at 2.9 sigma earlier in this project), break-even == chance by the gambler's-ruin identity, spread charged inside the barrier, and a sign-flip null taking the max over the whole family for the family-wise bar. The prior is written into the docstring before any result exists: OFI is established as a CONTEMPORANEOUS explainer whose predictive power decays within seconds, and it reproduced that here at +0.56 against the same-bar return. So the expectation is that it explains the bar it is measured in and says nothing about the next. What is actually being tested is the gap between that literature - equities, sub-second, size-weighted book data - and this setting: retail FX CFD feed, 5-minute bars, event counts without sizes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
e7a9fe22a7 |
research: signed order-flow imbalance instead of bare quote-move counts
Replaces bidmoves/askmoves with bid_up, bid_dn, ask_up, ask_dn. A bid ticking UP and an ask ticking DOWN both mean buy-side pressure, and a counter that only records "the bid changed" cannot tell them apart - it throws away the direction, which is the only part that could ever point a trade. This is order-flow imbalance in the Cont/Kukanov/Stoikov sense, in its event-count form; the feed carries no sizes so it cannot be size-weighted. Caught before the 3-hour build rather than after, which was the point of smoke-testing on a bounded sample first. Verified against the previous column set on the same 3M ticks: 21,971 bars, ticks/bar 76, up 38, dn 38, spread 0.000126, rvol 4.800e-07, gaps 3.95/31.4 - all identical - and bid_up+bid_dn reproduces the old bidmoves count of 74 exactly, as it must. The orientation check that matters: OFI correlates +0.56 with the SAME-bar return. That is the contemporaneous signature the literature reports, and it is also the cheapest guard against the failure mode that would otherwise pass silently - a sign flip would read -0.56 and every downstream test would then be fitting the negative of the intended feature. Expectations set in the docstring rather than discovered later: OFI is well established as a contemporaneous EXPLAINER of price change and its predictive power decays within seconds. At M5 with multi-hour horizons the prior should be that it explains the bar it is measured in, not the next one. Measuring it anyway is the point - but a +0.56 contemporaneous correlation is not evidence of an edge and must not be reported as one. Merge/mean bookkeeping is now index-driven off COLUMNS instead of positional, so adding a feature cannot silently mis-merge a bar that straddles a batch boundary. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
0526f066ec |
research: stream tick files into bars with microstructure features
sqx.decode_iter() turns the decoder into a generator, and ticks_to_bars.py reduces a
symbol to bars in one bounded-memory pass. Necessary rather than tidy: EURUSD is ~458M
ticks, which is ~15 GB held as arrays, so nothing downstream can take the raw stream.
23 years collapses to ~2.4M M5 bars that every test can load instantly.
Aggregation is vectorised with reduceat rather than looping per tick. The only real
complexity is that a bar can straddle a batch boundary, so the last partial bar of each
batch is carried and merged into the first of the next; per-tick deltas are likewise seeded
from the previous batch's final tick, so the first tick of a batch is not silently treated
as having no predecessor. Verified against the per-tick implementation it replaces: 21,971
bars either way, and every reported median identical to the digit (ticks/bar 76, up 38, dn
38, bidmoves 74, askmoves 74, spread 0.000126/0.000250, rvol 4.800e-07, gaps 3.95/31.4).
Throughput 132k ticks/s, at which point the decoder itself is the bottleneck and the
aggregation costs ~12%.
Features are chosen by what the feed can honestly support. It carries (time, bid, ask,
volume) and no trade direction - SQX's record has one volume field and MT5's
TICK_FLAG_BUY/SELL are empty on FX - so true signed order flow does not exist here and is
not synthesised under a flattering name. What is available:
tick rule up/down mid-price changes; the standard Lee-Ready fallback
quote asymmetry bid updates vs ask updates - which side is being repriced harder
arrival rate inter-tick gaps, mean and max; urgency rather than size
realised variance sum of squared mid returns, a far better volatility estimate than
the bar range and only obtainable from ticks
spread mean and max within the bar
Of these only the tick rule and quote asymmetry can point a direction; the rest are
unsigned, like every feature that has measured above noise in this project so far.
Bars are stamped by the OPEN of their interval and built only from ticks inside it, so no
bar's features depend on a tick after it closes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
ef126845f8 |
research: decode StrategyQuant .dat tick files
Reads SQX tick format 4.2 in pure Python. Derived from SQX's own writer, disassembled out
of internal/libs/SQDataLib.jar (TickDataWriter, NewDataFormat{,Writter}) with the javap
bundled in the install - so this follows the format as specified rather than as guessed.
Why, when Scripts/ExportTicks.mq5 pulls the same four fields from MT5: DEPTH. The broker's
MT5 tick history covers a few years; this file starts 2011-09-19. Sample size has been the
binding constraint on every question in this project - the H1/128-bar setup yields ~300
independent trades in 18 years, enough to resolve only a +6pp edge - so 15 years of ticks
is worth a decoder.
Format: four writeUTF strings, ten zero bytes, one more writeUTF, then records of
(time, ASK, BID, volume) - ask before bid, and the writer swaps them when bid>ask so ask is
always the larger. Every BLOCK_LENGTH=1000 records: MAGIC (15 bytes 0x00..0x0e) + int32
block index + config + four raw int64s. In between, deltas against the previous record.
Config is two bytes = four nibbles laid out high-first, nibble = (logicType << 2) |
dataType, where dataType 0..3 selects a 1/2/4/8-byte payload by magnitude and logicType
supplies the sign (MINUS/PLUS carry unsigned magnitudes; ASIS is a plain signed read).
The scale is the one thing NOT in the file. SQX keeps `decimals` in external metadata, and
1216010000 is equally plausible at 10^3, 10^5 or 10^6 - nothing in the bytes distinguishes
them. Guessing would be precisely the silent, plausible-looking error this project keeps
getting caught by: a 100x price scale error crashes nothing, it just quietly rescales every
ATR-normalised feature downstream. So calibrate_decimals() matches against a known
reference series instead. Against the MT5 SP500 H1 export the answer is not marginal:
decimals=5 median rel.err 9.001630
decimals=6 median rel.err 0.004078 <-
decimals=7 median rel.err 0.899984
Verified on 4M ticks: strictly monotonic timestamps, zero negative spreads, price range
1118.03-2048.38 over 2011-09 to 2014-11 (correct for SP500), median spread 0.43 (matches
this broker's H1 record). The 0.41% residual is the expected artefact of comparing a tick
ask against the nearest H1 bar close.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
4881ad22b5 |
research: measure the spread channel, and separate its real signal from its own cost model
Spread is the one microstructure channel that survived the API audit - FX-available, and
genuinely historical in the tester ("During testing, the spread is not modeled but is taken
from historical data"), unlike swap (no history), signed tick flow (empty on FX) or depth
of market (absent on retail FX, never replayed).
test_spread.py measures four spread features against the triple-barrier label with the same
block-permutation null as test_volume.py. spr/atr - cost relative to the volatility a trade
must overcome - is the strongest reading anywhere in this project so far: significant on 5
of 8 instrument/geometry cells and 2-4x the magnitude of any volume feature.
Which immediately looked too good, because the label is computed WITH the spread charged
inside the barriers. A bar with high spread/ATR has its barriers shifted more adversely and
is mechanically likelier to resolve as a loss - so the feature would partly predict its own
cost model, which is not tradeable information.
Tested directly by relabelling at zero cost and re-measuring the identical feature:
EURUSD 2:3 +0.000655 -> +0.000485 (p 0.030 -> 0.066, loses significance)
EURUSD 1:2 +0.000626 -> +0.000399 (p 0.003 -> 0.017)
USDJPY 2:3 +0.000418 -> +0.000164 (never significant either way)
USDJPY 1:2 +0.000876 -> +0.000532 (p 0.003 -> 0.003)
XAUUSD 2:3 +0.000769 -> +0.000744 (p 0.027 -> 0.027)
XAUUSD 1:2 +0.000876 -> +0.000698 (p 0.003 -> 0.003)
So roughly 20-40% of it WAS the tautology, and the majority is not. What remains is a
volatility-regime reading: spread is near-fixed while ATR is not, so spr/atr is high
exactly when realised volatility is running below its own ATR estimate - which genuinely
predicts whether ATR-scaled barriers get reached at all.
Note what that does and does not buy. Like volume, spread is UNSIGNED: it informs Neutral
vs directional, never Buy vs Sell. It is the best-measured feature in this project and it
still cannot pick a side.
No EA changes in this commit - measurement only, and the MQL5 side already has an
uncompiled backlog.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
d80d9444a5 |
feat(ai): widen the volume feature block from 1 value to 4
The block fed exactly one number: (v[i] - v[i-1]) / v[i-1]. That is the first difference, and it cannot express three things that matter - the LEVEL relative to a baseline (two dead bars and two frantic bars both read ~0 change), and the two volume-vs-range interactions, where heavy participation that went NOWHERE (absorption) and heavy participation that travelled (continuation) mean opposite things and currently collapse onto the same value. research/test_volume.py measures each candidate's mutual information with the triple- barrier label across 3 instruments x 2 geometries, against a BLOCK-permutation null - blocks sized to the barrier horizon, because adjacent labels share almost their entire outcome window and a free shuffle yields a null so tight that everything looks significant. Finite-sample MI bias (~7/n here) is reported alongside rather than subtracted, since the permutation null already absorbs it. Result: volLevel beats the shipped change ratio outright on 4 of 6 cells (EURUSD 2:3 +0.000118 excess at p=0.006, USDJPY 1:2 +0.000284 at p=0.002); absorption is the single strongest reading anywhere in the sweep at EURUSD 1:2 (+0.000404, p=0.002) though it is null on XAUUSD; vol x range clears on 4 of 6. The shipped change ratio is itself significant on 5 of 6, so it stays. Kept OUT: a session-relative z-score against the same hour-of-day's own recent history. It was the weakest candidate - null on both EURUSD cells - and it is the only one needing per-hour rolling bookkeeping in MQL5. Not worth the state for a reading that did not survive its own null on the primary instrument. Magnitudes, stated plainly because they are the point: the excess MI is ~2e-4 nats against a label entropy near 1.05. That is under a tenth of one percent of the label's uncertainty. It is real, it repeats across instruments, and it is nowhere near an edge - this is worth having because it costs one 50-bar loop, not because it changes the answer. Prior work stands: the whole single-series feature family measured at the noise floor. m_neuronsCount is already in the fingerprint, so the width change re-keys existing caches by itself, which is correct - the input vector genuinely changed shape. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
8710240cd5 |
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
ae738f59f0 |
chore(research): drop committed __pycache__, add .gitignore
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
bd076ddbad |
research: test the shipped classic patterns for entry edge - and the lookahead that faked one
Transcribes all 26 classic vote models (MA 4, RSI 4, MACD 6, Ichimoku 12) from
Signals/*.mqh into vectorised Python, with their shipped constructor weights, then tests
them as entry triggers on 178k-bar FX histories.
Pre-registered by construction: the rules were written long before this test and nothing
about them is fitted here, so there is no in-sample/out-of-sample split to draw and the
whole history is usable. Break-even == chance by the gambler's-ruin identity, so "beats a
coin" and "makes money" are one question. Sequential non-overlapping trades only; a
sign-flip null over the whole pattern family gives the family-wise bar.
The result that matters is a negative one, and it took two lookahead fixes to see:
- _price_extremum reproduced the standard library's CENTRED MinValue(pos-2,5) window,
which reads up to 2 bars newer than the extremum it describes.
- turning_points marked a turn AT bar i, which is only knowable once bar i+1 closes.
Together those two bars of leakage WERE the entire apparent edge. MACD_p4 on EURUSD 1:2
read +5.05pp at +4.05 sigma before, -0.02pp at -0.02 sigma after; USDJPY 1:2 went +5.24pp
-> +0.02pp. RSI_p2's large NEGATIVE went the same way (-10.33pp -> -1.34pp), which is the
tell: a leak inflates whatever sign it lands on.
With both closed, across 4 instruments x 3 geometries: no pattern, no vote threshold, no
quorum and no event+confirmation rule separates from chance. One cell in ~180 tests stars
(SP500 2:6 vote>=30) and it is non-monotone in the threshold either side of the hit.
test_exits.py answers the trade-management half with the control that makes it mean
something: hold entries fixed, vary only the exit, and run every rule again on RANDOM
entries at the same bars. Breakeven-at-1R, chandelier trails, partials and time stops all
move E[R] - and move it by the same amount on random entries. No rule beats its own
control (max +0.99 sigma over 32 comparisons). Management reshapes the win-rate/payoff
split; it does not manufacture expectancy from a directionless entry.
Residual E[R] across every cell is -0.01 to -0.08 R, which is approximately the spread.
Incidental, both worth fixing in the EA:
- CSignalMA pattern 1 is unsatisfiable at the shipped EMA default. For an EMA,
MA[i]-MA[i-1] and c[i]-MA[i] are both positive multiples of (c[i]-MA[i-1]), so
"close below the MA while the MA rises" cannot occur. Dead code (weight 10).
- Ichimoku pattern 11 (Sanyaku, weight 100, the method's top signal) fires on 27% of
bars because it is a conjunction of three standing STATES with no event term, so it
dominates the averaged vote while carrying no trigger information.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
b6a067b266 |
research: offline validation kit, and the answer it produced
Keeps the Python that turned a per-hypothesis cost of minutes into
seconds, so the next person (or the next me) can re-run any of this
without MetaTrader in the loop.
kit.py vectorised triple-barrier labeller + scale-free feature set.
The labeller is a faithful port - stop tested before target
within a bar, so a bar spanning both scores as the loss -
and reproduces the EA's own distribution to 0.05pp
(24.93/22.01/53.06 vs 24.9/22.0/53.1) at 12x the speed.
wf.py purged, embargoed walk-forward gradient boosting.
sim.py sequential NON-OVERLAPPING trade simulation.
sweep.py the cell x geometry sweep.
detail.py full threshold profile for one configuration.
WHAT IT FOUND, and the order matters because the first answer was wrong:
Naive walk-forward looked like an edge - precision rising monotonically
with model confidence, 24.4/24.6/25.3/26.2/27.1%, topping out at +2.66pp
and 2.9 sigma. All of it pseudo-replication: a 128-bar barrier means
adjacent bars share almost their whole outcome window, so one trade was
being counted up to 128 times. Counting each trade ONCE (sim.py) the
ordering collapses to 25.9/31.1/26.8/28.1/25.8 and nothing is
significant. Same error family as the MI null that assumed independence.
That exposed a structural problem bigger than the result: at a 128-bar
horizon, 18 years of SP500 H1 yields at most ~300 independent trades,
which can only resolve a +6pp edge at 2 sigma. Real edges are 1-3pp. The
shipped configuration is not merely unproven - it is statistically
UNFALSIFIABLE on the available history.
The cost/power screen then showed SP500 is among the worst cells
available: 54k bars and spread/ATR 0.34, against EURUSD/USDJPY at 178k
bars and 0.05. Weeks of training went into the hardest instrument on the
list, 3x less data and 7x the relative cost.
Final sweep - 4 instruments x 3 geometries, purged walk-forward,
independent trades: nothing clears +2 sigma. The one survivor (EURUSD
2:3 h48, +2.76pp at +1.57 sigma) dissolves under its full threshold
profile: non-monotone across thresholds, and its per-fold win rate decays
monotonically through time (52.9 -> 45.9 -> 41.7 -> 35.2 -> 26.2).
Conclusion: with price/volume-derived technical features there is no
tradeable entry-direction edge on these instruments - now tested with a
model class that finds interactions, on 3x the data, with honest
statistics.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|