Warrior_EA/EXPERIMENTS.md

# Training experiment log

Results from multi-chart comparison runs on SP500 H1. Kept because the raw MT5 logs get
wiped between runs and these are the measurements the design decisions rest on.

---

## Run 3 — 2026-07-29 21:06 → 2026-07-30 08:46 (~12h)

**Build:** logit-adjusted loss at uncapped `tau = 1.0`, conv pooling removed, BN on all
charts except the control. **Verdict: the whole run is a write-off.** Kept as the record of
one specific failure.

Six charts, identical config except dense depth and the one `EnableBatchNorm = false`
control (verified from the startup config lines — all six matched on every other field).

| chart | eras | peak bal | median coverage | median precision | eras calling <50% of bars |
|---|---|---|---|---|---|
| CONV_2L `[4109]` | 766 | 64% | **100%** | 6.0% | **0 / 766** |
| MLP_3L `[39c1]` | 485 | 64% | **100%** | 6.0% | **0 / 485** |
| MLP_4L `[d3b2]` | 460 | 64% | **100%** | 6.2% | **0 / 460** |
| HYBRID_2L `[4109]` | 390 | 64% | **100%** | 5.9% | **0 / 390** |
| LSTM_2L `[4109]` | 292 | 64% | **100%** | 5.9% | **0 / 292** |
| MLP_3L no-BN `[271b]` | 49 (stopped) | 59% | 83% | 5.4% | 19 / 49 |

Directional base rate is ~6.1%, so **median precision ≈ the base rate: no information at
all.** Not one era in 1,993 across the five batch-norm charts ever called a direction on
fewer than half of all bars.

### Root cause

`tau = 1.0` against a bounded sigmoid head. Offsets are `tau·log(prior_c)`, spread 3.42 on
this 30:1 imbalance, against a head whose widest expressible logit gap is
`CLASS_LOGIT_SCALE × 1 = 6`. The prior correction consumed **57% of the entire range**, so
the network's only available response was to saturate Buy/Sell to 1.0 to overcome a −3.42
training handicap. The offsets are absent at inference, so every bar came out directional.

**The damage was present at era 1 and permanent** — Neutral recall was 2–7% in the very
first era of every batch-norm chart, and 292–766 eras never recovered it. This was not
drift; the optimization target itself was wrong from the first gradient step.

Fixed in `0e5f1bb` by expressing strength relative to the range actually available.

### What the plateau ladder did — correctly

Dozens of `PLATEAU stage 3 … nothing safe to deploy` cycles. The per-class recall floor
refused every one of these models. Only the no-BN control ever deployed, and only because
it had an early genuinely-balanced checkpoint before it collapsed.

### Caveat on the new selection metric

Precision-led selection (`a142749`) would **not** have rescued this run. Every one of the
1,993 eras was an over-caller; ranking correctly among uniformly bad options still yields a
bad option (best-precision era per chart was 6.5% at ~100% coverage). Selection prevents
*choosing* wrongly when good checkpoints exist. The `tau` cap is what makes good checkpoints
exist at all. They fix different failures and both were needed.

### Caveat on batch norm

This run does **not** support "batch norm is required" — it is confounded. The no-BN control
was the *only* chart that stayed selective (19 of 49 eras below 50% coverage) precisely
because batch norm preserves logit spread into the head, which under a broken `tau` made
saturation *easier*. The real evidence for batch norm is Run 2 below.

---

## Run 2 — 2026-07-29, ~3h

**Build:** minority replay (28×) + post-hoc prior, batch norm, derived topology, conv
pooling still present.

| chart | topology | BN | peak balanced | peak Buy/Sell recall |
|---|---|---|---|---|
| `042e` | MLP_4L | on | **60.7%** — converged, auto-deployed era 58 | 51% / 45% |
| `6ca2` | MLP_3L | on | 57.0% | 54% / 26% |
| `cfcc` | HYBRID_2L | on | 53.0% | 53% / 12% |
| `cfcc` | LSTM_2L | on | 52.3% | 46% / 16% |
| `c3a7` | MLP_3L | **off** | 45.7% | 1% / 42% |
| `cfcc` | CONV_2L | on | 45.3% (510 eras) | 49% / **0%** |

- **Batch norm worth +11.3 points** on identical MLP_3L (57.0 vs 45.7), stable across 150+
  and 200+ eras. Verified which chart was which by parsing `.nnw` layer records: BN-off =
  5 layers, BN-on = 9, MLP_4L = 11. **This is the evidence for batch norm.**
- Depth ordering reversed vs the pre-BN era — deepest now best, closing out the
  over-parameterization diagnosis.
- MLP_4L deployed at balanced 60.7%, blended OOS 83.9%, calling ~6% of bars at 20–35% win
  rate against a ~6% base rate — the only tradeable model produced so far.
- **Only 1 era in 1,301 cleared the ≥40% per-class recall floor.**
- Failure mode was directional mode collapse: each model took one direction to ~50% recall
  and abandoned the other, direction chosen arbitrarily.
- CONV pinned at ~40% for 510 eras with Sell recall 0% → traced to the pooling stage
  reducing across filter channels instead of time (`70cdec2`).

---

## Reference frontier (simulated, base rate 6.1%)

Why selection moved off balanced accuracy — measured at fixed signal strength:

| tau | balanced acc | call rate | precision |
|---|---|---|---|
| 0.00 | 34.0% | 0.2% | **27.3%** |
| 0.15 | 34.2% | 0.2% | **33.3%** |
| 0.35 | 36.3% | 2.0% | 15.5% |
| 1.00 | **53.5%** | 49.6% | **6.4%** |

Balanced accuracy rises monotonically as the model calls more and is right less. Deploy
order inverts from `1.00 > 0.50 > 0.35` under the old metric to `0.35 > 0.50 > 1.00` under
precision-led selection.