The sequence rewrite made LSTM/HYBRID recurrences over 20 bars, but every
weight - including the gate biases - is initialized around zero. That puts
the forget gate at sigmoid(0) = 0.5, so the cell state is halved every step:
the first bar survives into the output scaled by ~0.5^20, and the gradient
reaches it scaled by the same factor.
The layer was therefore a one-bar model wearing a 20-bar interface. A one-bar
model has no signal on this task, so the head learned the base rate and
emitted Neutral everywhere - the flat 0.34 IS error across twelve eras and
OOS recall Neutral:100% seen on SP500 H1.
Measured at the shipped H1 shapes (H=64, stepInputs=21, T=20) - influence of
bar 0 on the output relative to bar 19:
bias 0.0 -> 3.0e-05 forward, 3.3e-05 backward (dead)
bias 1.0 -> 1.2e-02 forward, 1.4e-02 backward
bias 2.0 -> 2.5e-01 forward, 2.7e-01 backward (a real 20-bar field)
This is the standard fix, not a tuned knob: Gers/Schmidhuber/Cummins (2000)
introduced the forget gate with a positive bias, and Jozefowicz/Zaremba/
Sutskever (ICML 2015) recommend a bias of 1 as a default (whence Keras'
unit_forget_bias). Both 1 and 2 are standard; the sweep picks 2 because at a
20-step window a bias of 1 still leaves the oldest bar at ~1% influence.
lstm_seq_flowcheck.cpp is added as a permanent regression check and asserts
the shipped constant keeps >=5% reach in both directions. It complements
lstm_seq_gradcheck.cpp: that one proves the BPTT is CORRECT, this one proves
it is USABLE. The gradient check passed at 2.3e-10 throughout - correct math
over a recurrence that carries nothing looks exactly like a bad architecture.
Both builds compile 0 errors, 0 warnings. Initialization only, so the .nnw
format is unchanged; LSTM and HYBRID must retrain to pick it up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
