1. THE POOL FIX WAS LANDING ON A TOPOLOGY THAT COULD NOT SEE IT.
ComputeFirstLayerWidth budgets against EstimatedInSampleBars, which counts
this chart's own bars PLUS the training pool. On a COLD fleet start every
chart derives and pins its topology BEFORE any chart has published a pool
file - measured on the 18:13 start, model creation at 18:13:21 against a
first publish at 18:13:48. All six sized as if training alone, wrote that
into .cfg, and adopted it back on every later start even with the pool full.
SP500 ran a first layer floored to 16 while adopting 30229 peer rows.
Adopt-don't-compare exists to protect weights shaped by those sizes. It was
also running for a model with NO .nnw, where there is nothing to protect and
the .cfg is just a record of one unlucky moment. The four derived sizes are
now re-measured when no weights exist.
Safe on all three counts that matter: free (nothing to discard), cannot loop
(once weights exist the .cfg is authoritative again), and cannot fragment the
pool - the derived width is NOT in BuildModelFingerprint, which keys only on
the FEATURE layout. Verified: field 2 of the fingerprint is
LEGACY_HISTORY_BARS_SLOT, not the first-layer width.
TO TAKE EFFECT the weights must be wiped while the TrainPool is KEPT - the
census has to be non-empty at derivation time. A full wipe empties the pool
and reproduces the original condition exactly.
2. THE KEEP-SCREEN LATCHED ON AN UNDERPOWERED SAMPLE.
MI_MIN_SAMPLES is a floor for "can this be computed", and it was being used
as the bar for "is this answer final". The screen fired on the first era
clearing 200 rows and latched, measuring at 202-773 samples where a warm
chart gives ~2065. Columns kept then tracked SAMPLE SIZE rather than
information - EURUSD kept 0 of 49 at n=202, SP500 kept 15 at n=773, and the
ordering across all six charts was very nearly monotone in n.
A thin sample is still measured and printed, but it no longer closes the
question: below MI_GOOD_SAMPLE_FRACTION of the target the result is labelled
underpowered and a later era supersedes it, bounded by the same attempt
budget. An underpowered screen that latches is worse than one that waits,
because it looks like a result.
Build tag -> fleet-pool-v2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
TWO CHANGES, BOTH RETRAIN-FORCING BY INTENT.
1. SP500 was training alone, and one alt-data column was the reason.
The alt block's width joins the model fingerprint, and the pool reader only
adopts peer rows whose fingerprint and width match. The exporter gives each
instrument the series that apply to it - FX 15 columns, metals/oil 14, SP500
13 - so the fleet ran as three incompatible pools:
EURUSD/USDJPY/USDCAD adopt ~57-60k peer rows each
XAUUSD/XTIUSD adopt 6.4k / 20.3k
SP500 "EVERY peer file was REJECTED, so this chart is
training alone" - 0 rows
SP500 therefore trained on 2279 independent observations against a 600-wide
input with its first layer floored at 16, printing its own "expect
overfitting" warning. It is the one chart with no pool and the worst
capacity ratio in the fleet by a factor of three.
Fresh models now pin ALTDATA_FLEET_COLUMNS - the 12-column intersection -
instead of their own file header. An existing model still adopts its .cfg
pin, so this re-keys nothing that is already trained.
Intersection rather than union: filling an absent series with its median
makes that column constant per instrument, which lets a pooled model
identify the source instrument and stop learning the shared mechanism. It
is also 6 columns narrower. Cost is six columns whose retained information
is UNMEASURED - the keep-screen reports a bitmask nothing has mapped back
to names.
2. The MI keep-screen disabled itself for the whole run on any cold start.
ReportFeatureLabelInformation set m_miReportDone on ENTRY. On a cold start
the label cache is allocated before it is filled, so BuildMiSample finds no
row carrying a resolved label and returns 0 - a sixth exit, and the only
one the 8c1266d instrumentation did not cover, which is why it printed
nothing. observed then stayed -1, the permutation loop never iterated, and
the report emitted "-1.00000 nats over 0 permutations" beside a plausible
"strongest single feature 0.05979" that was a STALE m_miBestColumn from an
earlier scoring call. The first ensemble member propagated the latch to
g_ensembleChartMiReportDone and silenced every member on the chart.
The flag now latches only once a measurement exists. A short sample is
reported as a deferral naming the two numbers that identify it (cached bars
vs bars carrying a resolved label) and retried, up to
MI_REPORT_MAX_ATTEMPTS.
Build tag -> fleet-pool-v1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The keep-screen answered whether pruning is worth doing - consistently, across
all six charts:
chart kept width first-layer budget
EURUSD 17/52 624 -> 204 11.3 -> 34.3
USDCAD 19/52 624 -> 228 10.0 -> 27.2
USDJPY 18/52 624 -> 216 11.2 -> 32.4
XAUUSD 17/51 612 -> 204 7.8 -> 23.2
SP500 18/50 600 -> 216 3.8 -> 10.5
XTIUSD 16/51 612 -> 192 3.8 -> 12.1
~1 column in 3 carries the association and the rate is stable across six
independent charts - noise would not reproduce that tightly. Pruning nearly
triples the capacity budget and lifts XAUUSD off the 16-wide floor. SP500 and
XTIUSD (the two pool-poor charts) improve ~2.8x and still miss it; they need the
12-bar window cut as well, which is a separate lever costing nothing in feature
semantics and not touching pool compatibility.
Headline MI is strong everywhere under the pivot-event label: 0.008-0.0099 nats
against a ~0.002 null, strongest column 0.047-0.077 against a ~0.006 null-max
(8-13x).
WHAT THIS COMMIT ADDS is the last fact needed before a mask can be built: WHICH
columns, as a hex bitmask, so two charts' masks can be compared by eye and by
grep. Identical masks across the fleet mean ONE fleet-wide mask keeps every chart
in a single pool group; divergent masks would split six charts into six groups of
one, and pooling is the only thing currently holding the FX charts above the
capacity floor - so a per-chart prune could cost more capacity than it buys.
Still report-only. No fingerprint change, no retrain forced.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 1 of the prune, stopping deliberately short of pruning - two blockers make
an immediate mask the wrong move, and this is the measurement that decides
whether pruning is worth doing at all.
WHY NOT PRUNE YET:
* the screen runs with cross-asset ABSENT - its own log line says the numbers
"describe a NARROWER vector than training will use". A mask built from it
would have no evidence either way about the cross-asset block.
* a per-chart mask FRAGMENTS THE POOL. The mask must participate in the
fingerprint, and the pool only accepts peers with an identical feature
layout. Pooling is currently the only thing keeping the FX trio off the
capacity floor - the three pool-poor charts (SP500, XAUUSD, XTIUSD) are
exactly the three still floored. Six per-chart masks = six pool groups of
one, and pruning could cost more capacity than it buys.
WHAT THIS ADDS: the per-column MI was always computed inside ScoreMiSample and
thrown away except for the sum and the max. It is retained now, and the same
permutation draws that build the headline null also accumulate a PER-COLUMN null,
which is what a per-column p-value needs - distinct from the null-of-the-max,
which answers the single family-wise question "is the strongest column real".
Selection uses Benjamini-Hochberg at q=0.10, NOT the family-wise bar. FWER
controls the chance of one false positive, which is right for a verdict and far
too conservative for selection - it would discard every genuinely weak-but-useful
feature. BH bounds the expected SHARE of kept columns that are noise, which is
what a feature set cares about.
The report prints the decision in capacity units: columns kept, the resulting
input width, and the first-layer budget before and after against the 16-wide
floor. 3 of 52 is not a feature set; 45 of 52 is not worth a fingerprint re-key.
The cross-asset caveat prints itself when it applies.
Context that makes this worth doing at all: under the pivot-event label the MI
screen now reads "above the noise floor - a real association" - mean 4x the null
(p=0.005), strongest column 7.7x the null-max, excess 0.80% of label entropy,
against 1.3x / 1.15x / ~0.1% under the old label. The noise-floor verdict that
closed several earlier directions was a property of the OLD label.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The MI suite kept its one irreplaceable job - the label-alignment
lookahead scan, whose margin is priced by the headline permutation
null and whose validity is proven by the positive control. Everything
that judged or vetoed on top of that measurement is gone:
- m_dirEvidence deploy veto deleted from all four deploy sites. The
policy is that screens are priors, not gates; the family-wise
selection test on held-out precision is the deploy protection, and
a marginal per-bar MI test cannot veto a model that reads the
window jointly (the report itself said so on every print).
- Per-column CFeatureSelector deleted; BlockPermuteLabels (the null
engine ScoreMiSample depends on, ragged-tail fix intact) moves to
AutoTune.mqh as a free function.
- Feature-lag profile deleted, with its MI_LAG_* constants and
BuildMiSample's featureBarOffset; MiShiftPad no longer pads by
m_historyBars.
Compile: 0 errors, 0 warnings (stage).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
FeatureScreen.mqh's MI/permutation-null diagnostics (mean/best-col
report, excursion report, lag-profile family-wise test, barrier-
geometry scan) and AutoTune.mqh's TuneIndicatorsByFilter install gate
each spelled out the add-one-smoothed Monte-Carlo p-value
(1+atLeast)/(draws+1) independently. Added PermutationPValue(atLeast,
draws) to System/BinomialStats.mqh (returns 1.0 for draws<=0, matching
every existing call site's own guard) and replaced all six inline
expressions with a call to it. Pure arithmetic substitution, no
control-flow change.
Session C of the feature-selection/labeling refactor track. AutoTune.mqh was
two responsibilities in one 1,584-line file: SEARCH (TuneIndicatorsByFilter,
coordinate-descent over indicator settings) and MEASUREMENT (the "does this
feature vector predict this label at all" evidence screen and its three
sub-reports). Split along that seam into a new Expert/AIBase/FeatureScreen.mqh.
Moved, verbatim (diffed byte-for-byte against the pre-split content - zero
lines differ beyond the file-boundary comment headers): ReportFeatureLabel-
Information, ReportExcursionInformation, ReportFeatureLagProfile, Report-
BarrierGeometryScan, ApplyAdoptedGeometry.
Stayed in AutoTune.mqh: the MI engine (FeatureColumnMI/BuildMiSample/
ScoreMiSample) both files call - a shared dependency used by two consumers is
not itself a reason to split further; TuneIndicatorsByFilter; the export
utilities; and TuneIndicatorsAndTrain, the entry point that decides which of
the two branches a given model runs - it is the coordinator, not a member of
either side.
Still body-only method definitions of CExpertSignalAIBase, same as every
other Expert\AIBase\*.mqh file - MQL5 has no partial classes, so this is a
file-organisation move (legibility, SRP-per-file), not a coupling reduction.
The include site says order between AIBase\*.mqh files is irrelevant, so the
new include was added next to AutoTune.mqh's; the file-scope g_ensembleChart*
globals both files reference stay declared in AutoTune.mqh's header, ahead of
the new include either way.
Verified: brace counts split exactly 105 -> 57+48; every one of the 14
function definitions HEAD had in AutoTune.mqh accounted for in exactly one of
the two files, no duplicates; the WARRIOR_EXPORT_FEATURES ifdef/endif pair
(unrelated, lines 35/147) untouched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
