SniperGold_ML/docs/P3_S25_1D_R_INTERRUPTION_RECOVERY.md

# P3-S25.1D-R — INTERRUPTION / RESUME RECOVERY

```text
Document type : RECOVERY REPORT (governance-adjacent, verification-only)
Phase         : P3-S25.1D-R — Recovery of the failing interruption/resume
                verification of P3-S25.1D
Date          : 2026-08-28
Scope         : ONLY the isolated interruption/resume test. No real-source
                ingestion, no CHUNK 11, no ML, no production changes.
Authoritative evidence : ml/p3/p3_s251_external_ingest/multichunk_benchmark/
                         recovery/output/*.json
```

## 1. Session origin and objective

This session recovers the unresolved interruption/resume verification of
P3-S25.1D. The previous session ended (~2 h 39 m) after it reached its action
limit while investigating the interruption/resume failure; an unresolved
verifier result was recorded and a likely verification-driver defect was
identified. This session isolates that defect, independently verifies the
corrected path, and does NOTHING else.

## 2. Pre-session state audit

- Latest handover read: no `SESSION_HANDOVER_*P3_S25_1D*` file exists in the
  repo (the expected `P3_S25_1D_MULTI_CHUNK_TURN_BENCHMARK.md` /
  `P3_S25_1D_EXECUTION_STRATEGY.md` were never created). The authoritative
  phase reports used were `docs/P3_S25_1D_VERIFICATION_DRIVER_HARDENING.md`
  and `docs/P3_S25_1D_MULTI_CHUNK_INVARIANCE.md`, plus the machine-readable
  manifest below.
- Machine-readable authority for the failure:
  `multichunk_benchmark/output/multichunk_benchmark_manifest.json`
  -> `status.interruption = false`, `all_pass = false`. This is the
  authoritative record of the ORIGINAL failure.
- Working tree at session start: `git rev-parse HEAD` =
  `a52b38eeb4ab74a500f778a028ee8ac7336920c1` == `git ls-remote origin HEAD`;
  branch `main`; remote = `SniperGold_ML.git`; stash EMPTY.
- **Preflight deviation (documented)**: `git status --short` reported
  untracked P3-S25.1D artifacts left by the interrupted previous session:
  `docs/P3_S25_1D_MULTI_CHUNK_INVARIANCE.md`,
  `docs/P3_S25_1D_VERIFICATION_DRIVER_HARDENING.md` and the whole
  `ml/.../multichunk_benchmark/` directory. No tracked file was modified
  (`git diff --stat` empty). These exact artifacts are the subject of this
  recovery and are committed unchanged at close-out; nothing was reset,
  cleaned, stashed, merged, cherry-picked or re-written.

## 3. Real checkpoint immutability (before any test)

```text
path                 : ml/p3/p3_s251_external_ingest/output/s251_real_checkpoint.json
checkpoint_sha_before: 47da8f020426bec67e9f7865519e900a28c286ba1d13a15bee6a8c0f5307afc7
last_completed_chunk : 10
next_chunk_id        : 11   (derived: last_completed_chunk + 1)
next_byte_offset     : 163,578,081  (checkpoint next_byte_start / byte_offset)
```
The checkpoint was NOT modified at any point during this session.

## 4. Original failure (recorded)

- `output/multichunk_benchmark_manifest.json` (run ended 2026-08-28 07:31:10Z):
  - verifier_hardening_8_of_8 : true
  - mutation_12_of_12         : true
  - semantic_invariance       : true
  - **interruption            : false**   <- the failing test
  - batches_pass 1/2/4/6      : true
  - all_pass                  : false
- After the manifest was written, the previous session re-ran the
  interruption test alone at 07:39 and its PASS result overwrote
  `output/multichunk_interruption.json`; the ORIGINAL failing JSON (reported
  mismatch `continuous_run_hash = a4293…` vs `resume_output_hash = 7d7…`)
  is NOT preserved anywhere on disk. The manifest remains the authoritative
  record that the interruption test FAILED. The original 7d7… value cannot
  be recovered from any preserved file.

## 5. Isolation (machine-readable)

`recovery/output/interruption_failure_isolation.json` records:

```text
test_name                    : MULTI_CHUNK_INTERRUPTION_RESUME
failure_type                 : RESUME_BOUNDARY_MISMATCH
continuous_boundary_algorithm: line-safe actual consumed byte_end chaining
                               (pos += len(raw)), identical to production
resume_boundary_algorithm    : OBSOLETE nominal split_atomic 24 MiB multiples
                               (the original defective resume under isolation)
continuous_hash              : a4293a8adeef6cbc5ec464c574b1bc3930568e47a76f1f68234d4fd2787893b7
resume_hash                  : a4293a8adeef6cbc5ec464c574b1bc3930568e47a76f1f68234d4fd2787893b7
boundary_sequences_identical : false
```

### Boundary comparison (continuous vs nominal-boundary resume)

| chunk | continuous byte_start | resumed byte_start | continuous byte_end | resumed byte_end |
|------:|----------------------:|-------------------:|--------------------:|-----------------:|
| 0     | 0                     | 0                  | 25,165,841          | 25,165,841       |
| 1     | 25,165,841            | **25,165,824**     | 50,331,671          | 50,331,671       |
| 2     | 50,331,671            | **50,331,648**     | 75,497,504          | 75,497,504       |
| 3     | 75,497,504            | **75,497,472**     | 100,663,365         | 100,663,320      |
| 4     | 100,663,365           | **100,663,296**    | 125,829,196         | 125,829,152      |
| 5     | 125,829,196           | **125,829,120**    | 150,994,944         | 150,994,944      |

Earliest divergence: chunk 1 — continuous starts at 25,165,841 (the actual
line-safe end of chunk 0) whereas the nominal resume starts at 25,165,824,
re-entering the 17-byte straddle tail of the tick record that chunk 0 had
already consumed. Per the briefing rule (section 6) differing boundary
sequences are sufficient evidence of a verifier/driver defect — recorded.

### Why the original run produced a hash mismatch

Under line-safe reads the re-read 17-byte fragment parses as a malformed
partial line (`column_count`), so the VALID-row hash is unchanged
(`resume_hash == continuous_hash` in this reconstruction). The original
failing run's semantic mismatch (7d7…) is consistent with the defect class
already recorded in `P3_S25_1D_VERIFICATION_DRIVER_HARDENING.md`: chunking at
nominal fixed byte multiples drops each straddling record (the documented
`N_BATCH = N_REF - 5`), and with any carry/offset misalignment. Mutation 7 of
this recovery demonstrates the same semantic class: a 1-byte resume offset
loses one real record (row count 1,689,968 -> 1,689,967, output-hash
mismatch). The canonical invariant is byte-contiguity: the next chunk MUST
start at the actual consumed end of the previous one.

### Smallest reproduction

2 atomic chunks, interruption after chunk 0, nominal-boundary resume:
continuous chunk 1 [25,165,841..50,331,671) vs resumed chunk 1
[25,165,824..50,331,671) -> boundary divergence at the FIRST resumed chunk
(full details in the isolation JSON, `smallest_reproduction`).

## 6. Root cause classification

**A — VERIFIER DEFECT ISOLATED AND FIX VERIFIED** (see §10).

- Root cause is in the DISPOSABLE verification driver (resume boundary
  reconstruction), NOT in the production parser (`s251_parse.py`), NOT in
  the production aggregator (`s251_aggregate.py`), NOT in the real
  checkpoint. No production-equivalent logic defect was found (the byte
  stream itself is processed identically by both paths once boundaries are
  correct).

## 7. Smallest reproducible failure progression

Per the briefing, the failing case was reduced to the smallest reproduction
before any fix verification:

```text
2 atomic chunks  -> boundary divergence reproduced at chunk 1  (STOP, minimal)
```
No larger reproduction needed; the defect is a pure boundary-alignment
defect present at the first resumed chunk.

## 8. Correction (verifier only)

The corrected resume chains every resumed chunk on the ACTUAL line-safe
consumed `byte_end` of the previous chunk and seeds the first resumed chunk
with the DEEP-COPIED output carry of the last completed (trusted) prefix
chunk — exactly the canonical batch-driver algorithm
(`multichunk_benchmark.run_batch`). Implemented fresh in
`recovery/test_interruption_resume_recovery.py`. No production file was
touched.

## 9. Corrected isolated-test result (machine-readable)

`recovery/output/interruption_recovery_result.json`:

```text
corrected_run1 : all_match true (stops 1 and 3)
corrected_run2 : all_match true (stops 1 and 3, fully recomputed)
reproducibility: semantic_outputs_equal_run1_run2 = true
```

For every interrupted stop verified:

```text
trusted prefix              : chunks [0, stop)  (in-flight chunk untrusted)
prefix boundaries == continuous : true
resumed boundary sequence == continuous : true
per-chunk equality (id, byte_start, byte_end, byte_len, rows,
                    m15/m30 counts+hashes, input/output state hashes,
                    final state hash)            : true for every chunk >= stop
carry chain (input/output state hashes)          : ok
aggregate equality (rows, m15, m30, output hash,
                    first/last ts, final state, output carry) : true
```

Key hashes (all runs, all stops):

```text
rows_processed  : 3,379,465
m15_count       : 17,374
m30_count       : 9,315
output_hash     : a4293a8adeef6cbc5ec464c574b1bc3930568e47a76f1f68234d4fd2787893b7
m15_hash        : 9c3c4b85d3c0a4209602f5797083b1d0fa573ab355c17c13ed3327b01ebe1abc
m30_hash        : 320caf981f6d597fc5bcaa393fe8753b7a5c19e1f6a0c7d5decbb80a50c22fbb
final_state_hash: cb1e7d17d69cfec4e9697f575fc7458e6473da7172b4528e2a62846fb1e30ade
first_ts        : 1,052,092,863
last_ts         : 1,076,518,492
```

These equal the recorded P3-S25.1D continuous reference hash exactly.

## 10. Decision-gate classification

Per the briefing decision rule:

- `A — VERIFIER DEFECT ISOLATED AND FIX VERIFIED`: confirmed (boundary
  divergence reproduced; corrected test passes deterministically twice;
  8/8 interruption mutations detected).
- `B — INTERRUPTION SEMANTICS VERIFIED`: confirmed for the isolated case
  (exact output/state equivalence, see §9).
- `C — PIPELINE / CHECKPOINT DEFECT`: NOT found.
- `D — UNRESOLVED`: no (the exact boundary defect was conclusively isolated;
  the original 7d7… byte value is unrecoverable because the previous session
  overwrote its own failing JSON before stopping — this is recorded, not
  papered over).

This does NOT certify multi-chunk execution at large, and does NOT set
`SAFE_CHUNKS_PER_TURN`.

## 11. Mutation detection (interruption-specific, 8/8)

`recovery/output/interruption_mutations.json` — 3-atomic-chunk disposable
range, stop after chunk 0:

| # | Mutation                      | Detected | Evidence |
|---|-------------------------------|:--------:|----------|
| 1 | wrong resume byte offset      | yes      | boundary-mismatch + chunk-sig chunk 1 |
| 2 | skipped chunk                 | yes      | coverage, row-count, output-hash mismatch |
| 3 | duplicated chunk              | yes      | coverage [1,1], chunk-sig chunk 2 |
| 4 | wrong boundary sequence       | yes      | boundary-mismatch chunk 2 |
| 5 | wrong input carry             | yes      | chunk-sig + output-hash mismatch |
| 6 | post-chunk carry used as input| yes      | chunk-sig + rowset-count + output-hash |
| 7 | mutated checkpoint state      | yes      | boundary + row-count (1 record lost) + hash |
| 8 | dropped prefix output         | yes      | chunk-count + row-count + hash mismatch |

Result: **8/8 DETECTED** (fail-closed; no mutation escaped).

## 12. Real checkpoint after recovery

```text
checkpoint_sha_after : 47da8f020426bec67e9f7865519e900a28c286ba1d13a15bee6a8c0f5307afc7
== checkpoint_sha_before : TRUE
last_completed_chunk : 10
next_chunk_id        : 11
next_byte_offset     : 163,578,081
```

## 13. CHUNK 11 / new source processing

```text
CHUNK 11 = NOT PROCESSED
```
No new real-source byte beyond 163,578,081 was read. All testing used only
the already-committed disposable bench range [0, 150,994,944) (6 × 24 MiB
atomic chunks within chunks 0–10). No Dukascopy, no ML, no production MQL5,
no F1–F4 / Candidate Setup / FEATURE_CONTRACT/label changes.

## 14. P3-S25.1D overall status

**UNRESOLVED (unchanged by this recovery).** The isolated interruption/resume
path is now verified, but the broader P3-S25.1D multi-chunk turn benchmark,
the full interruption matrix and its manifest are NOT re-run here and remain
open. `SAFE_CHUNKS_PER_TURN` is NOT set. `docs/CURRENT_PROJECT_STATE.md` is
NOT changed by this recovery.

## 15. Required artifacts (this recovery)

```text
recovery/interruption_failure_isolation.py
recovery/test_interruption_resume_recovery.py
recovery/test_interruption_mutations.py
recovery/run_interruption_recovery.py
recovery/output/interruption_failure_isolation.json
recovery/output/interruption_recovery_result.json
recovery/output/interruption_mutations.json
recovery/output/interruption_recovery_manifest.json
docs/P3_S25_1D_R_INTERRUPTION_RECOVERY.md            (this report)
docs/SESSION_HANDOVER_2026-08-28_P3_S25_1D_R_INTERRUPTION_RECOVERY.md
```

The original failed P3-S25.1D evidence (manifest `interruption: false`) is
preserved untouched; nothing historical was overwritten by this session.

*End of P3-S25.1D-R interruption recovery report.*