SniperGold_ML/docs/P3_S25_1D_VERIFICATION_DRIVER_HARDENING.md

# P3-S25.1D — VERIFICATION DRIVER HARDENING

```text
Document type : PHASE REPORT / GOVERNANCE-ADJACENT (verification evidence)
Date          : 2026-08-28
Phase         : P3-S25.1D — Verification Driver Hardening & Bounded Multi-Chunk Turn Benchmark
Scope         : VERIFICATION / EXECUTION-EFFICIENCY ONLY. No real-source ingestion.
Status        : (see multichunk_benchmark_manifest.json — authoritative)
```

## Purpose

Harden the disposable verification driver against the recurring
carry / state-recording defects that produced false or misleading evidence in
earlier sessions, and establish immutable input/output state fingerprints for
each atomic 24 MiB chunk.

## Hardening requirements enforced (per P3-S25.1D section 6)

The driver implements the following invariant workflow per atomic chunk:

```
A. input_carry   = deepcopy(current_carry)   BEFORE any aggregation
B. recorded input_carry is IMMUTABLE for the remainder of verification
C. canonical (production) ordering            (stable, by integer timestamp)
D. output_carry  = deepcopy(current_carry)   AFTER aggregation (separate state)
E. canonical serialisation                    (deterministic, key-sorted JSON)
F. independent input_state_hash / output_state_hash (sha256 over canonical form)
G. fail-closed assertions                     -> PASS only if all hold, else VERIFIER_ERROR
```

Any failure returns `VERIFIER_ERROR` (never a forced PASS). The driver also
enforces **byte-contiguity**: chunks are read line-safely (a chunk of up to
`ATOMIC` bytes is extended to the next newline) and chained on the *actual
consumed* byte offset, mirroring the production parser (`pos += len(raw)`).
This prevents nominal-boundary drift that otherwise duplicates/skips a record
straddling a chunk edge and silently breaks the single-stream invariant.

## Verifier self-test (section 7) — 8 required defects

A synthetic self-test deliberately reproduces each previously-observed driver
defect and asserts the hardened verifier REJECTS it:

| # | Defect injected                                        | Detection |
|---|--------------------------------------------------------|-----------|
| 1 | capture input carry after aggregation                   | input captured after aggregation |
| 2 | same mutable carry object into aggregation             | same mutable carry object |
| 3 | tuple sorting instead of canonical order               | tuple-sorting-used |
| 4 | post-chunk carry used for reproducibility              | post-chunk-carry-used |
| 5 | record input carry mutated after capture               | input-mutated-after-capture |
| 6 | omit input-state hash                                 | input-state-hash-missing |
| 7 | serialized carry field order altered                    | serialized-carry-field-order |
| 8 | chunk boundary shifted by one byte                       | chunk-boundary-shifted |

Result: **8/8 VERIFIER-DEFECTS DETECTED** (machine-readable:
`output/verifier_hardening_results.json`).

## Mutation coverage (section 23) — 12 required classes

| Mutation | Detection |
|----------|-----------|
| 1 wrong chunks-per-turn value | chunks-per-turn-value |
| 2 skipped atomic chunk          | chunk-skipped-or-duplicated |
| 3 duplicated atomic chunk       | chunk-skipped-or-duplicated |
| 4 byte offset shifted           | byte-continuity            |
| 5 wrong source identity        | source-identity|
| 6 wrong input carry             | wrong-input-carry |
| 7 input carry captured post-agg | input-carry-post-aggregation |
| 8 output carry used as input    | output-carry-used-as-input |
| 9 canonical ordering replaced   | canonical-ordering-replaced |
|10 output row deletion           | row-deleted |
|11 output row duplication         | row-duplicated |
|12 semantic output mutation       | semantic-output-mutated |

Result: **12/12 DETECTED** (`output/multichunk_mutation_results.json`).

## Boundary-continuity (blocking finding + resolution)

The first integration of the multi-chunk engine exposed a real defect: batch
configs and reference agreed only when chunks were chained on the actual
line-safe consumed byte offset. When chained on fixed 24 MB multiples, the
semantic produced `N_BATCH = N_REF - 5` rows (a record straddling each nominal
edge was processed inconsistently). The test harness correctly returned
CHUNK-BATCH INVARIANCE FAILURE (fail-closed). After the driver was hardened to
chain on actual consumed offsets, `rows / M15 / M30 / hashes / final carry`
became byte-identical across reference and batch 1/2/4/6
(`words_match = True`, `semantic_invariant = True`).

## Evidence files
- `output/verifier_hardening_results.json`
- `output/multichunk_mutation_results.json`
- `output/multichunk_invariance.json`
- `output/multichunk_interruption.json`
- `output/multichunk_benchmark.json`
- `output/multichunk_benchmark_manifest.json`

All under `ml/p3/p3_s251_external_ingest/multichunk_benchmark/`.