P3-S25.1D — VERIFICATION DRIVER HARDENING

Document type : PHASE REPORT / GOVERNANCE-ADJACENT (verification evidence)
Date          : 2026-08-28
Phase         : P3-S25.1D — Verification Driver Hardening & Bounded Multi-Chunk Turn Benchmark
Scope         : VERIFICATION / EXECUTION-EFFICIENCY ONLY. No real-source ingestion.
Status        : (see multichunk_benchmark_manifest.json — authoritative)

Purpose

Harden the disposable verification driver against the recurring carry / state-recording defects that produced false or misleading evidence in earlier sessions, and establish immutable input/output state fingerprints for each atomic 24 MiB chunk.

Hardening requirements enforced (per P3-S25.1D section 6)

The driver implements the following invariant workflow per atomic chunk:

A. input_carry   = deepcopy(current_carry)   BEFORE any aggregation
B. recorded input_carry is IMMUTABLE for the remainder of verification
C. canonical (production) ordering            (stable, by integer timestamp)
D. output_carry  = deepcopy(current_carry)   AFTER aggregation (separate state)
E. canonical serialisation                    (deterministic, key-sorted JSON)
F. independent input_state_hash / output_state_hash (sha256 over canonical form)
G. fail-closed assertions                     -> PASS only if all hold, else VERIFIER_ERROR

Any failure returns VERIFIER_ERROR (never a forced PASS). The driver also enforces byte-contiguity: chunks are read line-safely (a chunk of up to ATOMIC bytes is extended to the next newline) and chained on the actual consumed byte offset, mirroring the production parser (pos += len(raw)). This prevents nominal-boundary drift that otherwise duplicates/skips a record straddling a chunk edge and silently breaks the single-stream invariant.

Verifier self-test (section 7) — 8 required defects

A synthetic self-test deliberately reproduces each previously-observed driver defect and asserts the hardened verifier REJECTS it:

#	Defect injected	Detection
1	capture input carry after aggregation	input captured after aggregation
2	same mutable carry object into aggregation	same mutable carry object
3	tuple sorting instead of canonical order	tuple-sorting-used
4	post-chunk carry used for reproducibility	post-chunk-carry-used
5	record input carry mutated after capture	input-mutated-after-capture
6	omit input-state hash	input-state-hash-missing
7	serialized carry field order altered	serialized-carry-field-order
8	chunk boundary shifted by one byte	chunk-boundary-shifted

Result: 8/8 VERIFIER-DEFECTS DETECTED (machine-readable: output/verifier_hardening_results.json).

Mutation coverage (section 23) — 12 required classes

Mutation	Detection
1 wrong chunks-per-turn value	chunks-per-turn-value
2 skipped atomic chunk	chunk-skipped-or-duplicated
3 duplicated atomic chunk	chunk-skipped-or-duplicated
4 byte offset shifted	byte-continuity
5 wrong source identity	source-identity
6 wrong input carry	wrong-input-carry
7 input carry captured post-agg	input-carry-post-aggregation
8 output carry used as input	output-carry-used-as-input
9 canonical ordering replaced	canonical-ordering-replaced
10 output row deletion	row-deleted
11 output row duplication	row-duplicated
12 semantic output mutation	semantic-output-mutated

Result: 12/12 DETECTED (output/multichunk_mutation_results.json).

Boundary-continuity (blocking finding + resolution)

The first integration of the multi-chunk engine exposed a real defect: batch configs and reference agreed only when chunks were chained on the actual line-safe consumed byte offset. When chained on fixed 24 MB multiples, the semantic produced N_BATCH = N_REF - 5 rows (a record straddling each nominal edge was processed inconsistently). The test harness correctly returned CHUNK-BATCH INVARIANCE FAILURE (fail-closed). After the driver was hardened to chain on actual consumed offsets, rows / M15 / M30 / hashes / final carry became byte-identical across reference and batch 1/2/4/6 (words_match = True, semantic_invariant = True).

Evidence files

output/verifier_hardening_results.json
output/multichunk_mutation_results.json
output/multichunk_invariance.json
output/multichunk_interruption.json
output/multichunk_benchmark.json
output/multichunk_benchmark_manifest.json

All under ml/p3/p3_s251_external_ingest/multichunk_benchmark/.

4.8 KiB Raw Permalink Blame History