SniperGold_ML/qualification/QUALIFICATION_REPORT.md

# P3-DATA-ENGINE-004 QUALIFICATION REPORT

Session: SOURCE-GRAMMAR CORRECTION + RE-QUALIFICATION (NO REAL-DATA EXECUTION)
Generated: 2026-09-07T10:33:06.186520+00:00

Specification: docs/P3_DATA_ENGINE_V1_SPEC.md (SHA-256 `effbaf2624cd46137c22a246fdabff4067052807b6b6224be464366bec4745d9`)
Governance: P3-DATA-ENGINE-001 GOV-CLOSE-001 (DESIGN FROZEN) + P3-DE-004 CONTROLLED AMENDMENT (spec 1.1.0, Appendix C); G-14 NOT AUTHORIZED
Versions: spec_version=1.1.0 (amended by P3-DATA-ENGINE-004; Appendix C), engine_version=1.1.0, parser_version=PARSER_V1.1, algorithm_version=ALG_V1, schema=CTS_V1 / CBS_V1 (unchanged), dataset_version=DS_V1.0.0, grammar_registry=['tickstory_mt5', 'dotted_v1']

## Gates

- unit : PASS
- golden : PASS
- source_grammar : PASS
  - suite cases 15, e2e PASS
- property : PASS
- adversarial : PASS
- mutation : PASS
  - detection 8/8 (per-mutation ledger in JSON)
- independent_verifier : PASS
- legacy_diff : PASS
  - items classified 9, unresolved 0, deferred DEFERRED, legacy untouched True
- dataset_builder : PASS
- cli : PASS

## Verdict

**QUALIFIED**

## Safety

- 34gb_source_processed : NO
- real_data_pilot_executed : NO
- real_source_ingested : NO
- workload_46_executed : NO
- chunk_760_plus_touched : NO
- legacy_checkpoint_modified : NO
- legacy_pipeline_modified : NO
- specification_amendment_applied : YES (approved P3-DE-004 controlled amendment; Appendix C; SHA verified)
- governance_baseline_altered : NO (unchanged except the approved specification amendment)
- g14_authorization_changed : NO (G-14 remains BLOCKED / NOT AUTHORIZED)
- forecasting_performed : NO
- research_dataset_generated : NO
- production_checkpoint_created : NO

## Limitations

- NOT VERIFIED: byte-range/tick/hash differential vs the real 34.5 GB source (G-13 pilot scope; session forbids opening the source).
- NOT VERIFIED: full 34 GB headless pass (G-14 NOT AUTHORIZED).
- DEFERRED: G-17 HMAC hardening.
- PARTIAL: CLI 'verify --mode range/legacy' live-range modes exercised only on synthetic fixtures.
- PARTIAL: the source-preservation sidecar (spec 8.6) was validated on synthetic six-column fixtures only; no real source bytes were opened.
- source_tz_offset_minutes production value preserved at 0 from the P3-DE-003 evidence chain (spec Appendix C.2); G-6 requires explicit establishment at source initialization in the pilot phase.

## Interventions

1. **Producer/verifier epoch mismatch caught by dual-path tests** -- Corrected the Hinnant civil-days formula; re-verified producer == verifier on all fixtures (authorized=authorized (implementation), necessary)
2. **Bar content-id raises on 17-field CBS rows** -- Producer and verifier serializers now select the 14 canonical fields from full CBS rows (authorized=authorized (implementation), necessary)
3. **Resume re-aggregates an already processed workload** -- Every processed workload now writes a per-TF part file (empty when it contains no finalized bars) as an aggregation-completeness marker; resume skips workloads whose parts exist (authorized=authorized (implementation), necessary)
4. **Journal chain broke at seq 1 during read-back** -- Standardized the line hash over the line text without the trailing newline (authorized=authorized (implementation), necessary)
5. **pyarrow replace_schema_metadata TypeError (expected bytes, int found)** -- Stringify all schema metadata values (authorized=authorized (implementation), necessary)
6. **Spec gap: carry field list (§11.4) omits first_src_line needed by CBS_V1** -- Checkpointed carry objects additionally store first_src_line; documented in the implementation map (authorized=authorized (implementation detail; no spec amendment), necessary)
7. **CLI pause/stop lacked --out-root; verify --mode dataset raced final checkpoint writes** -- Added --out-root to pause/stop; reordered finalize to verify -> COMPLETED -> evidence -> RUN_COMPLETE (authorized=authorized (implementation), necessary)
8. **P3-DE-003 pilot audit: the real six-column Tickstory grammar differs from the dotted corpus grammar** -- Amended spec Appendix C (C.1 grammar, C.2 timestamp, C.3 volume/last/versions, C.4 implementation-visible changes, C.5 exclusions); specification SHA updated to effbaf2624cd46137c22a246fdabff4067052807b6b6224be464366bec4745d9 (authorized=authorized (approved amendment package), necessary)
9. **Grammar presets, compact timestamp and the 'last' column required implementation changes** -- versions registry + version bumps; config source_grammar identity field + fail-closed validation; parse_compact_datetime; _parse_chunk_tickstory; preservation serialization + per-chunk sidecar writes; certificate grammar_id + compact sniffing; init certificate/config mismatch BLOCK; independent verifier v_parse_chunk_tickstory + preservation id checks; evidence + verify_dataset preservation content-id verification (authorized=authorized (implementation of the approved amendment), necessary)
10. **P3-DE-003 independent pilot audit: CLI status/resume on a run-less directory misbehaves** -- Return the _err code as the first tuple element; callers return it; checkpoint-load exceptions become exit 4; regression tests cli_status_runless_dir / cli_resume_runless_dir added (authorized=authorized (implementation), necessary)
11. **Resume-session inspection found a broken partial edit in the qualification driver** -- Restored both import lines; added gates['source_grammar'] (suite_cases + e2e); updated session/SHA/governance/versions/safety/limitations/interventions and the Markdown header (authorized=authorized (implementation; resume within P3-DE-004 scope), necessary)