SniperGold_ML/docs/P3_S25_1_EXTERNAL_TICK_INGESTION.md

P3-S25.1 — EXTERNAL TICK INGESTION (status)

Date     : 2026-08-26
Session  : P3-S25.1 RECOVERY #3
Status   : INGEST NOT COMPLETED — SAFE RESUMED EXECUTION NOT YET PERFORMED

Source

PATH : MQL5\Files\XAUUSD_mt5_ticks.csv            (PRIMARY — Tickstory)
SIZE : 34,473,661,010 bytes  (~34.5 GB)
MTIME: 2026-08-24 22:44:41 UTC
SHA256: 5252ce8f5b0e7d286b71a3172c7ef9c1009c8e4b1254ac67400a803fcd85cba1
SCHEMA: date(YYYYMMDD), time(HH:MM:SS), bid, ask, last, volume   [no header]
ROWS (approx): 722,941,870  (per P3-S25 inventory)
TZ BASIS: UTC, offset 0 seconds (immutable run config)

Pipeline modules (ml/p3/p3_s251_external_ingest/)

s251_config.py      immutable run configuration
s251_parse.py       streaming parser + validation + canonicalisation (bounded)
s251_aggregate.py   streaming M15/M30 state-machine (bounded)
s251_ingest.py      chunk orchestrator + hashing + atomic checkpoint
s251_checkpoint.py  atomic JSON checkpoint / resume refusal on source change
s251_oracle.py      independent vectorised M15/M30 oracle (batch reference)
s251_subset.py      deterministic subset selection (fallback path)
s251_tests.py       T01..T20 validation suite
fixtures.py         test fixtures

Checkpoint / resume status (A7)

Valid real-source checkpoint: NONE.

The present output/p3_s251_checkpoint.json is a test-fixture artifact (run_id "src_clean", source = scratch/mt100.csv, 4,747 B). It is NOT a valid resume checkpoint for the real 34.5 GB source and cannot be used as one.

Overall status

Stage A diagnosis complete; Stage B (bounded resume) of the real source not started — no valid real-source checkpoint exists yet. See P3_S25_1_INTERRUPTION_DIAGNOSIS.md.

End of P3-S25.1 external tick ingestion status