4.6 KiB
P3-S25.1 REAL-SOURCE CHECKPOINT — CHUNK 0
Date : 2026-08-26
Session : P3-S25.1 STAGE-B — REAL TICK CHECKPOINT INITIALIZATION (CHUNK 0 ONLY)
Status : CHUNK 0 COMPLETE
Final class : A — REAL-SOURCE CHUNK 0 VERIFIED
Source
PATH : MQL5\Files\XAUUSD_mt5_ticks.csv (PRIMARY — Tickstory)
SIZE : 34,473,661,010 bytes (~34.5 GB) (verified by stat this session)
MTIME : 2026-08-24 15:44:41 UTC (epoch 1787586281.04)
SHA256 : 5252ce8f5b0e7d286b71a3172c7ef9c1009c8e4b1254ac67400a803fcd85cba1
(documented from P3-S25.1.R; NOT recomputed this session)
SCHEMA : date(YYYYMMDD), time(HH:MM:SS), bid, ask, last, volume [no header]
TZ : UTC (offset 0), immutable run config
Source identity reconciled this session:
- file exists ✓
- size == 34,473,661,010 ✓
- mtime epoch == 1787586281.04 (unchanged) ✓
- no full 34.5 GB SHA recomputed (no integrity contradiction) ✓
Run identity
run_id : P3_S251_REAL_TICK_RUN_001
parser version : p3_s251_parser_v1
algorithm ver : p3_s251_algorithm_v1
timestamp basis : utc (tz_offset 0)
chunk size : 12 MiB target (12,582,912 bytes) of raw input
Chunk size
Preferred 8–16 MiB raw input for chunk 0; this session used a
12 MiB line-safe target. The chunk extends to the next newline so the
boundary is always line-safe and the recorded byte_end is a deterministic
resume offset with no partial line. Bounded-memory processing; exactly ONE
chunk executed; no full-file load; no loop over the remaining file.
Chunk 0 result
chunk_id : 0
byte_start : 0
byte_end : 12,582,953 bytes
rows_processed : 281,735
first_timestamp : 1052092863 (2003-05-05 00:01:03 UTC)
last_timestamp : 1054563309
m15_rows_emitted : 1,613
m30_rows_emitted : 897
raw_chunk_sha256 : f2e7d7c8ef141987b01599612aea7e4b9f5adadfaf1aaebb30122491e30f9b70
parsed_chunk_sha256 : 2f793621f07695de3972369abf5cd3cf4227f1a2280b16f2b4240009ccb670c4
rows_hash : a88a7ea610a81c561691ccbb7de31e4ce03afff5392374b83a10d8fd0edb6891
monotonic : OK (0 non-monotonic)
malformed : 0 across all categories
elapsed : ~3.8 s
peak memory : ~102 MiB (read-only tracemalloc probe)
Checkpoint
File: ml/p3/p3_s251_external_ingest/output/s251_real_checkpoint.json
status : CHUNK_COMMITTED
last_completed_chunk : 0
byte_offset : 12,582,953 (= next_byte_start)
rows_processed : 281,735
m15_rows_emitted : 1,613
m30_rows_emitted : 897
partial_carry_m15 : preserved (open M15 bucket at 1054563300)
partial_carry_m30 : preserved (open M30 bucket at 1054562400)
checkpoint_sha256 : 302af53218c355e5d70bbdbe784270954981313f81119a7cc00bb6e083096a18
The checkpoint and manifest agree exactly (byte offset, rows, M15/M30 counts). The checkpoint file hash is recorded in the recovery log.
Resume state
next_chunk_id : 1
next_byte_offset : 12,582,953
chunk 0 duplicate : prevented (last_completed_chunk == 0, byte_offset > 0)
chunk 1 skip : prevented (0 + 1 == 1)
M15/M30 carry : preserved for seamless continuation
source identity : preserved (sha256/size/mtime reconcile with run record)
Verified by re-reading the checkpoint twice (deterministic equality) and by a disposable resume-planning check that consumed NO real data.
Resource usage
- elapsed ~3.8 s for chunk 0 (12 MiB, 281,735 rows).
- peak memory ~102 MiB — bounded, streaming, no full-file load.
- disk free D: ~912 GB — ample.
- exactly one bounded Python invocation per chunk; control returned after commit (no monolithic long-running process).
Reproducibility
The committed byte span [0, 12,582,953) was re-executed in a disposable
read-only path and compared to the committed recovery log. Exact agreement
for ALL of: row count, first/last timestamp, raw chunk SHA-256, parsed-record
SHA-256, rows hash, M15 count, M30 count, monotonic flag.
reproducible : true (all 9 checks pass)
Remaining work
Full ingestion : NOT STARTED (only chunk 0 of the 34.5 GB source done)
Next authorized : CHUNK 1 ONLY (from byte_offset 12,582,953)
Remaining chunks : the rest of the file, one bounded chunk per session
Population replay : NOT STARTED
External parity : NOT STARTED
ML / training : NOT STARTED (out of scope for this session)
Dukascopy : NOT USED
No ML, no production MQL5, no labels/features, no Dukascopy use, no deploy, no trading. External Tickstory CSV was READ ONLY.
End of P3-S25.1 real-source checkpoint chunk-0 report.