SniperGold_ML/docs/P3_S25_1_REAL_SOURCE_CHECKPOINT.md

# P3-S25.1 REAL-SOURCE CHECKPOINT — CHUNK 0

```text
Date        : 2026-08-26
Session     : P3-S25.1 STAGE-B — REAL TICK CHECKPOINT INITIALIZATION (CHUNK 0 ONLY)
Status      : CHUNK 0 COMPLETE
Final class : A — REAL-SOURCE CHUNK 0 VERIFIED
```

## Source

```text
PATH   : MQL5\Files\XAUUSD_mt5_ticks.csv            (PRIMARY — Tickstory)
SIZE   : 34,473,661,010 bytes  (~34.5 GB)           (verified by stat this session)
MTIME  : 2026-08-24 15:44:41 UTC                     (epoch 1787586281.04)
SHA256 : 5252ce8f5b0e7d286b71a3172c7ef9c1009c8e4b1254ac67400a803fcd85cba1
         (documented from P3-S25.1.R; NOT recomputed this session)
SCHEMA : date(YYYYMMDD), time(HH:MM:SS), bid, ask, last, volume  [no header]
TZ     : UTC (offset 0), immutable run config
```

Source identity reconciled this session:
* file exists ✓
* size == 34,473,661,010 ✓
* mtime epoch == 1787586281.04 (unchanged) ✓
* no full 34.5 GB SHA recomputed (no integrity contradiction) ✓

## Run identity

```text
run_id          : P3_S251_REAL_TICK_RUN_001
parser version  : p3_s251_parser_v1
algorithm ver   : p3_s251_algorithm_v1
timestamp basis : utc (tz_offset 0)
chunk size      : 12 MiB target (12,582,912 bytes) of raw input
```

## Chunk size

Preferred 8–16 MiB raw input for chunk 0; this session used a
**12 MiB line-safe** target. The chunk extends to the next newline so the
boundary is always line-safe and the recorded `byte_end` is a deterministic
resume offset with no partial line. Bounded-memory processing; exactly ONE
chunk executed; no full-file load; no loop over the remaining file.

## Chunk 0 result

```text
chunk_id             : 0
byte_start           : 0
byte_end             : 12,582,953 bytes
rows_processed       : 281,735
first_timestamp      : 1052092863  (2003-05-05 00:01:03 UTC)
last_timestamp       : 1054563309
m15_rows_emitted     : 1,613
m30_rows_emitted     : 897
raw_chunk_sha256     : f2e7d7c8ef141987b01599612aea7e4b9f5adadfaf1aaebb30122491e30f9b70
parsed_chunk_sha256  : 2f793621f07695de3972369abf5cd3cf4227f1a2280b16f2b4240009ccb670c4
rows_hash            : a88a7ea610a81c561691ccbb7de31e4ce03afff5392374b83a10d8fd0edb6891
monotonic            : OK (0 non-monotonic)
malformed            : 0 across all categories
elapsed              : ~3.8 s
peak memory          : ~102 MiB (read-only tracemalloc probe)
```

## Checkpoint

File: `ml/p3/p3_s251_external_ingest/output/s251_real_checkpoint.json`

```text
status               : CHUNK_COMMITTED
last_completed_chunk : 0
byte_offset          : 12,582,953   (= next_byte_start)
rows_processed       : 281,735
m15_rows_emitted     : 1,613
m30_rows_emitted     : 897
partial_carry_m15    : preserved (open M15 bucket at 1054563300)
partial_carry_m30    : preserved (open M30 bucket at 1054562400)
checkpoint_sha256    : 302af53218c355e5d70bbdbe784270954981313f81119a7cc00bb6e083096a18
```

The checkpoint and manifest agree exactly (byte offset, rows, M15/M30 counts).
The checkpoint file hash is recorded in the recovery log.

## Resume state

```text
next_chunk_id      : 1
next_byte_offset   : 12,582,953
chunk 0 duplicate  : prevented (last_completed_chunk == 0, byte_offset > 0)
chunk 1 skip       : prevented (0 + 1 == 1)
M15/M30 carry      : preserved for seamless continuation
source identity    : preserved (sha256/size/mtime reconcile with run record)
```

Verified by re-reading the checkpoint twice (deterministic equality) and by a
disposable resume-planning check that consumed NO real data.

## Resource usage

* elapsed ~3.8 s for chunk 0 (12 MiB, 281,735 rows).
* peak memory ~102 MiB — bounded, streaming, no full-file load.
* disk free D: ~912 GB — ample.
* exactly one bounded Python invocation per chunk; control returned after
  commit (no monolithic long-running process).

## Reproducibility

The committed byte span `[0, 12,582,953)` was re-executed in a disposable
read-only path and compared to the committed recovery log. Exact agreement
for ALL of: row count, first/last timestamp, raw chunk SHA-256, parsed-record
SHA-256, rows hash, M15 count, M30 count, monotonic flag.

```text
reproducible : true (all 9 checks pass)
```

## Remaining work

```text
Full ingestion      : NOT STARTED (only chunk 0 of the 34.5 GB source done)
Next authorized     : CHUNK 1 ONLY (from byte_offset 12,582,953)
Remaining chunks    : the rest of the file, one bounded chunk per session
Population replay   : NOT STARTED
External parity     : NOT STARTED
ML / training       : NOT STARTED (out of scope for this session)
Dukascopy           : NOT USED
```

No ML, no production MQL5, no labels/features, no Dukascopy use, no deploy,
no trading. External Tickstory CSV was READ ONLY.

*End of P3-S25.1 real-source checkpoint chunk-0 report.*