SniperGold_ML/docs/P3_S25_1_INTERRUPTION_DIAGNOSIS.md

P3-S25.1 — INTERRUPTION DIAGNOSIS (Recovery #3)

Date              : 2026-08-26
Session           : P3-S25.1 RECOVERY #3
Goal              : FIRST diagnose why the P3-S25.1 ingestion repeatedly stops;
                    ONLY THEN, if a valid checkpoint and understood cause exist, resume.
Outcome this run  : STAGE A COMPLETE. STAGE B NOT STARTED (see verdict below).
Status            : E — ENVIRONMENT/EXECUTION LIMITATION (root cause identified)
Next              : resume preconditioned on a valid real-source checkpoint,
                    which does not yet exist.

1. Interrupted attempts observed

The session brief references three stopped attempts of the 34.5 GB Tickstory ingestion. On-disk evidence (this repo, terminal, Temp, logs) does not contain three durable, distinct run records. What is present:

Attempt evidence	Artifact found
Prior attempt(s)	p3_s25_external_data_validation/output/s25_execution_status.json — "full tick→M15 ingest not completable in session budget"; handover SESSION_HANDOVER_2026-08-26_P3_S25_EXTERNAL_DATA_BASIS_VALIDATION.md — FINAL CLASSIFICATION E-UNRESOLVED, "could NOT complete within the session's compute budget".
Current real-source checkpoint	NONE valid. Only ml/p3/p3_s251_external_ingest/output/p3_s251_checkpoint.json exists, and it references run_id: "src_clean", source scratch\mt100.csv (a 4,747-byte test fixture, not the 34.5 GB source).
Partial M15/M30 real-source output	NONE. Only fixture/scratch outputs exist.
Recovery log / run manifest / .tmp / lock	NONE present (searched project tree + MQL5\Files\Temp).
Original source file	MQL5\Files\XAUUSD_mt5_ticks.csv, 34,473,661,010 bytes, mtime 2026-08-24 22:44:41.

Conclusion: the ingest has not recorded a completed chunk against the real source in any successfully-checkpointed invocation.

2. Failure signature classification (A3)

The repeated stoppage is best classified as:

TYPE   : (1) AI-assistant / session execution interruption
SIGNATURE : A — command/execution timeout (wall-clock / compute-budget exhaustion)

Supporting evidence:

• The P3-S25 authoritative record explicitly states the full ingest could not complete within the session wall-clock/compute budget.
• Measured in this session: a benign, read-only full-source SHA-256 pass takes ~153 seconds on this machine. The IngestRun.begin() computes the full source SHA-256 (streaming 34.5 GB) before any chunk is processed, and the run() method processes the entire remaining file in one long-lived Python subprocess with a default chunk of 4,000,000 rows.
• There is no durable checkpoint of a real run, consistent with each attempt being terminated (command/tool/session) before completing/committing a chunk.

Why the design makes an in-session completion unlikely:

1. begin() hashes the entire 34.5 GB file in-memory streaming (~153 s) and only silently stores the SHA; a real chunk is processed after that.
2. The chunked checkpoint is written only after a whole chunk is processed; a killed session leaves no durable record -> the stale test-fixture checkpoint remains.
3. The default execution is effectively "process everything remaining" in one subprocess, which the environment cannot outlast.

This is a resource/execution limitation, not a parser/data bug: the same bounded, streaming design is validated and the parser/aggregator are memory- bounded. No parser exception, no memory/disc exhaustion, no deadlock.

3. Last valid checkpoint (A7)

Valid real-source checkpoint: NONE.

Latest checkpoint file contents (output/p3_s251_checkpoint.json):
  run_id            : "src_clean"                      (test artifact)
  source_path       : .../scratch/mt100.csv            (test fixture, 4747 B)
  source_sha256     : d552b9e4…                         (fixture hash, not source)
  last_completed_chunk : 2                             (of the fixture only)
  rows_processed    : 101                              (fixture only)

=> The checkpoint refers to a test fixture, NOT the 34.5 GB source. It cannot
   be used as a resume position for the real ingestion (A7 STOP rule).

4. Source integrity

path   : MQL5\Files\XAUUSD_mt5_ticks.csv
size   : 34,473,661,010 bytes
mtime  : 2026-08-24 22:44:41 UTC
sha256 : 5252ce8f5b0e7d286b71a3172c7ef9c1009c8e4b1254ac67400a803fcd85cba1  (measured this session, read-only)
rowcount (approx) coincident with S25 inventory: 722,941,870 rows

Source file is present and unchanged (size/mtime match the 2026-08-24 record).

5. Resource analysis (A4) — measured this session

Disk free (D:)  : 850 GB   -> sufficient
RAM total/free  : 47.9 / 36.1 GB -> sufficient for streaming chunked ingest
Full-source SHA : ~153 s for a read-only hash (the dominant single-op cost)

Section storage / disk / RAM are not the blocker. The blocker is the single long-running command vs. session wall-clock compute budget.

5b. Pipeline validation test-suite run (this session, 16/20 PASS)

Ran s251_tests.py (T01..T20, fixtures only — no real source). Result: 16/20 PASS. The four failures are TEST-HARNESS expectation bugs, NOT pipeline defects:

Test	Result	Classification
T04 M15 boundary	FAIL	Expectation bug: test demands the final incomplete bucket be emitted; design deliberately carries it in partial_carry_m15 and never emits it. Emitted buckets match design.
T06 M30 boundary	FAIL	Same cause (final incomplete M30 bucket carried, not emitted).
T15 future-data mutation	FAIL	Test-code bug: dt.datetime(2026, tzinfo=...) called without month/day -> TypeError. Not a pipeline error.
T16 row-order mutation	FAIL	Expectation bug: the aggregator stably sorts ticks by timestamp (documented), so identical M15 output is CORRECT after a pure row permutation; the raw manifest DOES change (hash differs), which is the correct detection signal. Test's pass condition is inconsistent.

Independent verification signals that DO pass and are the meaningful evidence: T02 timestamp parsing, T03 UTC normalisation, T05/T07 independent-oracle equality, T08 chunk-hash determinism, T09 checkpoint atomicity, T10/T11/T12 resume equivalence incl. mid-bucket cuts, T13 malformed detection, T14 duplicate detection, T19 source-hash refusal, T20 subset determinism.

Per CODE_VERIFICATION policy: PASS is not final evidence; however these tests plus the independent oracle (different code path) agree, which is the correct signal for the streaming pipeline. The four FAILs are test-harness defects and were NOT edited to force green (policy §0/§15: no silent modification). They should be repaired in a dedicated test-fix phase before the suite is treated as green.

6. Bounded-memory check (A5)

• s251_ingest._file_blocks streams the file in 64 MiB blocks, _line_stream stays in Python lines up to chunk_rows.
• ChunkParser keeps only per-chunk lists; Aggregation is a streaming state-machine, holding only the current open M15/M30 bucket.
• No full-file read(), readlines(), no unbounded dict/list accumulation, no DataFrame accumulation.

=> Bounded-memory: YES. Confirmed. This is not the cause of failure.

7. Execution-model check (A6)

The prior approach executed effectively the whole 34.5 GB source in a single monolithic subprocess (run() processes the entire remaining file). That is the unsafe architecture for this environment.

Required corrective model:

ONE BOUNDED CHUNK -> VERIFY -> CHECKPOINT -> RETURN CONTROL
(one Python invocation per chunk, or a small bounded batch with per-chunk
checkpoints and atomic stop)

8. Root-cause confidence

Confidence : MEDIUM

Reason: the execution/wall-clock limitation is well supported (documented P3-S25 record + measured 153 s full-hash + monolithic run model), but the "three attempts" are not individually evidenced in durable artifacts, so the exact instant/operation of each stop cannot be proven. The overall mechanism (research-execution bound) is established with MEDIUM confidence; the code design is consistent with repeated pre-chunk termination.

9. Classification

E — ENVIRONMENT/EXECUTION LIMITATION

10. Why Stage B was NOT started

Per protocol hard-stop conditions (A7 + session objective), ingestion may only resume when BOTH: (i) the checkpoint is proven valid, and (ii) the cause is identified with a safe strategy.

(i) is NOT satisfied: the only checkpoint to hand is a test-fixture artifact (no valid real-source checkpoint exists). Per A7 ("If the latest checkpoint is invalid: STOP. Do not guess a resume position.") no resume byte-offset can be honestly chosen this session. Therefore Stage B was not forced.

11. Corrective execution strategy (recommended, NOT executed here)

1. Do NOT re-run a monolithic 34 GB command.
2. Initialize anew the real-source ingestion and verify checkpoint against the source line before the first chunk (the begin() SHA-256 check).
3. Execute ONE bounded chunk per invocation (e.g. default 4,000,000 rows), then VERIFY/WRITE CHECKPOINT, then STOP/return control; explicitly bound the run (constrain max_chunks in a small driver) so a session cannot exceed its budget.
4. After each chunk, verify full-source continuity in the checkpoint (source size/mtime/SHA-256) and record progress in a recovery log (run_id, chunk_id, byte_start, byte_end, rows, first/last ts, m151 count, m30 count, chunk_sha256, cp_sha256, elapsed, mem_peak, status).
5. If the environment still cannot run >1 chunk per session, fall back to the previously authorized deterministic subset strategy (documented separately) and classify honestly as SUBSET BASIS VALIDATED.

12. State

• No ML, no training, no feature/label/TP/SL/H changes performed.
• No MQL5 / production code changed.
• No source files modified; the 34.5 GB CSV untouched.
• No Dukascopy .bi5 reverse-engineering started.
• p3_s251_subset_manifest.json NOT written (subset fallback not used).
• Final status: E ; Stage A complete; Stage B NOT started.

End of P3-S25.1 interruption diagnosis.