SniperGold_ML/docs/CURRENT_PROJECT_STATE.md

22 KiB

CURRENT PROJECT STATE

Document type : GOVERNANCE / OPERATIONAL STATE
DOCUMENT ROLE : FACTUAL_STATE
Created       : 2026-08-28 (P3-GOV-1 - Research Governance & Authoritative State Audit)
Revision      : v36 (2026-09-06 - P3-S25.1AR STEADY-STATE INGESTION WORKLOAD
                (chunks 738-759): factual cursor advanced - post-chunk-759
                checkpoint; CHUNK 760 authorization unchanged)
AUTHORITY     : HUMAN-READABLE FACTUAL SNAPSHOT ONLY.
                This document MUST agree with canonical machine evidence.
                This document MUST NOT authorize execution.
                This document MUST NOT override canonical checkpoint state.
                The AUTHORIZATION STATE block below is informational only;
                it records the current owner decision status. It grants
                nothing. Actual authorization lives in CURRENT_DECISION_GATE.md.
MACHINE-READABLE CHECKPOINT IS AUTHORITATIVE OVER PROSE.

Read this FIRST in every new session, then the newest session handover, then the current decision gate, then the session-start protocol v2.


A. Repository identity

Authoritative workspace:
D:\TradingTerminal\HFM Metatrader 5\MQL5\Shared Projects\SniperGold_ML_authoritative

Remote:
https://forge.mql5.io/chiki2bum2/SniperGold_ML.git

Branch:
main

Working tree:
CLEAN    (local HEAD == origin/main)

Stash:
NONE

Untracked:
NONE

B. FACTUAL EXECUTION STATE

Source: canonical machine checkpoint ml/p3/p3_s251_external_ingest/output/s251_real_checkpoint.json — this document reports those values; the checkpoint is authoritative over this prose.

Current phase           : P3-S25.1 Stage-B (PERIODIC INTEGRITY AUDIT + REAL-TICK INGESTION)
last_completed_chunk    = 759
current_chunk           = 759
next_chunk              = 760        (derived: last_completed_chunk + 1)
next_byte_offset        = 18442355762  (= next_byte_start)
SAFE_MAX_CHUNK_SIZE     = 24 MiB     (P3-S25.1C; per-chunk truth = chunk manifests)
full ingestion          = NOT COMPLETE (34.5 GB / 722,941,870-row dataset NOT complete)

OPERATIONAL EXECUTION CONFIGURATION (P3-GOV-7 promotion, 2026-09-01):

operational workers      = 24 (PROMOTED; previous operational value 8; owner
                          decision P3-GOV-7; steady-state ingestion runner
                          WORKERS)
SAFE_MAX_CHUNK_SIZE      = 24 MiB (UNCHANGED)
nominal workload         = 512 MiB (UNCHANGED)
parser / aggregator      = UNCHANGED (s251_parse / s251_aggregate)
merge semantics          = UNCHANGED
checkpoint semantics     = UNCHANGED
SAFE_CHUNKS_PER_TURN     = NOT ESTABLISHED (UNCHANGED)
CHUNK 364                = NOT AUTHORIZED (UNCHANGED - factual cursor only)

Checkpoint values (from the canonical checkpoint, machine-verified):

Checkpoint            : ml/p3/p3_s251_external_ingest/output/s251_real_checkpoint.json
run_id                : P3_S251_REAL_TICK_RUN_001
source_path           : D:\TradingTerminal\HFM Metatrader 5\MQL5\Files\XAUUSD_mt5_ticks.csv
source_sha256         : 5252ce8f5b0e7d286b71a3172c7ef9c1009c8e4b1254ac67400a803fcd85cba1
source_size           : 34,473,661,010 bytes
checkpoint chunk_size field : 12582912 (12 MiB) = INITIAL-run chunk-size
                              budget, set once at run creation; NOT the
                              per-chunk size of later chunks (D3 semantics;
                              field unchanged)
effective chunk size  : chunks 0-8 = 12 MiB (12582912 B); chunks 9-121 = 24 MiB
                        (25165824 B) - per-chunk truth is each chunk manifest
                        (output/s251_real_chunkNN_manifest.json chunk_size_bytes)
SAFE_MAX_CHUNK_SIZE   : 24 MiB (P3-S25.1C); SAFE_CHUNKS_PER_TURN = NOT ESTABLISHED
status                : CHUNK_COMMITTED
last_completed_chunk  : 759
current_chunk         : 759
next chunk            : 760
next byte offset      : 18,442,355,762 (=18442355762)
cumulative rows       : 388,839,639
cumulative M15        : 425,274
cumulative M30        : 213,979
last_timestamp        : 1,607,436,129 (=1607436129)
checkpoint sha256     : 1b0be1cd84ac9be88186d8a8ff2675a8f123a3a75f37a7f48b2118c6d9bd363f
                        (recomputed from the committed P3-S25.1AR workload checkpoint file)
checkpoint updated_utc: 2026-09-06T08:03:40.179585+00:00
full ingestion        : NOT COMPLETE

SOURCE SETUP: VERIFIED WITH LIMITATION - size and SHA were reconciled with the run records on chunk 0; the full 34.5 GB SHA was NOT recomputed (no integrity contradiction observed, so no full re-hash was forced).


C. RESEARCH STATE

P3-S25.1D broader multi-chunk benchmark             = OPEN
SAFE_CHUNKS_PER_TURN (multi-chunk turn benchmark)   = NOT ESTABLISHED
P3-S25.1D-R (interruption verifier recovery)        = COMPLETE (isolated path only)
External Tickstory basis validation                 = IN PROGRESS
                                                     (run P3_S251_REAL_TICK_RUN_001,
                                                      chunks 0-451 committed incl.
                                                      P3-S25.1H + P3-S25.1J + P3-S25.1K + P3-S25.1L + P3-S25.1M + P3-S25.1N + P3-S25.1O + P3-S25.1P 512 MiB / 8-worker workloads +
                                                      P3-S25.1R 512 MiB / 24-worker validation (chunks 188-209) +
                                                      P3-S25.1S 512 MiB / 24-worker workload (chunks 210-231) +
                                                      P3-S25.1T 512 MiB / 24-worker workload (chunks 232-253) +
                                                      P3-S25.1U 512 MiB / 24-worker workload (chunks 254-275) +
                                                      P3-S25.1V 512 MiB / 24-worker workload (chunks 276-297) +
                                                      P3-S25.1W 512 MiB / 24-worker workload (chunks 298-319) +
                                                      P3-S25.1X 512 MiB / 24-worker workload (chunks 320-341) +
                                                      P3-S25.1Y 512 MiB / 24-worker workload (chunks 342-363) +
                                                      P3-S25.1Z 512 MiB / 24-worker workload (chunks 364-385) +
                                                      P3-S25.1AA 512 MiB / 24-worker workload (chunks 386-407) +
                                                      P3-S25.1AB 512 MiB / 24-worker workload (chunks 408-429) +
                                                      P3-S25.1AC 512 MiB / 24-worker workload (chunks 430-451) + P3-S25.1AD 512 MiB / 24-worker workload (chunks 452-473) + P3-S25.1AE 512 MiB / 24-worker workload (chunks 474-495) + P3-S25.1AF 512 MiB / 24-worker workload (chunks 496-517) + P3-S25.1AG 512 MiB / 24-worker workload (chunks 518-539) + P3-S25.1AH 512 MiB / 24-worker workload (chunks 540-561) + P3-S25.1AI 512 MiB / 24-worker workload (chunks 562-583) + P3-S25.1AJ 512 MiB / 24-worker workload (chunks 584-605) + P3-S25.1AK 512 MiB / 24-worker workload (chunks 606-627) + P3-S25.1AL 512 MiB / 24-worker workload (chunks 628-649) + P3-S25.1AM 512 MiB / 24-worker workload (chunks 650-671) + P3-S25.1AO 512 MiB / 24-worker workload (chunks 672-693) + P3-S25.1AP 512 MiB / 24-worker workload (chunks 694-715) + P3-S25.1AR 512 MiB / 24-worker workload (chunks 738-759); full 34.5 GB dataset
                                                      NOT complete)
P3-S24 conclusion                                   = D - CURRENT EDGE NOT DEMONSTRATED
P3-GOV-5 preventive-control design                  = COMPLETE - DESIGN ONLY, COMMITTED
                                                     (classification B; controls DESIGNED;
                                                      no control implemented; design artifacts
                                                      durably committed 2026-08-30; see
                                                      docs/P3_GOV_5_PREVENTIVE_CONTROL_DESIGN/
                                                      and SESSION_HANDOVER_2026-08-30_...)

D. AUTHORIZATION STATE

CHUNK 11                 = COMPLETED (2026-08-30; authorized by the session
                             prompt, protocol v2 section 7)
P3-S25.1H (chunks 12-33) = COMPLETED (2026-08-31; ONE 512 MiB / 8-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7)
SAFE_CHUNKS_PER_TURN     = NOT ESTABLISHED
SAFE_MAX_CHUNK_SIZE      = 24 MiB (no dynamic-chunk change)
P3-S25.1J (chunks 34-55) = COMPLETED (2026-08-31; ONE 512 MiB / 8-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7)
P3-S25.1K (chunks 56-77) = COMPLETED (2026-08-31; ONE 512 MiB / 8-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7)
P3-S25.1L (chunks 78-99) = COMPLETED (2026-08-31; ONE 512 MiB / 8-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7)
P3-S25.1M (chunks 100-121) = COMPLETED (2026-08-31; ONE 512 MiB / 8-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7)
P3-S25.1N (chunks 122-143) = COMPLETED (2026-08-31; ONE 512 MiB / 8-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7)
P3-S25.1O (chunks 144-165) = COMPLETED (2026-08-31; ONE 512 MiB / 8-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7)
P3-S25.1P (chunks 166-187) = COMPLETED (2026-09-01; ONE 512 MiB / 8-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7)
P3-S25.1R (chunks 188-209) = COMPLETED (2026-09-01; ONE 512 MiB / 24-worker real
                             validation workload authorized by the session prompt,
                             protocol v2 section 7; requested 24, executor
                             processes 22, non-empty workers 22 - discrepancy
                             recorded; all gates PASS; classification A)
P3-S25.1S (chunks 210-231) = COMPLETED (2026-09-01; ONE 512 MiB / 24-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7; requested 24, executor
                             processes 22, non-empty workers 22 - discrepancy
                             recorded; all gates PASS; classification A)
P3-S25.1T (chunks 232-253) = COMPLETED (2026-09-01; ONE 512 MiB / 24-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7; requested 24, executor
                             processes 22, non-empty workers 22 - discrepancy
                             recorded; all gates PASS; classification A)
P3-S25.1U (chunks 254-275) = COMPLETED (2026-09-02; ONE 512 MiB / 24-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7; requested 24, executor
                             processes 22, non-empty workers 22 - discrepancy
                             recorded; all gates PASS; classification A)
P3-S25.1V (chunks 276-297) = COMPLETED (2026-09-02; ONE 512 MiB / 24-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7; requested 24, executor
                             processes 22, non-empty workers 22 - discrepancy
                             recorded; all gates PASS; classification A)
P3-S25.1W (chunks 298-319) = COMPLETED (2026-09-02; ONE 512 MiB / 24-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7; requested 24, executor
                             processes 22, non-empty workers 22 - discrepancy
                             recorded; all gates PASS; classification A)
P3-S25.1X (chunks 320-341) = COMPLETED (2026-09-02; ONE 512 MiB / 24-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7; requested 24, executor
                             processes 22, non-empty workers 22 - discrepancy
                             recorded; all gates PASS; classification A)
P3-S25.1Y (chunks 342-363) = COMPLETED (2026-09-02; ONE 512 MiB / 24-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7; requested 24, executor
                             processes 22, non-empty workers 22 - discrepancy
                             recorded; all gates PASS; classification A)
P3-S25.1Z (chunks 364-385) = COMPLETED (2026-09-02; ONE 512 MiB / 24-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7; requested 24, executor
                             processes 22, non-empty workers 22 - discrepancy
                             recorded; all gates PASS; classification A)
P3-S25.1AA (chunks 386-407) = COMPLETED (2026-09-02; ONE 512 MiB / 24-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7; requested 24, executor
                             processes 22, non-empty workers 22 - discrepancy
                             recorded; all gates PASS; classification A)
P3-S25.1AB (chunks 408-429) = COMPLETED (2026-09-03; ONE 512 MiB / 24-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7; requested 24, executor
                             processes 22, non-empty workers 22 - discrepancy
                             recorded; all gates PASS; classification A)
P3-S25.1AC (chunks 430-451) = COMPLETED (2026-09-03; ONE 512 MiB / 24-worker real
                             workload authorized by the session prompt,
                             protocol v2 section 7; requested 24, executor
                             processes 22, non-empty workers 22 - discrepancy
                             recorded; all gates PASS; classification A)
P3-S25.1AD (chunks 452-473) = COMPLETED (2026-09-03; ONE 512 MiB / 24-worker real workload authorized by the session prompt, protocol v2 section 7; requested 24, executor processes 22, non-empty workers 22 - discrepancy recorded; all gates PASS; classification A)
P3-S25.1AE (chunks 474-495) = COMPLETED (2026-09-03; ONE 512 MiB / 24-worker real workload authorized by the session prompt, protocol v2 section 7; requested 24, executor processes 22, non-empty workers 22 - discrepancy recorded; all gates PASS; classification A) CHUNK 496 execution      = NOT AUTHORIZED
P3-S25.1AF (chunks 496-517) = COMPLETED (2026-09-03; ONE 512 MiB / 24-worker real workload authorized by the session prompt, protocol v2 section 7; requested 24, executor processes 22, non-empty workers 22 - discrepancy recorded; all gates PASS; classification A) CHUNK 518 execution      = NOT AUTHORIZED
P3-S25.1AG (chunks 518-539) = COMPLETED (2026-09-03; ONE 512 MiB / 24-worker real workload authorized by the session prompt, protocol v2 section 7; requested 24, executor processes 22, non-empty workers 22 - discrepancy recorded; all gates PASS; classification A) CHUNK 540 execution      = NOT AUTHORIZED
P3-S25.1AH (chunks 540-561) = COMPLETED (2026-09-03; ONE 512 MiB / 24-worker real workload authorized by the session prompt, protocol v2 section 7; requested 24, executor processes 22, non-empty workers 22 - discrepancy recorded; all gates PASS; classification A) CHUNK 562 execution      = NOT AUTHORIZED
P3-S25.1AI (chunks 562-583) = COMPLETED (2026-09-03; ONE 512 MiB / 24-worker real workload authorized by the session prompt, protocol v2 section 7; requested 24, executor processes 22, non-empty workers 22 - discrepancy recorded; all gates PASS; classification A) CHUNK 584 execution      = NOT AUTHORIZED
P3-S25.1AJ (chunks 584-605) = COMPLETED (2026-09-03; ONE 512 MiB / 24-worker real workload authorized by the session prompt, protocol v2 section 7; requested 24, executor processes 22, non-empty workers 22 - discrepancy recorded; all gates PASS; classification A) CHUNK 606 execution      = NOT AUTHORIZED
P3-S25.1AK (chunks 606-627) = COMPLETED (2026-09-04; ONE 512 MiB / 24-worker real workload authorized by the session prompt, protocol v2 section 7; requested 24, executor processes 22, non-empty workers 22 - discrepancy recorded; all gates PASS; classification A) CHUNK 628 execution      = NOT AUTHORIZED
P3-S25.1AL (chunks 628-649) = COMPLETED (2026-09-04; ONE 512 MiB / 24-worker real workload authorized by the session prompt, protocol v2 section 7; requested 24, executor processes 22, non-empty workers 22 - discrepancy recorded; all gates PASS; classification A) CHUNK 650 execution      = NOT AUTHORIZED
P3-S25.1AM (chunks 650-671) = COMPLETED (2026-09-04; ONE 512 MiB / 24-worker real workload authorized by the session prompt, protocol v2 section 7; requested 24, executor processes 22, non-empty workers 22 - discrepancy recorded; all gates PASS; classification A) CHUNK 672 execution      = NOT AUTHORIZED
P3-S25.1AO (chunks 672-693) = COMPLETED (2026-09-05; ONE 512 MiB / 24-worker real workload authorized by the session prompt, protocol v2 section 7; requested 24, executor processes 22, non-empty workers 22 - discrepancy recorded; all gates PASS; classification A) CHUNK 694 execution      = NOT AUTHORIZED
P3-S25.1AP (chunks 694-715) = COMPLETED (2026-09-05; ONE 512 MiB / 24-worker real workload authorized by the session prompt, protocol v2 section 7; requested 24, executor processes 22, non-empty workers 22 - discrepancy recorded; all gates PASS; classification A) CHUNK 716 execution      = NOT AUTHORIZED
P3-S25.1AQ (chunks 716-737) = COMPLETED (2026-09-06; ONE 512 MiB / 24-worker real workload authorized by the session prompt, protocol v2 section 7; requested 24, executor processes 22, non-empty workers 22 - discrepancy recorded; all gates PASS; classification A)
P3-S25.1AR (chunks 738-759) = COMPLETED (2026-09-06; ONE 512 MiB / 24-worker real workload authorized by the session prompt, protocol v2 section 7; requested 24, executor processes 22, non-empty workers 22 - discrepancy recorded; all gates PASS; classification A) CHUNK 760 execution      = NOT AUTHORIZED
Authorizing document     = docs/CURRENT_DECISION_GATE.md (owner decision)

Interpretation rule (see protocol v2 section 6-7): next_chunk = 628 in section B is a FACT. It is NOT an authorization. No execution may begin until factual state (section B) and authorization state (section D) reconcile via an explicit owner decision recorded in CURRENT_DECISION_GATE.md.


E. Frozen foundations

The following are FROZEN and MUST NOT change without separate, explicit owner authorization:

- Runtime F1 (event contract)         -> docs/P3_S11_EVENT_CONTRACT_REPAIR.md
- Runtime F2 (zone contract)          -> docs/P3_S12_ZONE_CONTRACT_REPAIR.md
- Runtime F3 (candidate setup)        -> docs/P3_S13_CANDIDATE_SETUP_REPAIR.md
- Runtime F4 (MTF training alignment) -> docs/P3_S14_MTF_TRAINING_ALIGNMENT.md
- Candidate Setup canonical           -> docs/SNIPERGOLD_CANONICAL_SETUP_CONTRACT_v1.md,
                                         docs/SMC_CANDIDATE_SETUP_SPEC_v1.md
- Feature contract                    -> docs/FEATURE_CONTRACT.md
- Label contract v1                   -> docs/P3_S16_LABEL_CONTRACT.md,
                                         docs/P3_S18A_LABEL_CONTRACT_REVIEW.md
- Corrected UTC-clock research M30 basis -> P3-S22.3 (docs/P3_S22_3_RESEARCH_M30_REPAIR.md)
- Corrected population basis (707 / 610 / 579) -> P3-S22.4 (docs/P3_S22_4_CORRECTED_POPULATION_VALIDATION.md)
- P3-S20 frozen baseline              -> docs/P3_S20_WALK_FORWARD_BASELINE.md (FROZEN)
- P3-S23 corrected baseline           -> docs/P3_S23_CORRECTED_POPULATION_ML_BASELINE.md
- P3-S24 conclusion                   -> D - CURRENT EDGE NOT DEMONSTRATED

Full list (paths / versions / authorization): docs/FROZEN_ASSET_REGISTER.md and docs/AUTHORIZED_CHANGE_MATRIX.md.


F. Historical results - status

Result Status
Legacy SniperGold_ML.mqh + old directional-drift MLP FROZEN LEGACY / REFERENCE
Legacy calibrated AUC 0.627 / 0.621 (old feed) LEGACY / HISTORICAL ONLY
P3-S20 frozen walk-forward pooled 0.5792 (old population) FROZEN HISTORICAL RESULT
P3-S22.3 corrected UTC M30 construction CORRECTED RESEARCH BASIS
P3-S23 corrected pooled ROC-AUC 0.5235 CORRECTED-POPULATION RESULT
P3-S24 conclusion D - CURRENT EDGE NOT DEMONSTRATED

G. Current chunk policy

SAFE_MAX_CHUNK_SIZE  = 24 MiB
SAFE_CHUNKS_PER_TURN = NOT ESTABLISHED (broader P3-S25.1D multi-chunk benchmark
                       OPEN; verifier recovery isolated in P3-S25.1D-R only)
operational workers  = 24 (P3-GOV-7 promotion 2026-09-01; previous operational
                       value 8; SAFE_MAX_CHUNK_SIZE / 512 MiB workload /
                       SAFE_CHUNKS_PER_TURN unchanged; CHUNK 364 = NOT AUTHORIZED)

periodic 0-9 audit      = PASS_WITH_LIMITATION (documentation-only no-chain-hash)
chunk-size invariance   = PASS (12/24/48/64 MiB byte-identical)
reproducibility         = PASS (chunks 0-9 sequential replay, chunk10 9/9 exact,
                                 chunk11 9/9 EXACT)
evidence                = ml/p3/p3_s251_external_ingest/real_run_chunk_00000*.json
                          ml/p3/p3_s251_external_ingest/output/s251_real_*_manifest.json
periodic audit evidence = ml/p3/p3_s251_external_ingest/output/p3_s251_periodic_audit_10.json
chunk-11 gov6a controls = ml/p3/p3_s251_external_ingest/output/s251_chunk11_controls_before.json
                          ml/p3/p3_s251_external_ingest/output/s251_chunk11_controls_after.json

H. Next session behaviour

Read this file, the newest session handover, the decision gate and the session-start protocol v2, then verify Git, then reconcile FACTUAL EXECUTION STATE with AUTHORIZATION STATE. No work is authorized by this file. Only an explicit owner decision recorded in docs/CURRENT_DECISION_GATE.md (or the current session prompt) authorizes execution.

End of P3-GOV-3 current project state (FACTUAL_STATE).