Aliwkarimi98/SniperGold_ML
Watch
1
0
Fork
You've already forked SniperGold_ML
 1
forked from chiki2bum2/SniperGold_ML
Code Pull requests Activity
SniperGold_ML/docs/P3_GOV_7_24_WORKER_OPERATIONAL_PROMOTION.md

11 KiB
Raw Permalink Blame History

P3-GOV-7 — 24-WORKER OPERATIONAL CONFIGURATION PROMOTION — DECISION / PROMOTION RECORD

DOCUMENT ROLE      : DECISION / PROMOTION RECORD
AUTHORITATIVE SOURCE: P3-S25.1Q machine evidence
                      (ml/p3/p3_s251_external_ingest/workload_benchmark/
                      worker_scaling_q/output/p3_s251q_decision.json,
                      p3_s251q_worker_24.json, p3_s251q_machine.json,
                      p3_s251q_resource.json, p3_s251q_checkpoint_sha_*.json)
                      + P3-S25.1R machine evidence
                      (ml/p3/p3_s251_external_ingest/output/
                      s251_real_workload20_manifest.json,
                      s251_real_workload20_evidence.json,
                      s251_real_workload20_interruption.json,
                      s251_real_workload20_mutations.json,
                      s251_real_workload20_controls_before/after.json,
                      s251_real_workload20_checkpoint_sha_before/after.json)
                      + canonical real checkpoint
                      (ml/p3/p3_s251_external_ingest/output/
                      s251_real_checkpoint.json, READ-ONLY)
PHASE              : P3-GOV-7 - STRICT GOVERNANCE + CONFIGURATION PROMOTION
Date               : 2026-09-01
AUTHORITY          : RECORD OF THE OWNER DECISION TO PROMOTE THE VALIDATED
                      24-WORKER EXECUTION CONFIGURATION INTO THE OPERATIONAL
                      INGESTION CONFIGURATION. THIS DOCUMENT GRANTS NO
                      EXECUTION AUTHORIZATION. CHUNK 210 REMAINS
                      NOT AUTHORIZED (factual cursor only).

1. Evidence basis — P3-S25.1Q (synthetic 24-worker scaling candidate)

Test               : P3-S25.1Q WORKER-SCALING OPTIMIZATION EXPERIMENT
Classification     : A - OPTIMIZATION CANDIDATE IDENTIFIED
Candidate          : requested workers = 24
Synthetic dataset  : disposable, schema-identical to the real Tickstory source
                     (scratch/p3_s251e_primary_fixture_24MiB.csv, gitignored;
                     sha256 2a2691159f1a250b537c82a623539190a59ffd8ea4089ec54245a8f8a7800be5)
Correctness gates  : 14/14 PASS (machine: worker_counts["24"].all_gates_pass = true)
                     correctness_14_point / boundary / state_carry /
                     interruption_resume / mutation_detection / reproducibility /
                     no_protected_state_mutation = all true
Speedup vs 8-worker baseline : 2.500 (machine: speedup_vs_8 = 2.5; span 19.543 s)
Efficiency          : 0.833 (machine)
max_safe_tested     : 24 (machine) | preferred_worker_count : 24 (machine)
baseline_worker_count : 8 (machine)
Reference (canonical) : rows 11,422,782 | output_sha256 cfa8c084f9bae714a4a15cb1da27e2442bd72bd11aee3f3604304a203e02ed4a
                        | final_state_hash 757a4d63a0f13c2757281402d5beea4278c982c2e29e36375faa30b9565994ff
Checkpoint protection : checkpoint byte-identical before/after (8c6cad1c...)

2. Evidence basis — P3-S25.1R (REAL 512 MiB / 24-worker validation)

Test               : P3-S25.1R REAL 512 MiB / 24-WORKER VALIDATION WORKLOAD
Classification     : A - REAL 24-WORKER VALIDATION PASSED (15/15 gates)
Run                : P3_S251_REAL_TICK_RUN_001, real Tickstory XAUUSD tick source
                     (D:\TradingTerminal\HFM Metatrader 5\MQL5\Files\XAUUSD_mt5_ticks.csv,
                     size 34,473,661,010 bytes, sha256 5252ce8f5b0e7d286b71a3172c7ef9c1009c8e4b1254ac67400a803fcd85cba1)
Real byte range    : [4483711426, 5020582350) (chunks 188-209; line-safe boundary)
Correctness        : 15/15 gates; parallel == canonical atomic == canonical
                     single-stream (EXACT; machine: gates.parallel_equals_canonical_atomic
                     = true, parallel_equals_canonical_stream = true,
                     canonical_atomic_equals_stream = true, all_gates = true)
Interruption/resume: all_match = true (21/21 parallel worker boundaries +
                     canonical stop-4, tail re-executed; disposable copies)
Mutation control   : 8/8 DETECTED (disposable; all 5 required classes covered)
Checkpoint integrity : advanced exactly once; identity_after OK; deterministic read-back
Environment        : Process Lasso NOT RUNNING, intervention NONE (tasklist)
Span               : processing wall-clock 15.85 s (machine)

2.1 Requested / validated worker population (REAL)

requested workers            = 24 (machine: requested_workers = 24)
executor process count       = 22 (machine: 22 atomic chunks cannot populate
                                   24 workers; legitimate empty-task case)
non-empty worker count       = 22 (machine: nonempty_worker_count = 22)
empty partition worker ids   = [22, 23]  (machine)
actual_matches_requested     = false (recorded discrepancy; no semantic impact;
                                   every gate exact)

2.2 Resource evidence (REAL, machine-derived)

CPU (machine sampled)        : average 0.9169 | peak 0.9796 (24 logical CPUs)
total physical RAM           : 51,468,738,560 bytes (~47.93 GiB)
available RAM before         : 45,181,456,384 bytes
min available RAM during     : 33,462,333,440 bytes
available RAM after          : 45,203,243,008 bytes
aggregate worker RSS         : 2,327,801,856 bytes
coordinator RSS              : 40,083,456 bytes
total Python RSS             : 2,367,885,312 bytes
per-worker tracemalloc peak  : 239,691,346 bytes (bounded by 24 MiB atomic chunk)
worker imbalance max/mean    : 1.0779
dispatch/spawn overhead      : 3.079 s | merge overhead : 0.2659 s

3. Operational decision (owner authorization, session P3-GOV-7)

Operational worker count = 24
Previous operational value = 8 (P3-S25.1Q baseline_worker_count; the
   8-worker configuration used by the operational real-run series
   workload12-19)
Decision : PROMOTE the REAL-VALIDATED 24-worker execution configuration
   into the operational ingestion configuration. Effective immediately as
   the operational worker-count configuration of the steady-state
   ingestion runner.
Scope    : CONFIGURATION PROMOTION ONLY. No real Tickstory data is
   processed by this phase. No workload is run. No checkpoint is advanced.

4. Effective operational configuration (after promotion)

operational workers      = 24
SAFE_MAX_CHUNK_SIZE      = 24 MiB (UNCHANGED)   = 25,165,824 bytes
nominal workload         = 512 MiB (UNCHANGED)  = 536,870,912 bytes
parser                   = UNCHANGED (s251_parse, parser_version p3_s251_parser_v1)
aggregator               = UNCHANGED (s251_aggregate incl. P3-S25.1I repair)
merge semantics          = UNCHANGED (source byte order, single coordinator)
checkpoint semantics     = UNCHANGED (line-safe boundary, atomic checkpoint,
                           advance exactly once per workload)
SAFE_CHUNKS_PER_TURN     = UNCHANGED (NOT ESTABLISHED - no inference
                           of a policy value from the 24-worker count)

5. Unchanged controls / state

atomic chunk size        = UNCHANGED (24 MiB; SAFE_MAX_CHUNK_SIZE 25,165,824)
workload size            = UNCHANGED (512 MiB; 536,870,912)
real checkpoint          = UNCHANGED (byte-identical; SHA before == after)
checkpoint cursor        = last_completed_chunk 209 | next_chunk 210 |
                           next_byte_start 5,020,582,350 (5020582350)
CHUNK 210                = NOT AUTHORIZED (factual state ONLY; this promotion
                           grants no execution authorization)
production MQL5/MQH      = UNCHANGED
frozen contracts         = UNCHANGED
historical evidence      = UNCHANGED (runs, manifests, evidence, reports)
P3-S25.1Q benchmark harness = UNCHANGED (disposable benchmark only)

6. Implementation

CURRENT_PROJECT_STATE.md   : v11 - OPERATIONAL EXECUTION CONFIGURATION
                             (operational workers = 24) recorded; factual
                             cursor unchanged.
CURRENT_DECISION_GATE.md   : v13 - promotion recorded (OPERATIONAL WORKER
                             CONFIGURATION = 24, PROMOTED); CHUNK 210 remains
                             NOT AUTHORIZED; next_chunk 210 factual.
Operational runner source  : verified - the current steady-state ingestion
                             runner (s251_stageb_workload20.py) already
                             embeds WORKERS = 24, ATOMIC_24 = 25,165,824 and
                             MAX_WORKLOAD_BYTES = 536,870,912, and references
                             the unchanged parser/aggregator.
Historical 8-worker runners: NOT modified (s251_stageb_workload12..19.py stay
                             byte-identical run records of the 8-worker
                             operational series; their evidence is immutable).

7. Evidence paths (reference; no datasets reproduced here)

Q decision        : ml/p3/p3_s251_external_ingest/workload_benchmark/worker_scaling_q/output/p3_s251q_decision.json
Q worker 24       : ml/p3/p3_s251_external_ingest/workload_benchmark/worker_scaling_q/output/p3_s251q_worker_24.json
Q report          : docs/P3_S25_1Q_WORKER_SCALING_OPTIMIZATION_REPORT.md
R manifest        : ml/p3/p3_s251_external_ingest/output/s251_real_workload20_manifest.json
R evidence        : ml/p3/p3_s251_external_ingest/output/s251_real_workload20_evidence.json
R interruption    : ml/p3/p3_s251_external_ingest/output/s251_real_workload20_interruption.json
R mutations       : ml/p3/p3_s251_external_ingest/output/s251_real_workload20_mutations.json
R controls        : ml/p3/p3_s251_external_ingest/output/s251_real_workload20_controls_before.json
                    ml/p3/p3_s251_external_ingest/output/s251_real_workload20_controls_after.json
R checkpoint SHA  : ml/p3/p3_s251_external_ingest/output/s251_real_workload20_checkpoint_sha_before.json
                    ml/p3/p3_s251_external_ingest/output/s251_real_workload20_checkpoint_sha_after.json
R report          : docs/P3_S25_1R_REAL_512MIB_24WORKER_VALIDATION_REPORT.md
checkpoint        : ml/p3/p3_s251_external_ingest/output/s251_real_checkpoint.json (READ-ONLY)

8. Limitations

1. Requested 24 worker processes; 22 atomic chunks -> 22 executor processes,
   [22,23] empty partitions. Recorded; gates exact; machine-derived.
2. 24 workers near-saturate the 24 logical CPUs (Q: avg 0.961/peak 1.000;
   R: avg 0.9169/peak 0.9796). Scaling above 24 was NOT tested (not in scope).
3. The promotion changes ONLY the operational worker-count setting; the
   historical 8-worker runner snapshots are intentionally NOT rewritten, so
   the pre-promotion operational value 8 remains visible only in those
   historical run records and their evidence.
4. The full 34.5 GB ingestion remains NOT COMPLETE; CHUNK 210 at byte
   offset 5,020,582,350 awaits a NEW explicit owner execution authorization.

9. Classification

CLASSIFICATION : A - 24-WORKER CONFIGURATION PROMOTED
Wording note    : "A" is not forced. The promotion is based on existing
                  machine evidence only (P3-S25.1Q + P3-S25.1R); no
                  benchmark was re-run, no real Tickstory byte was read,
                  no checkpoint was advanced, and all exact values above
                  are machine-derived.

End of P3-GOV-7 promotion record.