Aliwkarimi98/SniperGold_ML
Watch
1
0
Fork
You've already forked SniperGold_ML
 1
forked from chiki2bum2/SniperGold_ML
Code Pull requests Activity
SniperGold_ML/docs/P3_S16_SETUP_DATASET_CONTRACT_v1.md

362 lines
16 KiB
Markdown
RawPermalinkNormal ViewHistory

docs: record P3-S.16 setup dataset and label contract
2026-08-23 07:26:39 +07:00
# P3-S.16 SETUP-LEVEL DATASET CONTRACT v1 — SNIPERGOLD_ML
```text
Status : AUTHORITATIVE CONTRACT FOR THE LITERATURE-LEVEL TRAINING OBSERVATION.
Scope : Defines the canonical ML observation unit (ONE Candidate Setup) and
its deterministic transformation into ONE training observation, with
the separation of (i) immutable setup identity, (ii) derived training
features, and (iii) outcome labels. This contract is the ORACLE for
the P3-S.16 test suite and dataset generator (ml/p3/setup_dataset/).
Session : P3-S.16 — Setup-Level Dataset / Label Contract Implementation.
Date : 2026-08-23
Status tag: CONTRACT v1 (P3-S.16). Numeric research parameters are explicitly
OPEN where noted; they must NOT be selected by AUC/PF (session §29).
Provenance: P3-S.15 decision docs/P3_S15_ML_PATH_ARCHITECTURE_DECISION.md
P3-S.15 docs/P3_S15_SETUP_LEVEL_TRAINING_CONTRACT_PROPOSAL.md
frozen P3-S.10 docs/SNIPERGOLD_CANONICAL_SETUP_CONTRACT_v1.md (§E/§F)
P3-S.13 docs/P3_S13_CANDIDATE_SETUP_REPAIR.md (AFCandidateSetup F3)
P3.2/P3.2.2 forensic (docs/P3_2_LABEL_FORENSIC.md,
docs/P3_2_2_DEOVERLAP_SURVIVAL.md)
Human verification : REMAINS CANCELLED (historical only).
```
---
## A. OBJECTIVE
> How do we turn ONE verified Candidate Setup into ONE reproducible training
> observation with a causally valid setup-outcome label, WITHOUT reintroducing
> the overlap, redundancy, and look-ahead problems of the old bar-level ML
> pipeline?
This contract answers that question at the CONTRACT level. It fixes the
observation unit, its immutable identity, the separation boundaries, the
feature snapshot discipline, and the dataset mechanics — so that the generator
(`ml/p3/setup_dataset/`) can build a deterministic, auditable dataset and the
label computation can be validated against this contract as ORACLE.
---
## B. OBSERVATION UNIT — ONE CANDIDATE SETUP
```text
ONE TRAINING OBSERVATION = the occurrence of ONE CANDIDATE_SETUP
(entity AFCandidateSetup, frozen P3-S.13 §C), captured at the moment its chain
completes and (optionally) at the deterministic ENTRY timestamp (§D below).
The Candidate Setup is the ONLY valid training unit because:
- it is the runtime decision granularity (P3-S.15 §6.3),
- its identity is CAUSAL and DETERMINISTIC (P3-S.13 §J),
- its lifecycle is fully specified (P3-S.13 §D),
- the bar-level unit FAILED unit-validity (P3.2.2 §K: 97-99% redundant,
1.5% retention) — the setup unit removes per-bar redundancy.
```
```text
IMMUTABLE SETUP IDENTITY (the observation key — immutable, never derived):
setup_id : monotonic integer at CANDIDATE_SETUP.
direction : +1 bullish / -1 bearish (chain direction).
creation_timestamp : M15 creation_bar open time (CANDIDATE_SETUP bar).
creation_bar_index : absolute M15 bar index (monotonic, repo-defined).
setup_state : CANDIDATE_SETUP (or M3_CONFIRMED) at capture.
sweep_onset : Liquidity EVENT onset bar index + sweep_direction.
choch_onset : CHoCH/MSS EVENT onset bar index + choch_direction.
zone_type : OB | FVG.
zone_formation_timestamp : zone formation bar open time.
zone_bounds : {lower_bound, upper_bound} at creation (F2 contract).
zone_mit_at_creation : F2 mitigation state when the chain consumed it.
m15_entry_condition_ts : the M15 decision bar when the M15 entry condition
held (bar that advanced chaim to ENTRY_ARMED; the
creation bar for the CANDIDATE_SETUP completion).
m3_confirmation_ts : nullable — M3 bar timestamp if M3_CONFIRMED, else null.
provenance : contract version + generator version + source
dataset hash (see §K).
source_timeframe_set : {H4, M30, M15, M3} with as-of timestamps per TF.
UNIQUENESS: two setups are the same observation iff their CAUSAL key matches:
(direction, sweep_onset, choch_onset, zone_formation_timestamp,
creation_timestamp). (P3-S.10 §K, P3-S.13 §J.)
```
```text
SEPARATION OF THREE STORES (this contract — NEVER mixed):
STORE 1 — IMMUTABLE IDENTITY (§B): fixed at creation; part of the key;
never derived, never a label, never a
learning feature that encodes the outcome.
STORE 2 — DERIVED TRAINING FEATURES (§G / proposal): functions of causally
available SMC state as-of the ENTRY
timestamp. Respects FEATURE_CONTRACT.md
(not modified).
STORE 3 — OUTCOME LABEL (§ label contract): the TP-before-SL outcome
measured from ENTRY forward. NEVER a
feature of STORE 2.
```
---
## C. SETUP IDENTITY vs FEATURE vs LABEL — SEPARATION RULE
```text
The three stores must remain DISTINCT in the dataset schema:
- a field that is part of the CAUSAL IDENTITY is not a learning feature;
- a LEARNING FEATURE must be causally available at ENTRY (no future data);
- the LABEL is a function of FUTURE price after ENTRY and must never appear
in the feature store.
The dataset row therefore has three explicit namespaces:
identity_* : immutable key + provenance (Store 1)
feature_* : causally-available SMC features at entry (Store 2)
label_* : outcome class + horizons + levels (Store 3)
```
---
## D. ENTRY TIMESTAMP (MANDATORY — §10 of the brief)
```text
DECISION: the deterministic ENTRY timestamp for outcome measurement is
entry_timestamp := CLOSE of the CANDIDATE_SETUP creation bar
(= M15 creation_bar close time),
at timeframe M15.
Justification (causality, NOT profitability):
- The Candidate Setup COMPLETES at creation_bar (chain whole, direction
fixed, M15 entry condition already satisfied within the chain). The first
closed bar at/after completion is the earliest causally-available
executable point.
- M3 confirmation is OPTIONAL and occurs AFTER creation; it is a
confirmation, not a validity gate (§M canonical: absence does not
invalidate; contrary reduces only micro-score). Therefore it CANNOT be the
entry timestamp without breaking the "optional" semantics.
- M15 entry-condition (ENENTRY_ARMED) and zone membership are already
satisfied at creation_bar by construction; no separate entry re-trigger.
- This matches the P3.2/P3.2.2 entry semantics (A = close of decision bar,
robustness B = next-bar open, |A-B| ~ 0.008 ATR, P3.2 §6).
ENTRY CONTRACT:
entry_timestamp : creation_bar CLOSE (M15), computed as open_time + 900s.
entry_price_source : close of the creation_bar.
price_convention : bid for long / ask for short is a FUTURE EXECUTION
detail; for the LABEL CONTRACT (research, not trade)
a single mid/reference price is used per symbol, with
execution spread/slippage OUT of scope (see §H below).
price_reference : the creation_bar close (mid, contract-defined).
spread handling : NOT applied in the research label (would require an
execution model). Documented; applied only in a later
execution-aware phase, never in P3-S.16.
timeframe : M15 (fixed by the setup carrier).
If entry could not be fixed deterministically the project would STOP as
BLOCKED. It is fixed; no blocker.
```
---
## E. DATA CONTRACT FIELDS (canonical training observation row)
```text
---- identity_* (Store 1) ----
identity_setup_id
identity_direction
identity_creation_timestamp
identity_creation_bar_index
identity_setup_state
identity_sweep_onset_index
identity_sweep_direction
identity_choch_onset_index
identity_choch_direction
identity_zone_type
identity_zone_formation_timestamp
identity_zone_lower
identity_zone_upper
identity_zone_mit_at_creation
identity_m15_entry_condition_ts
identity_m3_confirmation_ts (nullable)
identity_provenance_*
identity_source_timeframes
---- feature_* (Store 2) — PROPOSAL, see §G / P3-S.15 §8 ----
feature_h4_context (gate direction as-of entry)
feature_m30_context (gate direction as-of entry)
feature_sweep_age_bars
feature_choch_age_bars
feature_choch_latency_after_sweep
feature_zone_age_bars
feature_zone_width_atr
feature_price_in_zone_offset_atr
feature_m15_distance_to_zone_atr
feature_displacement_at_creation
feature_m15_entry_condition
feature_m3_present
feature_m3_direction
feature_sweep_size_atr
feature_atr_at_entry
---- label_* (Store 3) — see label contract ----
label_outcome (WIN | LOSS | UNRESOLVED | AMBIGUOUS)
label_tp_level
label_sl_level
label_horizon_bars
label_tp_hit_bar (nullable)
label_sl_hit_bar (nullable)
label_timeout (bool)
label_invalidated_early (secondary; observation may be resolved by
lifecycle invalidation — recorded, see label contract)
label_contract_version
```
---
## F. SETUP RELATIONSHIP / DE-OVERLAP (§18 — independence of observations)
The training unit is now the Candidate Setup. Two DISTINCT setups may still
overlap in TIME (their outcome windows can share bars). For ML label
independence (P3.2.2 §J: serial dependence must be ~0), define causally:
```text
SAME SETUP : identical identity key — same observation (never duplicated).
FOLLOW-ON SETUP : a new distinct setup formed while the PREVIOUS setup's
outcome window is STILL OPEN (creation_timestamp is within
[prev.entry, prev.entry+horizon]). Its outcome window
overlaps the previous one -> NOT an independent sample.
INDEPENDENT SETUP : a distinct setup whose ENTRY occurs AFTER the previous
setup's outcome window has CLOSED (gap-separated by the
setup-level purge; see below).
OVERLAPPING SETUP : generic term for a setup sharing outcome-window bars with
another distinct setup.
DE-OVERLAP RULE (setup-level, analogous to P3.2.2 §B adapted to setups):
define an EPISODE as a maximal run of distinct setups where successive
entries are within the outcome horizon of the previous entry.
LEAD SETUP (kept) : the FIRST setup of each episode.
FOLLOW-ON (reported): all later setups of the episode (excluded from the
primary independent dataset, but preserved & reported,
NOT deleted).
VERIFY: for the de-overlapped stream, next_lead_entry - lead_entry > horizon.
PURPOSE: independent training observations. NOT a means to reduce samples for
better metrics; not driven by AUC.
```
---
## G. FEATURE SNAPSHOT CONTRACT (PROPOSAL — Store 2)
Every feature is causally available at entry_timestamp. This section is a
PROPOSAL from P3-S.15 §8 / §20, repeated here as the contract must bind them.
It does NOT modify docs/FEATURE_CONTRACT.md (which remains unchanged and
bar-level; the setup-level feature namespace is NEW and lives only in this
contract proposal).
```text
FEATURE-TIMESTAMP DISCIPLINE (per feature):
feature source TF as-of rule avail at entry? leakage?
h4_context narrative state H4 newest closed H4 bar close_time <= entry YES NO
m30_context context state M30 newest closed M30 bar close_time <= entry YES NO
sweep_age_bars liquidity event M15 entry_bar - sweep_onset YES NO
choch_age_bars structure event M15 entry_bar - choch_onset YES NO
choch_latency structure changeM15 (choch_onset - sweep_onset) YES NO
zone_age_bars zone formation M15 entry_bar - zone_formation YES NO
zone_width_atr zone M15 (upper-lower)/ATR(entry) YES NO
price_in_zone zone/price M15 (entry_close - zone)/width, etc. YES NO
dist_to_zone_atr zone/price M15 distance(entry_close, zone)/ATR YES NO
displacement_at_creation attribute M15 value at creation_bar (closed) YES NO
m15_entry_condition condition M15 boolean at creation_bar YES NO
m3_present / dir micro confirmation M3 m3_bar <= W_m3 after creation (causal) YES NO
sweep_size_atr event M15 sweep move / ATR(entry) YES NO
atr_at_entry ATR M15 ATR(14) over closed bars ending at entry YES NO
RULE: if feature availability is uncertain at ENTRY, the generator must
BLOCK/fail that row (do not infer). (§21.)
```
---
## H. PRICE / SPREAD / EXECUTION (label-basis)
```text
The research label uses a single REFERENCE PRICE (mid = (bid+ask)/2, or a
single close series) at the entry bar. TP/SL are computed as price offsets in
ATR units from that reference. Spread/slippage/commission are NOT applied in
P3-S.16 (an execution model is a later, separate phase). The contract records
this limitation explicitly so the research label is never misread as an
execution P&L.
```
---
## I. DATA SPLIT / REPRODUCIBILITY (§23, §24) — FUTURE REQUIREMENT (defined, not run)
```text
SPLIT (future, not executed in P3-S.16):
- temporal split by entry_timestamp (chronological; no shuffle across time).
- purge gap == outcome horizon between train/validation/test label windows.
- NO overlap between label windows across split boundaries.
- NO future information across any boundary.
- exact split percentages = FUTURE RESEARCH PARAMETERS (OPEN; not optimized).
REPRODUCIBILITY (every future dataset generation must record):
- source-data hash (per symbol/TF feed),
- setup-dataset hash (deterministic over the generator),
- contract version (this doc + label contract),
- feature version, label version,
- seed where applicable (deterministic generator may be seed-free),
- generation timestamp.
The dataset must be reproducible from the same source + contract alone.
```
---
## J. LEGACY COMPATIBILITY (§25)
```text
The new setup-level dataset is SEVERED from the legacy bar-level pipeline:
LEGACY_L1_BAR_DRIFT : 19 features, 24-bar direction drift, per M15 bar, MLP.
FUTURE_L3_SETUP_OUTCOME : one Candidate Setup -> setup outcome (this contract).
The two are NEVER mixed. The legacy FEATURE_CONTRACT.md and the legacy model
remain reference-only and are NOT reused for the setup-outcome target.
```
---
## K. PROVENANCE (future observation provenance — §22)
```text
Every observation row must preserve:
setup_id, symbol, timeframe, creation_timestamp, entry_timestamp,
source-data provenance (dataset/hash), contract version, feature-contract
version, label-contract version.
The dataset must not depend on undocumented external state.
```
---
## L. CONTRACT VERDICT / BOUNDARY
```text
This contract settles the SEMANTICS (unit, identity, entry, label family,
de-overlap, sampling, feature discipline, reproducibility). The following are
carried as OPEN numeric/parameter requirements (documented; NOT optimized):
- TP/SL constants (ATR multiples) -> label contract OPEN
- outcome horizon bars (semantic basis fixed) -> label contract OPEN
- split percentages / purge gap=numeric -> §I OPEN
- W_m3 (M3 confirmation lookback) -> frozen F3 placeholder
- exact feature numeric lookbacks -> future feature contract
These OPEN parameters do NOT invalidate the CONTRACT semantics; they are the
"OPEN PARAMETERS" recorded per the session §33 classification intent, pending
authorized determination WITHOUT AUC/PF.
```
---
## M. EXPLICITLY UNCHANGED / NOT MODIFIED
```text
Candidate Setup runtime, F1, F2, F4 runtime MTF, FEATURE_CONTRACT.md,
SniperGold_ML.mqh, production .mq5/.mqh, legacy model: NONE changed by this
contract. No training, no model export, no deployment.
```
*End of P3-S.16 setup-level dataset contract v1.*
Copy permalink