P3-S.1 LIQUIDITY SWEEP HUMAN–MACHINE VALIDATION — PROTOCOL & INTEGRITY

Date       : 2026-08-22
Session    : P3-S.1 — Machine Annotation Freeze (f7_v2) -> Human Annotation (blind)
Checkpoint : b519a34 (P3-S.0 f7 fix committed; working tree clean)
Machine    : f7_v2 (f7 event lifecycle corrected; f10/f11 UNCHANGED)
Status     : MACHINE FREEZE = INTEGRITY PASS; HUMAN ANNOTATION = PENDING (awaiting annotators)

1. MACHINE v2 INTEGRITY — PASS

Automatic audit: ml/p3/smc_semantic/integrity_audit.py -> output/integrity_audit_f7_v2.json

A. CASE SET (v1 == v2)
   count              : 60 == 60
   case IDs           : IDENTICAL (SGML_SWEEP_001..060)
   decision timestamp : 0 diff
   symbol             : 0 diff (XAUUSD)
   decision_tf        : all M15
   case_set_hash      : da3a0d0154b7d4b2b853b088e1948fd780d7677fbb00929eef24fc4e0132733a
   -> NO CASE ADDED / REMOVED / RESAMPLED

B. MACHINE COUNTS
   machine YES = 45 | machine NO = 15
   all 15 NO  : age > SEQ_WINDOW(40) -> STALE_EXPECTED (NOT due to timestamp/missing/parser/N/A/TF)
   no NO has active f10/f11

C. V1 -> V2 DIFF (f7_v1_vs_v2_comparison.json)
   changed 22 | unchanged 38 | events removed 22 | events added 0
   anomalies outside lifecycle : 0 (f10/f11/close_price/timeframe UNCHANGED for changed cases)
   -> ALL changes = event expiration / lifecycle correction

D. FIELD CONTRACT
   22 mandatory columns present; N/A used for non-applicable fields (75 = 15 NO cases x 5 fields)
   no human-interpretation fields

Provenance v2 (corrected in this session):

annotation_version : f7_v2
source_commit      : b519a34
machine_v2_sha     : 6d7c0b41aeb6d8ab64dcda9d0723f8e82abce771ab86fc9834ddeb874dd58583
v1_legacy_sha      : e083913648f2059c7ebbaa049b6443e6c5ada3a3c7092ae7b391115d19c710e2
case_set_sha       : da3a0d0154b7d4b2b853b088e1948fd780d7677fbb00929eef24fc4e0132733a
feature_contract   : C44CC6F2B740C32D06F776BD7C3E669DC5A8A6DE0484230544EBFFCF517D38DD
dataset            : e85a0861... (P2.6)

2. MACHINE FREEZE

machine_annotations_f7_v2.csv = machine-side reference (FROZEN).

No further f7 semantics changes in this session.
f10/f11, OB, FVG, CHoCH, timeframe design, ML, regime, label contract: NOT touched.

3. BLINDING ARCHITECTURE

HUMAN PACKAGE (ml/p3/smc_semantic/human_package/)
  cases.csv                         : 60 cases (id, symbol, decision_timestamp, tf, note)
  human_annotation_template_f7.csv  : blank template (§7 format)
  HUMAN_ANNOTATION_PROTOCOL.md      : blind rules + chart rule + procedure
  context_m15/<case_id>.csv         : 200 M15 bars up to decision_timestamp (snapshot does NOT exceed it)

MACHINE PACKAGE (ml/p3/smc_semantic/machine_package/) — BLINDED
  machine_annotations_f7_v2.csv     : machine decisions
  machine_reasons_f7_v2.txt         : readable reasons
  README_BLINDED.md                 : warning: annotators MUST NOT open

Human annotators receive ONLY human_package/.

4. CHART DATA RULE

Annotator sees the XAUUSD chart (M15 decision; H4/M30/M3 context) ONLY up to decision_timestamp.
Must not see future candles / signals / trade outcomes / machine annotations.
Context snapshot context_m15/*.csv ends EXACTLY at decision_timestamp.

5. TWO-ANNOTATOR PROTOCOL

Human A = primary annotator   -> output/human_A_f7.csv
Human B = independent reviewer -> output/human_B_f7.csv   (does not see A)

Results are stored separately; not merged until both are done.
If only 1 annotator is available: finish Human A first; the final ground truth is NOT created before a second annotator / adjudication.

6. INTER-RATER & ADJUDICATION

comparison.py (run after A & B finish):
  human_interrater_f7_report.json :
    agreement rate; YES/NO; reference; direction; timeframe; rejection;
    confidence distribution; disagreement cases
  disagreement -> ADJUDICATION REQUIRED

adjudication -> output/human_adjudicated_f7.csv (status YES/NO/AMBIGUOUS)
  adjudicator sees chart + protocol + metadata — NOT the machine result
  goal = HUMAN CONSENSUS, not machine agreement

7. MACHINE vs HUMAN CONSENSUS

After consensus: comparison.py -> human_machine_f7_comparison.json

TP / FP / FN / TN ; Precision / Recall / F1
Level 1 outcome    : human YES == machine YES
Level 2 structural : + reference + direction + timeframe consistent
Level 3 semantic   : + rejection consistent (main target)
SEMANTIC FALSE AGREEMENT : human YES & machine YES with different reference/timeframe/rejection
TIMEFRAME MISMATCH       : grouped (H4 vs M30, M30 vs M15, ...)
REJECTION SEMANTICS      : human vs machine table per case

8. DECISION GATE

PASS              : machine semantics strongly matches human consensus (high L3)
PASS WITH CAVEAT  : mostly matches, systematic ambiguity present
FAIL              : machine definition materially differs from human semantics
INCONCLUSIVE      : low human agreement / sample not yet adequate

Forbidden: changing f7/threshold/InpSeqWindow/timeframe based on human labels; AUC/PF optimization; running ML. Comparison = validation, not optimization.

9. CURRENT STATUS

[x] machine v2 integrity PASS
[x] machine freeze (f7_v2)
[x] human package + machine package (blind) ready
[x] comparison.py extended (inter-rater + machine-vs-consensus + levels 1-3)
[ ] human_A_f7.csv      <- WAITING FOR ANNOTATOR A
[ ] human_B_f7.csv      <- WAITING FOR ANNOTATOR B
[ ] human_adjudicated_f7.csv
[ ] human_machine_f7_comparison.json
[ ] LIQUIDITY SWEEP SEMANTIC VERDICT

10. SUCCESS CRITERIA

machine v2 integrity PASS
AND human annotations completed
AND human agreement measured
AND adjudication completed
AND machine-vs-human comparison completed
AND semantic verdict established

If human agreement is low -> STOP; do not fix the machine based on subjective disagreement.

11. NEXT RESEARCH GATE

CHoCH may start validation ONLY after Liquidity Sweep receives its verdict (PASS / PASS WITH CAVEAT / FAIL / INCONCLUSIVE). OB/FVG/Displacement/Candidate Setup also wait for this verdict.

12. ARTIFACTS

docs/P3_S1_HUMAN_MACHINE_F7_VALIDATION.md
ml/p3/smc_semantic/integrity_audit.py
ml/p3/smc_semantic/gen_human_package.py
ml/p3/smc_semantic/comparison.py            (extended: inter-rater + machine-vs-consensus)
ml/p3/smc_semantic/output/integrity_audit_f7_v2.json
ml/p3/smc_semantic/human_package/           (cases.csv, template, protocol, context_m15/)
ml/p3/smc_semantic/machine_package/         (BLINDED)
ml/p3/smc_semantic/output/machine_annotations_f7_v2.csv   (FROZEN machine reference)
ml/p3/smc_semantic/output/machine_annotations_f7_v1_LEGACY.csv  (preserved)

6.6 KiB Raw Permalink Blame History