SniperGold_ML/ml/p3/smc_semantic/gen_human_package.py

# -*- coding: utf-8 -*-
"""GEN HUMAN PACKAGE — buat paket annotation blind (P3-S.1 §6-7).

HUMAN PACKAGE (human_package/):
  cases.csv                        : case_id, symbol, decision_timestamp, decision_tf, note
  human_annotation_template_f7.csv : template blank (60 case, format §7)
  HUMAN_ANNOTATION_PROTOCOL.md     : aturan blind + chart rule + prosedur A/B + adjudikasi
  context_m15/<case_id>.csv        : (opsional) 200 bar M15 terakhir s.d. decision_timestamp
                                     utk review offline; snapshot BERAKHIR di decision_timestamp

MACHINE PACKAGE (machine_package/ — BLINDED, jangan dibuka annotator):
  machine_annotations_f7_v2.csv    : salinan annotation machine
  machine_reasons_f7_v2.txt        : alasan readable per case
  README_BLINDED.md                : peringatan blinding

Tidak ada keputusan machine di HUMAN PACKAGE. Sampling TIDAK dijalankan ulang.
Usage: python gen_human_package.py
"""
import os
import sys
import csv
import json
import shutil
import datetime as dt

import numpy as np

HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
import smc_semantic_common as SC

OUT = os.path.join(HERE, "output")
HUMAN = os.path.join(HERE, "human_package")
MACH = os.path.join(HERE, "machine_package")
CTX = os.path.join(HUMAN, "context_m15")

CTX_BARS = 200   # bar M15 sebelum & termasuk decision_timestamp (snapshot TIDAK melewati)


def iso(ts):
    return dt.datetime.fromtimestamp(int(ts), tz=dt.timezone.utc).strftime("%Y-%m-%d %H:%M:%S")


def main():
    meta = json.load(open(os.path.join(OUT, "cases_meta.json"), encoding="utf-8"))
    cases = meta["cases"]

    for p in (HUMAN, MACH, CTX):
        os.makedirs(p, exist_ok=True)

    # ---------- HUMAN PACKAGE ----------
    # cases.csv
    with open(os.path.join(HUMAN, "cases.csv"), "w", newline="", encoding="utf-8") as f:
        w = csv.writer(f)
        w.writerow(["case_id", "symbol", "decision_timestamp", "decision_tf", "note"])
        for c in cases:
            w.writerow([c["case_id"], SC.SYMBOL, c["decision_timestamp"], SC.DECISION_TF,
                        "REVIEW HANYA SAMPAI decision_timestamp. Jangan lihat future."])

    # template blank
    fields = ["case_id", "liquidity_sweep", "reference", "direction", "timeframe",
              "reference_price", "sweep_price", "close_back_rejection", "reason",
              "confidence", "notes"]
    with open(os.path.join(HUMAN, "human_annotation_template_f7.csv"), "w",
              newline="", encoding="utf-8") as f:
        w = csv.writer(f)
        w.writerow(fields)
        for c in cases:
            w.writerow([c["case_id"], "", "", "", "", "", "", "", "", "", ""])

    # context pack M15 (snapshot berakhir di decision_timestamp)
    t, o, h, l, c, v, htf = SC.load_data()
    for case in cases:
        r = int(case["bar_idx"])
        s = max(0, r - CTX_BARS + 1)
        with open(os.path.join(CTX, case["case_id"] + ".csv"), "w",
                  newline="", encoding="utf-8") as f:
            w = csv.writer(f)
            w.writerow(["time", "open", "high", "low", "close", "tick_volume"])
            for i in range(s, r + 1):
                w.writerow([iso(t[i]), round(o[i], 2), round(h[i], 2),
                            round(l[i], 2), round(c[i], 2), int(v[i])])

    # ---------- MACHINE PACKAGE (BLINDED) ----------
    shutil.copy(os.path.join(OUT, "machine_annotations_f7_v2.csv"),
                os.path.join(MACH, "machine_annotations_f7_v2.csv"))
    rows = list(csv.DictReader(open(os.path.join(OUT, "machine_annotations_f7_v2.csv"),
                                    encoding="utf-8-sig")))
    with open(os.path.join(MACH, "machine_reasons_f7_v2.txt"), "w", encoding="utf-8") as f:
        for r in rows:
            f.write(f"{r['case_id']} | {r['decision_timestamp']} | "
                    f"{r['machine_decision']} | {r['primary_primitive']} | "
                    f"age={r['f7_state_age']} | {r['machine_reason']}\n")
    with open(os.path.join(MACH, "README_BLINDED.md"), "w", encoding="utf-8") as f:
        f.write("# MACHINE PACKAGE — BLINDED\n\n"
                "Folder ini berisi hasil machine annotation (f7_v2, source_commit b519a34).\n"
                "**DILARANG dibuka oleh annotator manusia sebelum human annotation selesai.**\n"
                "Hanya tim riset/adjudicator pasca-konsensus yang boleh membaca folder ini.\n")

    # protocol doc
    protocol = f"""# HUMAN ANNOTATION PROTOCOL — LIQUIDITY SWEEP (P3-S.1)

## Prinsip Blind
- Annotator HANYA menerima folder `human_package/`.
- Folder `machine_package/` berisi keputusan machine — **DILARANG dibuka sebelum annotasi selesai**.
- Human A (primary) dan Human B (independent reviewer) mengerjakan secara independen.

## Chart Data Rule (WAJIB)
- Buka chart XAUUSD (M15 utk keputusan; H4/M30/M3 sbg konteks) di terminal MT5.
- Scroll HINGGA `decision_timestamp` pada kolom `cases.csv`.
- **TIDAK boleh melihat candle setelah decision_timestamp** (future).
- Snapshot/context `context_m15/<case_id>.csv` berakhir TEPAT di decision_timestamp.

## Kolom Template (human_annotation_template_f7.csv)
| Kolom | Nilai |
|---|---|
| liquidity_sweep | YES / NO / AMBIGUOUS |
| reference | EQH / EQL / Swing High / Swing Low / Other / None |
| direction | Bullish / Bearish / None |
| timeframe | H4 / M30 / M15 / M3 / None |
| reference_price | (numeric opsional) |
| sweep_price | (numeric opsional) |
| close_back_rejection | YES / NO / AMBIGUOUS / N/A |
| reason | teks bebas — fakta yang teramati, bukan opini |
| confidence | HIGH / MEDIUM / LOW |

## Definisi kerja utk annotator (bukan definisi final — utk konsistensi labeling)
- Liquidity Sweep = harga menembus (wick) level likuiditas (equal high/low / swing) lalu
  menunjukkan rejection (close kembali) — nilai apakah ini TERJADI pada window sekitar
  decision_timestamp.
- Reference = jenis level likuiditas yang disapu.
- Direction = arah intent setelah sweep (Bullish = sell-side swept; Bearish = buy-side swept).

## Prosedur
1. Salin `human_annotation_template_f7.csv` -> `human_A_f7.csv` (Human A).
2. Human B mengerjakan salinan -> `human_B_f7.csv` (tanpa melihat A).
3. Simpan hasil di `ml/p3/smc_semantic/output/`.
4. Setelah A & B selesai: hitung inter-rater (human_interrater_f7_report.json).
5. Kasus berbeda -> adjudikasi -> `human_adjudicated_f7.csv` (adjudicator melihat chart,
   protokol, metadata — TIDAK melihat machine result).
6. Baru setelah consensus: machine vs human (human_machine_f7_comparison.json).

## Larangan
- Jangan mengubah f7/threshold/InpSeqWindow berdasarkan hasil annotasi (validation, bukan optimasi).
- Jangan menampilkan machine result ke annotator sebelum selesai.

## Case Set
- 60 kasus (SGML_SWEEP_001..060), XAUUSD M15, 2018-2026, seed 42 (case set hash
  {__import__('hashlib').sha256(open(os.path.join(OUT, 'cases_meta.json'),'rb').read()).hexdigest()[:16]}).
"""
    with open(os.path.join(HUMAN, "HUMAN_ANNOTATION_PROTOCOL.md"), "w", encoding="utf-8") as f:
        f.write(protocol)

    print("[saved] human_package/ (cases.csv, template, protocol, context_m15/)")
    print("[saved] machine_package/ (BLINDED: annotations + reasons)")


if __name__ == "__main__":
    main()