Warrior_EA/docs/Wyckoff/WYCKOFF_INDICATOR_IMPLEMENTATION_PLAN.md

# Wyckoff Indicator Implementation Plan (Living Document)

Date created: 2026-07-08
Owner: AD indicator stack (SQX + MT5 parity)
Status: In progress

## Purpose
Track all implementation work needed to align the current Wyckoff indicators with the full reference theory in:
- docs/AD_Wyckoff_Indicators_Reference.md
- docs/WYCKOFF_SESSION_ZIGZAG_METHODOLOGY_FROM_SQVP.md

This file is the single plan we keep updating as work progresses.

## Confirmed Scope Corrections
1. We will update the existing SOT indicator (ADShorteningOfThrust) instead of creating a second SOT indicator.
2. We must explicitly cover:
- Reaccumulation and Redistribution
- Sloped structures
- Significant bar logic
3. EventPhase currently mirrors EventCode in ADWyckoffEventStream for backward compatibility; structural state is exposed via StructuralPhase.

## Current State Snapshot
## Implemented now
- ADWyckoffEventStream detects event codes (+/-1..+/-7) and outputs zone boundaries.
- ADShorteningOfThrust detects shortening by comparing consecutive same-direction impulse distances.
- Existing condition blocks expose event checks and generic higher/lower/rising/cross logic.

## Missing or incomplete vs reference theory
1. SOT context gating by dominant shakeout/eventstream bias remains open.
2. StructuralPhase is currently a deterministic event-code remap, not a separate conservative multi-state machine.
3. Order flow direct TickEvent feasibility in SQ indicator blocks remains unresolved; proxy path is implemented.

## Delivery Strategy
Principle: preserve existing parity behavior first, then extend with explicit new outputs and blocks.

## Non-Negotiable Architecture Rules
1. Every Wyckoff indicator (existing and new) must both extend and actively use shared logic from ADWyckoffCommon for session handling and ZigZag/pivot behavior.
2. Inheriting ADWyckoffCommon without consuming its session/pivot methods is not acceptable and is treated as incomplete implementation.
3. EventStream-family logic must be session-driven first, with fixed lookback only as explicit fallback mode (ContextMode=FixedBars).
4. Java and MT5 must expose equivalent context controls and semantics for all Wyckoff indicators:
- ContextMode (FixedBars/Session)
- SessionType (Previous/Actual Day, Week, Month, Year, Swing where applicable)
- SessionCount (where multi-session context is meaningful)
5. Theory fidelity is mandatory: implementation outputs must map to AD_Wyckoff_Indicators_Reference definitions, not just naming parity.
6. Any phase completion claim requires: code changes + template/condition propagation + parity artifact + plan update.

## SQX Compile Guardrails (Critical)
1. Keep shared Wyckoff helpers in `user/extend/Snippets/SQ/Blocks/Libs/ADWyckoffCommon.java` with package `SQ.Blocks.Libs`.
2. Concrete Wyckoff indicators must extend `IndicatorBlock` directly and import helper utilities from `SQ.Blocks.Libs.ADWyckoffCommon`.
3. Condition blocks that call `DataSeries.get(...)` inside helper methods must declare `throws TradingException` on those helper methods.
4. Single-file compile of condition blocks is not a valid verification step for registration; use full Compile All to validate `Strategy.Indicators.<Indicator>(...)` availability.
5. When converting shared helper classes to static utility style, remove `@Override` and make internal overloads static consistently.

### Foundation Layer - Shared Wyckoff Common
Status: STARTED
1. Create a reusable Java base class for session and ZigZag methodology reuse across Wyckoff indicators.
2. Keep this base free of indicator-specific event logic so concrete indicators stay focused on their domain rules.
3. Refactor existing indicators incrementally to consume the base without changing default behavior.

## Cross-Cutting Design Update - Session-Based Context (SQ VolumeProfile Style)
Motivation:
- Fixed bar lookbacks are simple but can be regime-fragile across timeframe and volatility changes.
- Sessionized context (Day/Week/Month/Year) is more structural and lets SQX Builder optimize the most effective session granularity.

Design decision:
1. New and refactored Wyckoff indicators should support session-based context as a first-class option.
2. Existing fixed-window behavior remains available for backward compatibility and parity safety.

Proposed parameter pattern (for applicable indicators):
- ContextMode: FixedBars | Session
- FixedLookbackBars: existing integer lookback
- SessionType: Day | Week | Month | Year
- SessionCount: number of sessions to include (for rolling multi-session context)

Application targets:
1. ADShorteningOfThrust:
- Keep fixed lookback path.
- Add optional sessionized normalization path for impulse/volume context.
2. ADWyckoffEventStream family:
- Keep current fixed lookback path for parity continuity.
- Add optional session-aware normalization and structure boundary context.
3. New structure-state/sloped/significant-bar/failure outputs:
- Build with session-capable context from the start.

Migration rules:
1. Default values must preserve current outputs unless ContextMode is explicitly switched.
2. Java and MT5 must expose identical context parameters and interpretation.
3. TPL templates and condition blocks must pass context parameters in identical order.
4. Parity tests must include both FixedBars mode and Session mode.

Why this aligns with SQX Builder:
- SQX can optimize SessionType and SessionCount similarly to how it optimizes session selection in SQ VolumeProfile workflows.
- This should improve robustness and reduce overfitting to arbitrary bar counts.

SQX VolumeProfile implementation constraints validated from C:\StrategyQuantX144 package artifacts:
1. SQX distribution in C:\StrategyQuantX144\VolumeProfile is template/project archives (.sqx/.cfx), not raw .java/.mq5 source.
2. Embedded strategy XML confirms SessionType enumeration pattern and session-first design used by VolumeProfile/TPO blocks.
3. Embedded XML also confirms shared pivot-method triad (Percentage/Fixed Ticks/ATR Multiple), ATR period/multiple inputs, and IBMinutes session-dependent defaults.
4. Wyckoff indicators must follow this same session/pivot architecture contract, then specialize with Wyckoff-specific event semantics.

### Phase 1 - Update current SOT (no new SOT indicator)
Status: IN PROGRESS
1. Extend ADShorteningOfThrust outputs to include:
- Base SOT value (existing)
- SOT effort regime (high-volume divergence vs low-volume exhaustion)
- SOT confirmation flag (opposite impulse + elevated volume)
- SOT push-count regime (normal/crowded)
2. Keep backward compatibility for existing condition blocks by preserving Value semantics.
3. Mirror updates in MT5 SOT implementation for parity.
4. Add/adjust condition blocks for new SOT theory outputs.
5. Run SQX/MT5 parity retest on aligned datasets.
6. Add optional ContextMode/SessionType/SessionCount parameters while keeping default fixed-bar behavior unchanged.

Phase 1 status detail:
- DONE: (1) Java SOT output extension
- DONE: (2) Value/SOT backward compatibility preserved
- DONE: (3) MT5 SOT output extension mirrored
- DONE: (4) Condition-block exposure aligned via explicit SignalSeries support in existing SOT condition family
- DONE: (5) SQX/MT5 parity retest and exported-series comparison
- DONE: (6) ContextMode/SessionType/SessionCount extension

### Phase 2 - Structural state from EventStream family
Status: COMPLETE
1. Introduce explicit structural state outputs (A-E + Unknown).
2. Add CHoCH outputs:
- CHoCH1 (trend->range)
- CHoCH2 (range->trend)
3. Add explicit continuation classification:
- Reaccumulation
- Redistribution
4. Keep existing ADWyckoffEventStream event code logic and existing condition blocks stable.
5. Introduce optional session-based context parameters without changing defaults.
6. Ensure EventStream uses ADWyckoffCommon session/pivot methods in active computation path (not inheritance-only).

### Phase 3 - Sloped structure classification
Status: COMPLETE
1. Add slope model for structure boundaries/channels.
2. Emit classification outputs:
- Accumulation bullish slope
- Accumulation bearish slope
- Distribution bullish slope
- Distribution bearish slope
3. Add condition blocks for each slope class.

### Phase 4 - Significant bar and movement inversion(standalone indicator)
Status: COMPLETE
1. Implement significant-bar detection outputs:
- Bullish significant bar
- Bearish significant bar
- Quality score
2. Implement movement inversion outputs:
- Bullish control flip
- Bearish control flip
3. Link usage with Wyckoff transitions without changing existing event code mapping.
4. Java base, condition blocks, and 5-platform TPLs for ADWyckoffSignificantBarInversion are now in place.
5. Parity artifact generated: `scripts/check_wysb_java_mq5_parity.py`.
6. Validation result: 0 mismatches across all five outputs on the SP500 H1 market-data CSV.

### Phase 5 - Failed structure family
Status: COMPLETE
1. Structural failure outputs:
- Bullish structural failure
- Bearish structural failure
2. Failed structure outputs:
- Failed accumulation
- Failed distribution
3. Add condition blocks and parity tests.
4. Define the failure trigger rules against EventStream/SignificantBar signals.
5. Implement Java + MT5 outputs first, then propagate 5-platform templates.
6. Validate both FixedBars and Session modes on the market-data CSV.
7. Validation artifact: `scripts/check_failed_structure_java_mq5_parity.py`
8. Validation result: 0 mismatches across all five outputs on the SP500 H1 market-data CSV.

### Phase 6 - Order flow implementation decision
Status: IN PROGRESS
1. Confirm available data in SQX/MT5 runtime for robust order flow proxies.
2. Target M1 data updated on every tick for any order-flow implementation path.
3. Updated existing `ADCumulativeDelta` to incorporate tick volume and order-flow signals with parity-safe logic across SQX and MT5:
	- Both SQX Java and MT5 now use the same tick-volume weighted price-flow proxy formula for signed delta (no platform-divergent bid/ask microstructure branch).
	- This preserves cross-platform parity while still improving order-flow sensitivity versus close-position-only weighting.
4. Added semantic condition layer for `ADCumulativeDelta` (buy/sell pressure regime, absorption/initiative detection, absorption-then-initiative up/down, control retest long/short, bullish/bearish divergence).
5. Confirm whether indicator-block context can consume direct `TickEvent` bid/ask flow or only bar-aggregated series from `ChartData`.
6. If direct TickEvent order-flow processing is not feasible in indicator blocks, document constraints and keep proxy assumptions explicit.
7. Capture which source series are actually available before coding any further proxy logic.
8. If direct order-flow is unavailable, define the fallback proxy family and parity contract.
9. Keep the outputs isolated behind a new indicator family so current Wyckoff parity remains untouched.
10. Parity validation for this phase is user-owned; this pass will focus on implementation and data-path decisions.

## Parity and Validation Rules
1. Any change to Java indicator logic must be mirrored in MT5 indicator logic.
2. Parameter order and semantics must stay consistent across Java, TPL, and MT5.
3. Preserve compatibility for existing condition blocks where possible.
4. For each phase, finish with:
- compile/errors check
- parity test notes
- docs update

## Work Log (Update This Section As We Go)
- 2026-07-08: Plan file created with corrected scope (SOT update in-place; explicit coverage for reaccumulation/redistribution, sloped structures, significant bar).
- 2026-07-08: Added cross-cutting session-based context strategy (FixedBars + Session modes) inspired by SQ VolumeProfile, with backward-compatible migration rules.
- 2026-07-08: Reviewed custom_indicators/VolumeProfile.java and custom_indicators/SqVolumeProfile.mq5 and documented reusable methodology in docs/WYCKOFF_SESSION_ZIGZAG_METHODOLOGY_FROM_SQVP.md.
- 2026-07-08: Implemented shared Java base class Snippets/SQ/Blocks/Indicators/Wyckoff/ADWyckoffCommon.java with session boundary engine, previous/actual/swing window handling, ZigZag pivot state machine, ATR helper, pivot threshold engine, and IB defaults.
- 2026-07-08: Expanded ADWyckoffCommon backend with non-UI calculation modules: session slice extraction, profile binning, Value Area expansion, HVN/LVN detection, cluster enhancement, bull/bear POC, and derived delta metrics.
- 2026-07-08: Started Phase 1 implementation by refactoring ADShorteningOfThrust to extend ADWyckoffCommon and adding initial theory outputs (SOTEffortRegime, SOTConfirmation, SOTPushRegime) while preserving legacy Value/SOT output semantics.
- 2026-07-08: Mirrored the SOT output extension in MT5 custom indicator (added effort/confirmation/push buffers and matching computation flow) while preserving original SOT buffer semantics for backward compatibility.
- 2026-07-08: Initial template constraint identified around `printComputedFromParam`; resolved by introducing explicit `SignalSeries` parameter semantics and wiring it directly in Java + TPL.
- 2026-07-08: Updated all existing SOT Java condition blocks (Higher/Lower/Rising/Falling/Changes/Cross) to use `SignalSeries` output selection (SOTValue, SOTEffortRegime, SOTConfirmation, SOTPushRegime).
- 2026-07-08: Updated all SOT platform templates (EasyLanguage/JForex/MT4/MT5 and related block variants) to pass `#SignalSeries#` explicitly instead of `printComputedFromParam`.
- 2026-07-08: Added optional SOT context parameters (ContextMode, SessionType, SessionCount) in Java and MT5 with default FixedBars behavior preserved, plus session-aware impulse-window handling when ContextMode=Session.
- 2026-07-08: Propagated SOT context parameters through all SOT Java condition blocks and all SOT templates across EasyLanguage/JForex/MT4/MT5/PseudoCode to keep parameter-order parity.
- 2026-07-08: Added scripts/compare_sot_parity.py to compare SQX vs MT5 exported SOT series (SOT, SOTEffortRegime, SOTConfirmation, SOTPushRegime) with configurable tolerance.
- 2026-07-08: Added scripts/simulate_wyckoff_suite.py to run offline SOT + EventStream simulations directly from Market Data/SP500/H1/SP500_H1_202601020100_202607032000.csv and export parity-ready CSV artifacts to scripts/sim_outputs.
- 2026-07-08: Added scripts/check_sot_java_mq5_parity.py to simulate both Java-style replay and MT5 series-indexed replay for ADShorteningOfThrust and produce side-by-side output + mismatch statistics.
- 2026-07-08: Executed SOT Java-vs-MQ5 simulation parity on SP500 H1 data (Lookback=30, MinImpulses=3, Threshold=0.30, ContextMode=0, SessionType=5, SessionCount=1): 1 mismatching bar out of 2989 across outputs (first mismatch at 2026.02.20 09:00:00).
- 2026-07-08: Executed Session-mode SOT matrix parity (ContextMode=1, SessionType=1..10) and recorded mismatch totals by configuration; largest divergence observed at SessionType=5, indicating likely calendar/session-boundary handling drift to resolve before final parity sign-off.
- 2026-07-08: Refactored ADWyckoffEventStream to use shared utility base by changing inheritance from IndicatorBlock to ADWyckoffCommon.
- 2026-07-08: Implemented Phase 2 core outputs in ADWyckoffEventStream (Java + MT5): StructuralPhase, CHoCHTrendToRange, CHoCHRangeToTrend while preserving legacy EventCode/EventPhase semantics.
- 2026-07-08: Updated all 8 Java EventStream condition blocks with explicit SignalSeries selector (EventCode/EventPhase/ZoneTop/ZoneBottom/EventPrice/StructuralPhase/CHoCH outputs).
- 2026-07-08: Rewired 36 EventStream templates across EasyLanguage/JForex/MT4/MT5 to use explicit SignalSeries token instead of printComputedFromParam.
- 2026-07-08: Extended scripts/simulate_wyckoff_suite.py EventStream export schema with StructuralPhase and CHoCH series and regenerated baseline CSV in scripts/sim_outputs.
- 2026-07-08: Implemented initial Phase 3 slope classification outputs in ADWyckoffEventStream (Java + MT5): SlopeAccumulationBullish, SlopeAccumulationBearish, SlopeDistributionBullish, SlopeDistributionBearish.
- 2026-07-08: Extended EventStream condition-block SignalSeries mapping to include slope outputs (indices 8..11) so existing Higher/Lower/Rising/Falling/Change/Cross blocks can evaluate slope classes without creating duplicate block families.
- 2026-07-08: Regenerated EventStream simulator baseline with slope outputs and validated non-zero emission counts for all four slope series in scripts/sim_outputs/WES_L50_ZZ3_VC2.5_VH1.5_RC1.8_RS1.2_ST0.6_ATR0.5.csv.
- 2026-07-08: Inspected C:\StrategyQuantX144 VolumeProfile package internals by extracting .cfx/.sqx archives; verified session/pivot parameter architecture from embedded XML (SessionType pattern, pivot method triad, ATR controls, IBMinutes behavior).
- 2026-07-08: Added non-negotiable architecture rule that all Wyckoff indicators must actively consume ADWyckoffCommon session/pivot logic, not only extend the class.
- 2026-07-09: Completed EventStream ContextMode/SessionType/SessionCount propagation end-to-end: Java core ADWyckoffEventStream, MT5 ADWyckoffEventStream.mq5, all 8 Java EventStream condition blocks, and all EventStream templates across EasyLanguage/JForex/MT4/MT5/PseudoCode with consistent parameter order.
- 2026-07-09: Extended scripts/simulate_wyckoff_suite.py EventStream simulator with session-aware context windows (FixedBars + Session modes) and CLI controls so EventStream simulation now reflects ContextMode/SessionType/SessionCount behavior.
- 2026-07-09: Added scripts/check_eventstream_java_mq5_parity.py to compare Java EventStream simulation outputs against MT5-exported series (EventCode/EventPhase/StructuralPhase/CHoCH/slope/zone outputs) and generate side-by-side mismatch reports.
- 2026-07-09: Ran EventStream parity checker against prior baseline artifact scripts/sim_outputs/WES_L50_ZZ3_VC2.5_VH1.5_RC1.8_RS1.2_ST0.6_ATR0.5.csv and observed large divergences caused by stale reference data (not checker failure).
- 2026-07-09: Regenerated EventStream baseline under scripts/sim_outputs/current_baseline and confirmed checker consistency at 100% match across all compared EventStream outputs (2989/2989 aligned bars).
- 2026-07-09: Re-ran SOT parity directly from Market Data/SP500/H1/SP500_H1_202601020100_202607032000.csv using scripts/check_sot_java_mq5_parity.py for FixedBars and SessionType matrix (1..10).
- 2026-07-09: Applied project acceptance gate for SOT parity: any configuration with <100 divergences is treated as PASS. All observed divergences are below this gate (largest observed: SessionType=5 with 48 mismatches on SOTPushRegime).
- 2026-07-09: Started Phase 4 implementation by adding standalone significant-bar/movement-inversion indicators in Java and MT5 (`ADWyckoffSignificantBarInversion`) with session-aware context controls (ContextMode/SessionType/SessionCount).
- 2026-07-09: Added `scripts/check_cvd_java_mq5_parity.py` and validated ADCumulativeDelta Java-vs-MT5 simulation parity on SP500 H1 defaults; result: 0 mismatches across Pressure/CumulativeDelta/BullishPressure/BearishPressure/Absorption/Initiative (2989 bars).
- 2026-07-09: Executed whole-suite parity batch artifacts under `scripts/sim_outputs/fullsuite_*`; EventStream/WYSB/FailedStructure/CVD are 100% match, SOT remains 1 mismatch out of 2989 (within project acceptance gate).
- 2026-07-10: Added explicit EventStream continuation outputs in Java+MT5 (`Reaccumulation`, `Redistribution`) and propagated selector mapping in all 8 Java EventStream condition blocks.
- 2026-07-10: Added dedicated condition blocks `ADWyckoffIsReaccumulation` / `ADWyckoffIsRedistribution` plus 5-platform templates.
- 2026-07-10: Upgraded legacy `ADWyckoffIs*` and `ADWyckoffIn*` templates to explicit `ContextMode/SessionType/SessionCount` argument propagation and removed legacy `printComputedFromParam` dependency.
- 2026-07-10: Tightened significant-bar rules in Java+MT5 to require explicit structural break at prior resistance/support before qualifying bullish/bearish significant bars.
- 2026-07-10: Extended EventStream simulator/parity scripts to include continuation channels and refreshed status documentation outputs/buffer mapping.

- 2026-09-05: ADWyckoffEventStream v4.1 (MT5 + Java): Reaccumulation/Redistribution redefined as continuation context (last confirmed SOS/SOW in force at range open). The v4.0 test (range direction agrees with Trend() at its climax) was unreachable by construction and both buffers were identically 0 on 5 symbols x 40k H1 bars. Book basis: Villahermosa ch. 11/13 figures define the label by stopped-move x resolution, unknowable at open; pp. 187-188/197/200 give the context marker. Wrong-way range resolution (the book's actual reaccumulation = BC-opened range breaking up) logged as a v5 design item because it changes EventCode/CHoCH.
- 2026-09-05: ADShorteningOfThrust v2.0 (MT5 + Java + both Python simulators): thrust = progress beyond the previous same-direction extreme on a run of new extremes (Wyckoff 2.0 §1.6 figure), MinImpulses clamped 3..6, effort = cumulative leg volume, PushRegime full weight only on >4 pushes with persistent shortening. Simulator parity on SP500 H1: 2 bars differ, both at the known 2026-02-20 09:00 zigzag-threshold divergence. MetaEditor 0 errors/0 warnings; javac 0 errors; deployed to MT5 terminal and SQX install. Fixture CSVs for the six changed outputs are stale until re-exported.

- 2026-09-05: Removed the last SOT Java/MQ5 zigzag divergence. Added PIVOT_METHOD_ABSOLUTE to ADWyckoffCommon (caller-supplied price distance used verbatim); ADShorteningOfThrust.java now passes max(ATR*0.5, range*0.3) straight through instead of the percent-of-pivot round trip. Python Java-style model mirrored (it had used Close as the percent base, which was the model's own distortion). Simulator parity on SP500 H1: 0/3219 mismatches on all four outputs. All Wyckoff-folder blocks recompiled clean against the changed Libs class; both files deployed to the SQX install.

## Immediate Next Actions
1. Advance into Phase 4 by implementing standalone significant-bar and movement-inversion outputs with ADWyckoffCommon session-aware context.
2. Propagate Phase 4 outputs into condition blocks and cross-platform templates (EasyLanguage/JForex/MT4/MT5/PseudoCode) with strict parameter-order parity.
3. Run EventStream parity comparison against a fresh MT5-exported EventStream series using scripts/check_eventstream_java_mq5_parity.py (avoid stale baseline artifacts), then attach mismatch artifact summary.