chiki2bum2/SniperGold_ML
Watch
1
0
Fork
You've already forked SniperGold_ML
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
SniperGold_ML/docs/DESIGN.md

14 KiB
Raw Permalink Blame History

Algo Forge — DESIGN

Source of inspiration: SniperGold SMC Pro+ (c) Waseem Shahrukh — https://www.mql5.com/en/code/75466

Per-engine technical specification document. Project structure & status: see PROGRESS.md and README.md.

Engine 1 — MTF Bar Data Collector + Cache

File: MQL5\Include\AlgoForge\AF_Engine1_MTFData.mqh — class AFEngine1MTF, struct AFBar.

1. Responsibilities

  • Collect OHLCV bars from several explicit timeframes, as required by the N/C/E/P agents (Engine 2) and the display (Engine 3).
  • Provide an index-based internal cache so data can be reused repeatedly without repeated History reads (anti-freeze). This pattern is proven in SniperGold v4.5 (RefreshMTFCache / BuildMTFStruct).
  • Chart-TF independent: all access uses an explicit symbol + explicit ENUM_TIMEFRAMES. PERIOD_CURRENT is rejected at Register().
  • Closed-bar lock: the cache contains ONLY closed bars (non-repainting).

2. API

Lifecycle

AFEngine1MTF e1;
int s1 = e1.Register(PERIOD_M15, 600);          // slot index or AF_E1_ERR_SLOT
bool changed = e1.Refresh();                     // call each OnCalculate/OnTick

Read (idxFromRight: 0 = NEWEST closed bar)

bool     e1.IsReady(slot);
int      e1.Count(slot);                         // number of closed bars in the cache
bool     e1.GetBar(slot, idx, AFBar &out);
bool     e1.GetBarByTime(slot, datetime, AFBar &out);
int      e1.FindBarIndex(slot, datetime);
double   e1.Open/High/Low/Close(slot, idx);
datetime e1.Time(slot, idx);
long     e1.TickVolume(slot, idx);
double   e1.ATR(slot, period=14);

Diagnostics

int  e1.HistoryCalls(slot);   // number of CopyRates executed (unit test anti-freeze)
int  e1.RefreshCount(slot);
int  e1.TotalHistoryCalls();
long e1.LastBars(slot);

3. Cache Mechanism

  1. Refresh() checks Bars(symbol, tf) for each slot.
  2. Only if Bars() changed (or the cache is not ready + throttle AF_E1_RETRY_SEC) → Build(slot) is called → CopyRates executes.
  3. Build: CopyRates(symbol, tf, 0, maxBars+1, rates) with as-series array; the forming bar (not yet closed) is dropped based on IsBarClosed(): barTime + PeriodSeconds(tf) <= TimeCurrent().
  4. The cache is stored in series: bars[0] = newest closed bar. Capacity maxBars.

Invariants (verified by unit tests)

  • Count(slot) <= Bars(symbol, tf).
  • All cache bars are closed (T4).
  • HistoryCalls increases by exactly 1 per new bar per TF (T5 anti-freeze).
  • Strictly decreasing time order from index 0 (T3); valid OHLC (T2).

4. Limits & Result Codes

  • AF_E1_MAX_SLOTS = 8, AF_E1_MAX_BARS = 5000, AF_E1_RETRY_SEC = 5.
  • AF_E1_OK / AF_E1_ERR_SLOT / AF_E1_ERR_NOTREADY / AF_E1_ERR_RANGE.

5. MQL5 Implementation Notes

  • No array-element references in MQL5 (T &x = arr[i] = error). All access uses direct index (m_slots[i].field).
  • Slots use a dynamic m_slots[] array (a struct containing dynamic arrays is safe).
  • #property version must be X.YY format (e.g. "1.00") to avoid warnings.

6. Unit Test

  • Harness: MQL5\Experts\AlgoForge_Engine1_UnitTest.mq5 (EA, log prefix AFTEST).
  • Run in the Strategy Tester (XAUUSD, model every tick; the "1-min OHLC" model rejects sub-chart TF requests).
  • Result 2026-08-21: PASS=115070 FAIL=0 (20 days) · PASS=8870 FAIL=0 (verification).

Engine 2 — 4 Independent Signal Agents (N/C/E/P) + Fuzzy + Aggregator

Files:

  • MQL5\Include\AlgoForge\AF_Engine2_Agents.mqh — 4 agents + fuzzy logic (classes AFAgentNarrative, AFAgentContext, AFAgentEntry, AFAgentPriceAction).
  • MQL5\Include\AlgoForge\AF_Engine2_Aggregator.mqh — separate aggregator (class AFAggregator) + facade AFEngine2Signals.

7. Key Principles

  1. 4 INDEPENDENT agents — each agent reads Engine 1 only through one slot (timeframe) of its own. No inter-agent calls/state; all methods are stateless (pure functions of Engine-1 closed bars).
  2. No mutual knowledge — the final composition is done by the separate aggregator, not between agents.
  3. Dynamic-weight fuzzy logic — each agent uses membership functions (AF_MF_Tri / AF_MF_Trap) + a light Mamdani evaluator (AFFuzzyEval: buyAcc/sellAcc/wTot, rules Rule(buySide, fire, weight)); weights adapt to market conditions measured from the agent's own data (trend strength, volatility, ranging).
  4. Non-repainting — all inputs are Engine-1 closed bars (closed-bar lock guaranteed by Engine 1; Engine 2 never reads History directly).

8. Input Timeframe

Agent TFs are HARDCODED (no manual InpHtfS1..S4 inputs — removed) — see macros AF_E2_TF_S1..S4 in AF_Defines.mqh:

AF_E2_TF_S1 = H4   (S1 = Narrative  / N)
AF_E2_TF_S2 = M30  (S2 = Context    / C)
AF_E2_TF_S3 = M15  (S3 = Entry      / E)
AF_E2_TF_S4 = M3   (S4 = PriceAction/P)

Consumers (indicator AF_Engine3_Display, Engine-2/3 unit tests) use these macros directly at Register(). Analysis basis: H4→M30→M15→M3 (top-down to the chart).

9. Agents

Agent Question Assessed elements (from its own slot) Dynamic weights
N (Narrative) "Which way is the market?" HH/HL/LL/LH pivots → trend (+clarity), CHoCH/MSS, BOS, liquidity sweep, premium/discount Clear trend → structure dominates; flat → zones/liquidity up
C (Context) "Which zone is price in?" OB (opposite bar before a strong move), FVG/imbalance, S/R (pivots), premium/discount High volatility → S/R & premium/discount down, OB/FVG up
E (Entry) "Is there entry confirmation?" Sweep, CHoCH, displacement, OB/FVG zones; rule ZONE + CONFIRMATION = setup Strong displacement → confirmation weight up
P (Price Action) "When to open?" Engulfing, pin bar, inside bar, 2-bar momentum, close position in range Ranging → reversal patterns up; trending → continuation up

Per-agent output: AFSignalOut { buy, sell, bias, confidence, dir, reason }.

10. Aggregator (separate)

AFAggregator::Compute(e1, slotE, n, c, e, p, out):

  1. Pass 1: initial bias = Σ (base weight × confidence × bias) / Σ (base weight × confidence). Base weights: N=0.30, C=0.30, E=0.25, P=0.15 (AF_AGG_W_*).
  2. Pass 2 (dynamic weights): agents aligned with the majority get a 1.5× boost; aggregate buy/sell = Σ (effective weight × buy/sell) / Σ weights.
  3. Final signal: dir = BUY/SELL/WAIT with threshold AF_AGG_BUY_TH=0.20 and minimum support AF_AGG_MIN_SUP=0.50.
  4. Levels: entry = close of the closed slot-E bar; sl/tp based on Engine-1 ATR (AF_AGG_SL_ATR=1.5, AF_AGG_TP_ATR=2.5) — only on BUY/SELL signals.
  5. confidence aggregate = support × (0.7 + 0.1 × number of aligned agents).

Facade AFEngine2Signals::Compute(e1, sN, sC, sE, sP, oN, oC, oE, oP, agg): one call runs the 4 agents + aggregator (used by Engine 3 / consumers).

11. Unit Test

  • Harness: MQL5\Experts\AlgoForge_Engine2_UnitTest.mq5 (log prefix AFTEST2).
  • Strategy Tester (XAUUSD, every tick):
    • T1 output validity per agent · T2 independence (changing agent X's input slot → other agents unchanged) + determinism · T3 closed-bar lock · T4 non-repaint (identical output within the same bar) · T5 aggregator validity · T6 synthetic aggregator.
  • Results 2026-08-21: PASS=21534 FAIL=0 (20 days) · PASS=9942 FAIL=0 (10-day verbose).
  • Note: agents use tamper slots (different TFs) for the independence test; T2 auto-retries when a tamper slot is not ready.

12. Implementation Constraints (additional, Phase-2 sessions)

  • replace_text_in_file multi-line edits often fail → use single-line edits.
  • The independence test needs a tamper slot with ≥ AF_E2_MIN_BARS (80) bars; choose a dense TF (M6/M12/M20/M30) so it is ready even on short runs.

Engine 3 — Display (reads Engine 1 & 2 output only)

Files:

  • MQL5\Include\AlgoForge\AF_Engine2_Display.mqh — display-context builder (Engine-2 layer): struct AFDisplayData, AFDispZone, AFDispLine, AFDispPivot + AF_BuildDisplayData(). All display computation (structure, swing points, OB, FVG, premium/discount, MTF levels, agent bias) happens HERE, using the same pure analysis helpers as the N/C/E/P agents (AF_BuildSwing, AF_DetectSweep, AF_RangeStat, etc.).
  • MQL5\Include\AlgoForge\AF_Engine3_Render.mqh — pure renderer (Engine-3 layer): struct AFRenderCfg + AFR_DrawAll() / AFR_Clear(). NO analysis computation; draws only from AFDisplayData + AFSignalOut + AFAggOut (structure, zones, signals, dashboard like the original 75466 code).
  • MQL5\Indicators\AlgoForge\AF_Engine3_Display.mq5 — indicator (chart window): inputs + OnInit/OnCalculate/OnDeinit; 0 buffers (object-only).
  • MQL5\Experts\AlgoForge_Engine3_UnitTest.mq5 — unit test (Strategy Tester).

13. Engine-3 Principles

  1. READS ONLY Engine-1 (AFEngine1MTF) & Engine-2 output (AFEngine2Signals / AFAggOut / AFDisplayData). Engine 3 computes no structure/zones itself.
  2. Signal–display consistency: AF_BuildDisplayData (Engine 2) uses the same helpers as the agents.
  3. Non-repainting: the indicator draws only when a new closed bar appears on the display TF (e1.Time(sDisp,0) changes); no redraw within the same bar.
  4. Anti-freeze: Engine 1 remains the only History reader (index cache).

14. Display elements (equivalent to original code 75466)

Element Source Description
BOS/CHoCH structure lines AFDispLine (AF_BuildStructLines) fractal pivots, BOS/CHoCH labels
HH/HL/LH/LL swing points AFDispPivot (AF_ClassifyPivots) bull/bear colors
Order Block AFDispZone (AF_CollectOBs) MIT filter + dedupe + size ≥ 0.15×ATR
FVG AFDispZone (AF_CollectFVG) MIT filter + size ≥ 0.02×ATR
Premium/Discount box from swHigh/swLow + position premium/equilibrium/discount bands
MTF PDH/PDL levels Engine-1 D1 slot (index 1) solid lines
MTF PWH/PWL levels Engine-1 W1 slot (index 1) dashed lines
Entry/SL/TP signals + arrows AFAggOut only when dir BUY/SELL
Dashboard AFDisplayData + AFAggOut bias, structure, liquidity, context, levels, N/C/E/P alignment, trade setup, legend

15. Indicator inputs

  • InpHtfS1..S4 = agent N/C/E/P TFs (Engine 2) — default H1/H1/M15/M15.
  • InpDispTF = structure/zone display TF — default PERIOD_CURRENT (= chart).
  • Toggles: structure, swing points, OB (+count), FVG (+count), premium/discount, MTF levels, signals, dashboard.
  • LuxAlgo-style colors (same defaults as original code 75466) + transparent panel.

16. Unit Test

  • Harness: MQL5\Experts\AlgoForge_Engine3_UnitTest.mq5 (log prefix AFTEST3).
  • Strategy Tester (XAUUSD, every tick, Visual=1 — chart objects are only created in visual mode).
  • T1 display-data validity (structure/eqPos/levels/bias) · T2 non-repaint (identical data within the same bar) · T3 closed-bar lock · T4 render creates chart objects (prefix AF3_) · T5 render determinism (same object count).
  • Results 2026-08-21: see PROGRESS.md.

17. Implementation Constraints (Phase 3)

  • ObjectCreate in the Strategy Tester only works in visual mode; the test EA probes once at startup and T4/T5 are auto-skipped (not failed) when objects are unsupported.
  • Visual mode slows the tester (20 days every tick ≈ 10 minutes) → for quick verification run a short range (e.g. 3–5 days).

Backtest Baseline (Phase 5) — verification & publication

File: MQL5\Experts\AlgoForge_Backtest_Baseline.mq5

  • config MQL5\Profiles\Tester\AlgoForge_Backtest_Baseline.XAUUSD.M15.*.ini
  • evaluation ml\backtest_eval.py.

18. Objective & Principles

  • Honest backtest of the baseline strategy (MLP freeze SniperGold_ML.mqh, AUC long 0.627 / short 0.621) net of spread.
  • NOT a fragile iCustom: 19 SMC features computed internally in the EA (identical to SniperGold_SMC_ProPlus_v4_4.mq5 ComputeMLFeatures + its dependencies), data source ONLY Engine 1 (AFEngine1MTF) — closed-bar lock, anti-freeze, consistent with the Algo Forge architecture.
  • Feature parameters hardcoded (AF_BT_* = v4.4 training defaults): SwingLen=50, InternalLen=5, Lookback=600, GrabWindow=8, EQ thr=0.10/3 bars, DeltaBars=10, HTF=D1/H4/H1, ConfluenceFilter=true.

19. Modes

Mode Function Output
0 CSV long/short prob per closed M15 bar AlgoForge_bt_prob_*.csv → backtest_eval.py (AUC/precision/calibration)
1 Net-of-spread trading in the Strategy Tester OrderSend market (ATR SL/TP, max hold); actual tester spread; OnTester summary

Inputs: InpMode, InpThreshLong/Short (default 0.60/0.60), InpSL_ATR=1.0, InpTP_ATR=1.5, InpMaxHoldBars=24 (= baseline label horizon), InpLot=0.01, InpMaxBars=700.

20. Results (2026-08-21, XAUUSD M15, every tick) — HONEST

Mode 0 (CSV/AUC), 2026.01.01–08.20, 14.850 bars:

Metric Value Baseline freeze
AUC LONG 0.5305 0.6270
AUC SHORT 0.5487 0.6207
Precision LONG @0.60 0.5352 (n=4454) —
Precision SHORT @0.60 0.5292 (n=7010) —
  • Poor calibration (prob 0.65+ → long frequency 0.545) → on the XAUUSD feed (not the XAUUSDc training feed) the baseline model is NOT calibrated. Do not claim an edge on another feed without a like-for-like gate.

Mode 1 (trade), 2026.05.01–08.20: trades=308, net=+1125.32, maxDD=1188.18, PF=1.32. Note: some signals failed to execute (requote 10018, no retry); the positive result is not significant (runtime AUC 0.53/0.55, small sample) — not an edge claim.

21. Constraints (Phase 5)

  • Tester CSV is written to the agent sandbox (Tester\Agent-*\MQL5\Files\) and reset per run → read/copy results immediately after the run (Python evaluation directly from the sandbox path).
  • MQL5 FILE_CSV delimiter in this terminal = TAB (not ;/,) → backtest_eval.py uses delimiter="\t".
  • PositionSelect overload ambiguity on build 6093 (string vs ulong) → helper BTSelectTicket() (loop PositionsTotal/PositionGetTicket).
  • OrderSend requotes (ret=10018) appear in the tester → recorded, no retry (honest).