| # Policy & Exploration Guide — DualEA System
|
|
|
| **ML policy gating, fallback modes, and exploration system**
|
|
|
| ---
|
|
|
| ## Table of Contents
|
|
|
| 1. [Policy System Overview](#policy-system-overview)
|
| 2. [Policy Structure](#policy-structure)
|
| 3. [Policy Gating Logic](#policy-gating-logic)
|
| 4. [Policy Fallback Modes](#policy-fallback-modes)
|
| 5. [Policy Scaling](#policy-scaling)
|
| 6. [Exploration Mode](#exploration-mode)
|
| 7. [Exploration Caps](#exploration-caps)
|
| 8. [Troubleshooting](#troubleshooting)
|
|
|
| ---
|
|
|
| ## Policy System Overview
|
|
|
| ### Purpose
|
|
|
| The **policy system** allows ML models to influence trading decisions through:
|
| - **Confidence gating**: Block low-confidence predictions
|
| - **Parameter scaling**: Adjust SL/TP/lots based on model output
|
| - **Risk management**: Dynamic position sizing based on predicted outcomes
|
|
|
| ### Files
|
|
|
| **policy.json**: ML-generated trading policy
|
| - Path: `Common/Files/DualEA/policy.json`
|
| - Generated by: `ML/policy_export.py`
|
| - Structure: Per-slice (strategy|symbol|timeframe) probabilities and scaling multipliers
|
|
|
| **policy.backup.json**: Last-known-good policy
|
| - Used for rollback on parse/plausibility failures
|
| - Updated on successful policy load
|
|
|
| **policy.reload**: Trigger file for hot-reload
|
| - PaperEA_v2: Hot-reload supported via `policy.reload` + HTTP polling
|
| - LiveEA: Load on init only (restart required for updates)
|
|
|
| ---
|
|
|
| ## Policy Structure
|
|
|
| ### JSON Schema
|
|
|
| ```json
|
| {
|
| "version": "1.0",
|
| "generated_at": "2025-01-15T10:00:00Z",
|
| "model_hash": "abc123...",
|
| "train_window": {"start": "2024-01-01", "end": "2025-01-15"},
|
| "metrics": {
|
| "roc_auc": 0.72,
|
| "brier_score": 0.18,
|
| "expected_r": 0.65
|
| },
|
| "min_confidence": 0.55,
|
| "slices": [
|
| {
|
| "strategy": "ADXStrategy",
|
| "symbol": "EURUSD",
|
| "timeframe": 60,
|
| "probability": 0.75,
|
| "sl_mult": 1.0,
|
| "tp_mult": 1.2,
|
| "lot_mult": 1.0,
|
| "trail_mult": 1.0
|
| },
|
| ...
|
| ]
|
| }
|
| ```
|
|
|
| ### Fields
|
|
|
| **Global**:
|
| - `version`: Schema version
|
| - `generated_at`: Timestamp of policy generation
|
| - `model_hash`: Trained model identifier for provenance
|
| - `train_window`: Data range used for training
|
| - `metrics`: Model performance metrics
|
| - `min_confidence`: Global minimum confidence threshold
|
|
|
| **Per-Slice** (as implemented in LiveEA; PaperEA_v2 uses minimal parsing):
|
| - `strategy`: Strategy name (e.g., "ADXStrategy")
|
| - `symbol`: Trading symbol (e.g., "EURUSD")
|
| - `timeframe`: Timeframe in minutes (e.g., 60 for H1)
|
| - `p_win`: ML model confidence (0.0-1.0)
|
| - `sl_scale`: Stop loss scaling multiplier (optional)
|
| - `tp_scale`: Take profit scaling multiplier (optional)
|
| - `trail_atr_mult`: Trailing stop ATR multiplier (optional)
|
| - `confidence`: Alternative confidence field (optional)
|
|
|
| ---
|
|
|
| ## Policy Gating Logic
|
|
|
| ### Configuration
|
|
|
| ```cpp
|
| input bool UsePolicyGating = true; // Enable ML policy gating
|
| input bool DefaultPolicyFallback = true; // Enable fallback modes
|
| input bool FallbackDemoOnly = true; // Restrict fallback to demo
|
| input bool FallbackWhenNoPolicy = true; // Fallback if policy not loaded
|
| input bool FallbackWhenSliceMissing = true; // Fallback if slice missing
|
| ```
|
|
|
| ### Decision Flow
|
|
|
| ```
|
| Is UsePolicyGating=true?
|
| No → PASS (no policy gating)
|
| Yes → Continue
|
|
|
| Is policy loaded (slices > 0)?
|
| No → Check FallbackWhenNoPolicy
|
| Yes → Check FallbackDemoOnly
|
| Demo account → FALLBACK (neutral scaling)
|
| Live account → BLOCK
|
| No → BLOCK (reason: "no policy loaded")
|
| Yes → Continue
|
|
|
| Does exact slice exist (strategy|symbol|TF)?
|
| No → Check FallbackWhenSliceMissing
|
| Yes → Check FallbackDemoOnly
|
| Demo account → FALLBACK (neutral scaling)
|
| Live account → BLOCK
|
| No → BLOCK (reason: "policy slice missing")
|
| Yes → Continue
|
|
|
| Is slice.probability >= policy.min_confidence?
|
| No → BLOCK (reason: "confidence too low")
|
| Yes → PASS + Apply scaling
|
| ```
|
|
|
| ### Code Example
|
|
|
| ```cpp
|
| bool ApplyPolicyGating(SignalData &signal, string &reason) {
|
| if(!UsePolicyGating) return true;
|
|
|
| // Check if policy loaded
|
| if(policyData.slices == 0) {
|
| return HandleFallback("no_policy", signal, reason);
|
| }
|
|
|
| // Find policy slice
|
| PolicySlice slice = policyData.Find(
|
| signal.strategy_name,
|
| symbol,
|
| timeframe
|
| );
|
|
|
| if(slice.probability < 0) { // Slice not found
|
| return HandleFallback("slice_missing", signal, reason);
|
| }
|
|
|
| // Check confidence threshold
|
| if(slice.probability < policyData.min_confidence) {
|
| reason = StringFormat("confidence %.2f < %.2f",
|
| slice.probability, policyData.min_confidence);
|
| return false;
|
| }
|
|
|
| // Apply policy scaling
|
| ApplyPolicyScaling(signal, slice);
|
|
|
| return true;
|
| }
|
| ```
|
|
|
| ---
|
|
|
| ## Policy Fallback Modes
|
|
|
| ### Fallback: No Policy
|
|
|
| **Triggers when**:
|
| - `UsePolicyGating=true`
|
| - Policy file not loaded OR `policy.slices == 0`
|
| - `FallbackWhenNoPolicy=true`
|
|
|
| **Behavior**:
|
| - Check `FallbackDemoOnly`:
|
| - If `true` and demo account → ALLOW with neutral scaling
|
| - If `true` and live account → BLOCK
|
| - If `false` → ALLOW with neutral scaling (any account)
|
| - Bypasses insights thresholds
|
| - Bypasses exploration caps
|
| - No SL/TP/lot/trail multipliers applied
|
|
|
| **Log Example**:
|
| ```
|
| FALLBACK: no policy loaded -> neutral scaling used for ADXStrategy on EURUSD/H1 demo=true
|
| ```
|
|
|
| **Telemetry**:
|
| ```csv
|
| policy_fallback,EURUSD,60,ADXStrategy,no_policy,true,true
|
| ```
|
|
|
| ### Fallback: Slice Missing
|
|
|
| **Triggers when**:
|
| - `UsePolicyGating=true`
|
| - Policy IS loaded (`policy.slices > 0`)
|
| - Exact strategy|symbol|timeframe slice not found in policy
|
| - `FallbackWhenSliceMissing=true`
|
|
|
| **Behavior**:
|
| - Same as "No Policy" fallback
|
| - Allows trade with neutral scaling
|
| - Demo-only restriction if `FallbackDemoOnly=true`
|
|
|
| **Log Example**:
|
| ```
|
| FALLBACK: policy slice missing -> neutral scaling used for BollAverages on GBPUSD/H1 demo=true
|
| ```
|
|
|
| ### Neutral Scaling
|
|
|
| **Definition**: No adjustments applied, use base parameters.
|
|
|
| ```cpp
|
| sl_mult = 1.0
|
| tp_mult = 1.0
|
| lot_mult = 1.0
|
| trail_mult = 1.0
|
| ```
|
|
|
| Trade proceeds with:
|
| - Original SL/TP from strategy
|
| - Original lot size from risk calculation
|
| - Original trailing stop settings (if enabled)
|
|
|
| ### Fallback Safety
|
|
|
| **Why demo-only by default?**
|
| - Fallback is permissive (allows trades without ML validation)
|
| - Intended for data collection, not production live trading
|
| - In live, you want explicit policy coverage for all slices
|
|
|
| **When to disable FallbackDemoOnly?**
|
| - After verifying policy coverage is comprehensive
|
| - When transitioning from demo to live with same strategy set
|
| - With explicit risk acceptance of trading without ML guidance
|
|
|
| **Best Practice**:
|
| - Keep `FallbackDemoOnly=true` until policy proven
|
| - Monitor `policy_fallback` telemetry events
|
| - Aim to eliminate fallbacks by improving policy coverage
|
|
|
| ---
|
|
|
| ## Policy Scaling
|
|
|
| ### Policy Scaling (LiveEA Only)
|
|
|
| > **Note:** Full policy scaling with per-slice multipliers is implemented in LiveEA. PaperEA_v2 has minimal policy parsing (checks `min_confidence` only).
|
|
|
| **Scaling Application (LiveEA):**
|
| ```cpp
|
| void ApplyPolicyScaling(SignalData &signal, PolicySlice &slice) {
|
| // SL scaling
|
| double slDistance = MathAbs(signal.entry_price - signal.stop_loss);
|
| if(signal.direction == 1) { // Buy
|
| signal.stop_loss = signal.entry_price - (slDistance * slice.sl_scale);
|
| } else { // Sell
|
| signal.stop_loss = signal.entry_price + (slDistance * slice.sl_scale);
|
| }
|
|
|
| // TP scaling
|
| double tpDistance = MathAbs(signal.take_profit - signal.entry_price);
|
| if(signal.direction == 1) { // Buy
|
| signal.take_profit = signal.entry_price + (tpDistance * slice.tp_scale);
|
| } else { // Sell
|
| signal.take_profit = signal.entry_price - (tpDistance * slice.tp_scale);
|
| }
|
|
|
| // Trailing scaling (if enabled)
|
| if(TrailEnabled && slice.trail_atr_mult > 0) {
|
| // Apply trail ATR multiplier
|
| }
|
| }
|
| ```
|
|
|
| ### Scaling Examples
|
|
|
| **Conservative Scaling** (low confidence):
|
| ```json
|
| {
|
| "probability": 0.60,
|
| "sl_mult": 0.8, // Tighter stop
|
| "tp_mult": 1.5, // Wider target (better RR)
|
| "lot_mult": 0.5, // Smaller position
|
| "trail_mult": 0.9 // Tighter trail
|
| }
|
| ```
|
|
|
| **Aggressive Scaling** (high confidence):
|
| ```json
|
| {
|
| "probability": 0.85,
|
| "sl_mult": 1.2, // Wider stop (more breathing room)
|
| "tp_mult": 0.8, // Closer target (take profits faster)
|
| "lot_mult": 1.5, // Larger position
|
| "trail_mult": 1.0 // Normal trail
|
| }
|
| ```
|
|
|
| **Neutral Scaling** (fallback):
|
| ```json
|
| {
|
| "probability": 0.0,
|
| "sl_mult": 1.0,
|
| "tp_mult": 1.0,
|
| "lot_mult": 1.0,
|
| "trail_mult": 1.0
|
| }
|
| ```
|
|
|
| ---
|
|
|
| ## Exploration Mode
|
|
|
| ### Purpose
|
|
|
| **Bootstrap insights** for strategy|symbol|timeframe slices that lack sufficient historical data.
|
|
|
| Without exploration:
|
| - New slices blocked by insights gating (no data → can't pass thresholds)
|
| - Chicken-and-egg problem: can't trade → can't collect data → can't build insights
|
|
|
| With exploration:
|
| - Limited trades allowed for no-data slices
|
| - Caps prevent excessive exposure
|
| - Data collected → insights built → insights gating takes over
|
|
|
| ### When Exploration Triggers
|
|
|
| **Conditions**:
|
| 1. `UseExploration=true`
|
| 2. Insights gating is enabled (`UseInsightsGating=true`)
|
| 3. No slice exists in insights.json for this strategy|symbol|timeframe
|
| 4. Exploration caps not exceeded
|
|
|
| **Important**: Exploration bypass ONLY when slice truly missing. If slice exists but fails thresholds (low win rate), it is BLOCKED (no bypass).
|
|
|
| ### Configuration
|
|
|
| ```cpp
|
| input bool ExploreOnNoSlice = true; // Enable exploration when no slice exists
|
| input int ExploreMaxPerSlicePerDay = 100; // Daily cap per slice (default 100)
|
| input int ExploreMaxPerSlice = 100; // Weekly cap per slice (default 100)
|
| ```
|
|
|
| > **Note:** Previous documentation listed defaults of 2/3. Actual code defaults are 100/100, effectively unlimited for most practical purposes. The `UseExploration` input does not exist; exploration is controlled via `ExploreOnNoSlice`.
|
|
|
| ---
|
|
|
| ## Exploration Caps
|
|
|
| ### Cap Types
|
|
|
| **Daily Cap**: `ExploreMaxPerSlicePerDay`
|
| - Resets at midnight (00:00 server time)
|
| - Per-slice basis (each strategy|symbol|TF tracked separately)
|
| - Default: 100 trades/day/slice
|
|
|
| **Weekly Cap**: `ExploreMaxPerSlice`
|
| - Resets on Monday
|
| - Week bucket = Monday of the week (yyyymmdd format)
|
| - Default: 100 trades/week/slice
|
|
|
| ### Counter Persistence
|
|
|
| **Files**:
|
| - `Common/Files/DualEA/explore_counts_day.csv`
|
| - `Common/Files/DualEA/explore_counts.csv`
|
|
|
| **Format**:
|
| ```csv
|
| key,date_yyyymmdd,count
|
| ADXStrategy|EURUSD|60,20250115,2
|
| BollAverages|GBPUSD|60,20250115,1
|
| ```
|
|
|
| For weekly:
|
| ```csv
|
| key,week_monday_yyyymmdd,count
|
| ADXStrategy|EURUSD|60,20250113,3
|
| ```
|
|
|
| ### Cap Checking
|
|
|
| ```cpp
|
| bool CheckExplorationCaps(string strategy, string symbol, int tf, string &reason) {
|
| string sliceKey = strategy + "|" + symbol + "|" + IntegerToString(tf);
|
|
|
| // Load counters
|
| int dayCount = LoadExploreCountDay(sliceKey);
|
| int weekCount = LoadExploreCountWeek(sliceKey);
|
|
|
| // Check daily cap
|
| if(ExploreMaxPerSlicePerDay > 0 && dayCount >= ExploreMaxPerSlicePerDay) {
|
| reason = StringFormat("explore_cap_day (day=%d/%d, week=%d/%d)",
|
| dayCount, ExploreMaxPerSlicePerDay,
|
| weekCount, ExploreMaxPerSlice);
|
| telemetry.Event("explore_block_day", sliceKey, dayCount, weekCount);
|
| return false;
|
| }
|
|
|
| // Check weekly cap
|
| if(ExploreMaxPerSlice > 0 && weekCount >= ExploreMaxPerSlice) {
|
| reason = StringFormat("explore_cap_week (day=%d/%d, week=%d/%d)",
|
| dayCount, ExploreMaxPerSlicePerDay,
|
| weekCount, ExploreMaxPerSlice);
|
| telemetry.Event("explore_block_week", sliceKey, dayCount, weekCount);
|
| return false;
|
| }
|
|
|
| // Increment counters
|
| IncrementExploreCountDay(sliceKey);
|
| IncrementExploreCountWeek(sliceKey);
|
|
|
| // Allow exploration
|
| Log(StringFormat("GATE: explore allow %s on %s/%d (day=%d/%d, week=%d/%d)",
|
| strategy, symbol, tf,
|
| dayCount+1, ExploreMaxPerSlicePerDay,
|
| weekCount+1, ExploreMaxPerSlice));
|
|
|
| telemetry.Event("explore_allow", sliceKey, dayCount+1, weekCount+1);
|
|
|
| return true;
|
| }
|
| ```
|
|
|
| ### Resetting Caps
|
|
|
| **Manual Reset**:
|
| Delete counter files:
|
| ```powershell
|
| Remove-Item "C:\Users\<you>\AppData\Roaming\MetaQuotes\Terminal\Common\Files\DualEA\explore_counts*.csv"
|
| ```
|
|
|
| **Automatic Reset**:
|
| - Daily: Midnight (00:00 server time)
|
| - Weekly: Monday 00:00
|
|
|
| ### Interaction with NoConstraintsMode
|
|
|
| When `NoConstraintsMode=true`:
|
| - **Insights gating**: BYPASSED
|
| - **Exploration caps**: BYPASSED (trades allowed regardless of caps)
|
| - **Exploration counters**: Still incremented for telemetry
|
|
|
| ---
|
|
|
| ## Troubleshooting
|
|
|
| ### Issue: Policy fallback always triggering
|
|
|
| **Symptoms**:
|
| ```
|
| FALLBACK: no policy loaded -> neutral scaling for ADXStrategy on EURUSD/H1 demo=true
|
| ```
|
|
|
| **Diagnosis**:
|
| 1. Check if `policy.json` exists:
|
| ```powershell
|
| Test-Path "C:\Users\<you>\AppData\Roaming\MetaQuotes\Terminal\Common\Files\DualEA\policy.json"
|
| ```
|
|
|
| 2. Check policy load logs (enable `DebugPolicy=true`):
|
| ```
|
| Policy loaded: 0 slices, min_confidence=0.00
|
| ```
|
|
|
| 3. Verify policy.json content:
|
| ```powershell
|
| Get-Content "...\DualEA\policy.json" | ConvertFrom-Json | Select-Object -ExpandProperty slices | Measure-Object
|
| ```
|
|
|
| **Solutions**:
|
| - Run `ML/policy_export.py` to generate policy
|
| - Copy policy.json to Common Files
|
| - Verify JSON is well-formed (no parse errors)
|
|
|
| ### Issue: Policy slice missing fallback
|
|
|
| **Symptoms**:
|
| ```
|
| FALLBACK: policy slice missing -> neutral scaling for BollAverages on GBPUSD/H1 demo=true
|
| ```
|
|
|
| **Diagnosis**:
|
| 1. Check policy slices:
|
| ```powershell
|
| Get-Content "...\policy.json" | ConvertFrom-Json | Select-Object -ExpandProperty slices |
|
| Where-Object {$_.strategy -eq "BollAverages" -and $_.symbol -eq "GBPUSD" -and $_.timeframe -eq 60}
|
| ```
|
|
|
| 2. No result → slice missing from policy
|
|
|
| **Solutions**:
|
| - Collect more training data for this slice
|
| - Re-train model with broader coverage
|
| - Accept fallback for new slices (data collection mode)
|
|
|
| ### Issue: Exploration caps reached immediately
|
|
|
| **Symptoms**:
|
| ```
|
| GATE: blocked ADXStrategy on EURUSD/H1 reason=explore_cap_day (day=2/2, week=2/3)
|
| ```
|
|
|
| **Diagnosis**:
|
| 1. Check counters:
|
| ```powershell
|
| Import-Csv "...\explore_counts_day.csv" | Where-Object {$_.key -like "*ADXStrategy|EURUSD|60*"}
|
| ```
|
|
|
| 2. Verify caps not too restrictive:
|
| ```cpp
|
| ExploreMaxPerSlicePerDay = 2 // Very conservative
|
| ```
|
|
|
| **Solutions**:
|
| - Increase caps (e.g., 5/10 for PaperEA, 1/2 for LiveEA)
|
| - Reset counters manually (delete CSV files)
|
| - Use `NoConstraintsMode=true` for initial data collection (bypasses all caps)
|
|
|
| ### Issue: Exploration not triggering
|
|
|
| **Symptoms**:
|
| ```
|
| GATE: blocked ADXStrategy on EURUSD/H1 reason=below_winrate (WR=0.42 < 0.50)
|
| ```
|
| (Slice exists but fails threshold, no exploration bypass)
|
|
|
| **Diagnosis**:
|
| Exploration only triggers when **slice missing entirely**, not when slice exists but fails thresholds.
|
|
|
| **Solutions**:
|
| - This is correct behavior (no-slice-only bypass)
|
| - To allow trading despite low performance:
|
| - Lower insights thresholds temporarily
|
| - Use `NoConstraintsMode=true`
|
| - Delete slice from insights.json to force exploration
|
| - Wait for performance to improve
|
|
|
| ### Issue: NoConstraintsMode not working
|
|
|
| **Symptoms**:
|
| Still seeing gate blocks despite `NoConstraintsMode=true`
|
|
|
| **Diagnosis**:
|
| 1. Verify setting applied:
|
| ```cpp
|
| if(NoConstraintsMode) Print("NoConstraintsMode is TRUE");
|
| ```
|
|
|
| 2. Check if using unified system:
|
| ```cpp
|
| input bool UseUnifiedSystem = true; // Required for NoConstraintsMode
|
| ```
|
|
|
| **Solutions**:
|
| - Ensure `UseUnifiedSystem=true`
|
| - Restart EA after changing `NoConstraintsMode`
|
| - Check logs for shadow decisions: `[SHADOW] gate=... result=block (but allowing)`
|
|
|
| ---
|
|
|
| **See Also:**
|
| - [Configuration-Reference.md](Configuration-Reference.md) - Policy and exploration parameters
|
| - [Execution-Pipeline.md](Execution-Pipeline.md) - When policy/exploration evaluated
|
| - [Observability-Guide.md](Observability-Guide.md) - Policy/exploration telemetry events
|
