forked from animatedread/Warrior_EA
77e8080cfe
1. The expectancy stop was stone dead at shipped defaults. Its only feed -
RecordTradeResult inside CTradeJournalManager::Update() - ran solely under
UseDatabaseRanking, which ships false, so the da54639 halt was armed
(ExpectancyMinTrades=40) and never received a single closed trade. A risk
rule must not be a side effect of an analytics toggle: the journal gains
InitTrackingOnly(), Update() runs unconditionally from OnTick and skips
only the DB insert when no DB was initialized.
2. Below-minimum lots were silently bumped UP to SYMBOL_VOLUME_MIN by
TCNormalizeVolume - correct for a user-entered fixed lot, but in the
risk-sizing path it turned a budget-capped 0.05 into 0.10 on min-0.10/
step-0.01 symbols: double the intended risk, after CapRiskAmount already
clamped, exactly the routine-stop-out-breaches-the-daily-limit scenario
the budget exists to close. CMoneyRiskBase now refuses the trade when the
risk-derived lot is below the broker minimum.
3. All trading was async fire-and-forget (SetAsyncMode(true)) with no
OnTradeTransaction handler and no retry: server retcodes were never
observed. Fail-safe for entries, not for closes - a silently rejected
close rode the position until the next bar (or next day for the timed
close window). Now synchronous, matching the risk-budget flatten's own
already-synchronous CTrade; on an H1 EA the latency is irrelevant.
4. FIXED_LOT bypassed the budget entirely (no CapRiskAmount, no
OpenRiskAtStops) - pre-halt it could commit more than the remaining daily
allowance. A fixed lot cannot be scaled, so the rule is binary: its
loss-to-stop fits the remaining allowance whole or the trade is refused;
unpriceable risk (no SL) is refused while the budget is enabled.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
357 lines
18 KiB
MQL5
357 lines
18 KiB
MQL5
//+------------------------------------------------------------------+
|
|
//| TradeJournalManager.mqh |
|
|
//| AnimateDread |
|
|
//| https://www.mql5.com |
|
|
//+------------------------------------------------------------------+
|
|
#property copyright "AnimateDread"
|
|
#property link "https://www.mql5.com"
|
|
#include "DatabaseManager.mqh"
|
|
#include "..\Variables\ConfidenceBridge.mqh"
|
|
//--- g_riskBudget, fed one result per closed position so the expectancy rule has a sample. Include-
|
|
//--- guarded, and this file is pulled in before Money\ and Signals\ pull the same header, so the
|
|
//--- single global is defined exactly once wherever the include order lands.
|
|
#include "..\Variables\RiskBudget.mqh"
|
|
//+------------------------------------------------------------------+
|
|
//| One closed trade, as persisted to/read from the TradeJournal |
|
|
//| table. Field ORDER matters - DatabaseReadBind()/InsertTradeRecord |
|
|
//| match it positionally against the table's own column order |
|
|
//| (TradeJournalSchema below), not by name. |
|
|
//+------------------------------------------------------------------+
|
|
struct STradeJournalRecord
|
|
{
|
|
long ticket;
|
|
int openYear, openMonth, openDay, openDayOfWeek, openHour, openMinute;
|
|
int closeYear, closeMonth, closeDay, closeHour, closeMinute;
|
|
string symbol;
|
|
string direction;
|
|
double entryPrice, exitPrice, slPrice, tpPrice, lots;
|
|
double profit; // real net P&L: deal profit + swap + commission
|
|
double riskDistance; // |entryPrice-slPrice| at open; 0 if no SL was set
|
|
double rMultiple; // realized price move / riskDistance; 0 if riskDistance is 0
|
|
double maePoints, mfePoints; // worst adverse / best favorable excursion, price units, >=0
|
|
double maeR, mfeR; // same, normalized by riskDistance; 0 if riskDistance is 0
|
|
double aiConfidence, dbConfidence; // 0..1, snapshotted at entry
|
|
string exitReason; // SL/TP/Expert/Manual/StopOut/Other - from the closing deal's DEAL_REASON
|
|
string filterID; // which engine was driving trades this run (AIType name, or "Classic")
|
|
};
|
|
//--- column order matches STradeJournalRecord's field order exactly (see struct comment above)
|
|
const string TradeJournalSchema =
|
|
"ticket INTEGER, "
|
|
"openYear INTEGER, openMonth INTEGER, openDay INTEGER, openDayOfWeek INTEGER, openHour INTEGER, openMinute INTEGER, "
|
|
"closeYear INTEGER, closeMonth INTEGER, closeDay INTEGER, closeHour INTEGER, closeMinute INTEGER, "
|
|
"symbol TEXT, direction TEXT, "
|
|
"entryPrice REAL, exitPrice REAL, slPrice REAL, tpPrice REAL, lots REAL, "
|
|
"profit REAL, riskDistance REAL, rMultiple REAL, "
|
|
"maePoints REAL, mfePoints REAL, maeR REAL, mfeR REAL, "
|
|
"aiConfidence REAL, dbConfidence REAL, "
|
|
"exitReason TEXT, filterID TEXT";
|
|
//+------------------------------------------------------------------+
|
|
//| In-memory tracking for a still-open position - MAE/MFE can only |
|
|
//| be measured live, tick by tick, while the position exists; there |
|
|
//| is no post-hoc MQL5 API to recover it once the position is gone. |
|
|
//+------------------------------------------------------------------+
|
|
struct SJournalOpenTrack
|
|
{
|
|
ulong ticket;
|
|
datetime openTime;
|
|
string symbol;
|
|
string direction;
|
|
double entryPrice, slPrice, tpPrice, lots;
|
|
double riskDistance;
|
|
double aiConfidence, dbConfidence;
|
|
string filterID;
|
|
double maePoints;
|
|
double mfePoints;
|
|
};
|
|
//+------------------------------------------------------------------+
|
|
//| Owns the TradeJournal table: detects this EA's own positions |
|
|
//| opening/closing (by polling PositionsTotal() every tick rather |
|
|
//| than hooking OnTradeTransaction - simpler and robust against |
|
|
//| partial fills/multiple deals per position), tracks MAE/MFE live |
|
|
//| while a position is open, and resolves the real closing P&L/ |
|
|
//| reason from deal history (HistoryDealGetInteger(DEAL_REASON) - |
|
|
//| broker-confirmed, not a heuristic) once it closes. |
|
|
//+------------------------------------------------------------------+
|
|
class CTradeJournalManager
|
|
{
|
|
private:
|
|
CDatabaseManager *m_dbm;
|
|
ulong m_magic;
|
|
string m_tableName;
|
|
SJournalOpenTrack m_tracked[];
|
|
|
|
int FindTracked(ulong ticket)
|
|
{
|
|
for(int i = 0; i < ArraySize(m_tracked); i++)
|
|
if(m_tracked[i].ticket == ticket)
|
|
return i;
|
|
return -1;
|
|
}
|
|
void RemoveTracked(int idx)
|
|
{
|
|
int last = ArraySize(m_tracked) - 1;
|
|
if(idx < 0 || idx > last)
|
|
return;
|
|
if(idx != last)
|
|
m_tracked[idx] = m_tracked[last];
|
|
ArrayResize(m_tracked, last);
|
|
}
|
|
string CurrentFilterID(void)
|
|
{
|
|
if(AIType == AI_NONE)
|
|
return "Classic";
|
|
return EnumToString((AI_CHOICE)AIType);
|
|
}
|
|
string ExitReasonFromDealReason(long reason)
|
|
{
|
|
switch((ENUM_DEAL_REASON)reason)
|
|
{
|
|
case DEAL_REASON_SL: return "SL";
|
|
case DEAL_REASON_TP: return "TP";
|
|
case DEAL_REASON_EXPERT: return "Expert";
|
|
case DEAL_REASON_CLIENT:
|
|
case DEAL_REASON_MOBILE:
|
|
case DEAL_REASON_WEB: return "Manual";
|
|
case DEAL_REASON_SO: return "StopOut";
|
|
default: return "Other";
|
|
}
|
|
}
|
|
//--- resolves the closing deal for a position no longer in PositionsTotal() - returns false if
|
|
//--- history hasn't caught up yet (rare timing edge case); caller keeps tracking it and retries
|
|
//--- next tick rather than dropping the trade unrecorded.
|
|
//--- sums profit across every OUT/INOUT deal for this position (covers a partial close followed by
|
|
//--- a final close, however rare) rather than trusting a single deal to represent the whole
|
|
//--- position; exitPrice/exitReason are taken from the LAST (most recent) such deal, representing
|
|
//--- how the position ultimately finished.
|
|
//--- Money that was at risk on this trade, in account currency: the entry-to-stop distance converted
|
|
//--- through the symbol's own tick value, which is what the sizing used in the first place. Returns 0
|
|
//--- when the trade carried no stop or the symbol's tick data is unavailable, and the caller then
|
|
//--- simply does not score it - a trade with no stop has no R and guessing one would corrupt the mean.
|
|
double RiskAmountOf(const SJournalOpenTrack &t)
|
|
{
|
|
if(t.riskDistance <= 0.0 || t.lots <= 0.0)
|
|
return 0.0;
|
|
double tickSize = SymbolInfoDouble(t.symbol, SYMBOL_TRADE_TICK_SIZE);
|
|
double tickValue = SymbolInfoDouble(t.symbol, SYMBOL_TRADE_TICK_VALUE);
|
|
if(tickSize <= 0.0 || tickValue <= 0.0)
|
|
return 0.0;
|
|
return (t.riskDistance / tickSize) * tickValue * t.lots;
|
|
}
|
|
bool ResolveClose(ulong ticket, double &exitPrice, double &profit, string &exitReason)
|
|
{
|
|
if(!HistorySelectByPosition((long)ticket))
|
|
return false;
|
|
int deals = HistoryDealsTotal();
|
|
bool found = false;
|
|
profit = 0.0;
|
|
for(int d = 0; d < deals; d++)
|
|
{
|
|
ulong dealTicket = HistoryDealGetTicket(d);
|
|
if(dealTicket == 0)
|
|
continue;
|
|
long entry = HistoryDealGetInteger(dealTicket, DEAL_ENTRY);
|
|
if(entry != DEAL_ENTRY_OUT && entry != DEAL_ENTRY_INOUT)
|
|
continue;
|
|
profit += HistoryDealGetDouble(dealTicket, DEAL_PROFIT) +
|
|
HistoryDealGetDouble(dealTicket, DEAL_SWAP) +
|
|
HistoryDealGetDouble(dealTicket, DEAL_COMMISSION);
|
|
exitPrice = HistoryDealGetDouble(dealTicket, DEAL_PRICE);
|
|
exitReason = ExitReasonFromDealReason(HistoryDealGetInteger(dealTicket, DEAL_REASON));
|
|
found = true;
|
|
}
|
|
return found;
|
|
}
|
|
bool InsertClosedTrade(const SJournalOpenTrack &t, double exitPrice, double profit, string exitReason)
|
|
{
|
|
MqlDateTime openT, closeT;
|
|
TimeToStruct(t.openTime, openT);
|
|
TimeToStruct(TimeCurrent(), closeT);
|
|
double move = (t.direction == "Buy") ? (exitPrice - t.entryPrice) : (t.entryPrice - exitPrice);
|
|
double riskDistance = t.riskDistance;
|
|
double rMultiple = (riskDistance > 0.0) ? move / riskDistance : 0.0;
|
|
double maeR = (riskDistance > 0.0) ? t.maePoints / riskDistance : 0.0;
|
|
double mfeR = (riskDistance > 0.0) ? t.mfePoints / riskDistance : 0.0;
|
|
string cols[] = {"ticket", "openYear", "openMonth", "openDay", "openDayOfWeek", "openHour", "openMinute",
|
|
"closeYear", "closeMonth", "closeDay", "closeHour", "closeMinute",
|
|
"symbol", "direction", "entryPrice", "exitPrice", "slPrice", "tpPrice", "lots",
|
|
"profit", "riskDistance", "rMultiple", "maePoints", "mfePoints", "maeR", "mfeR",
|
|
"aiConfidence", "dbConfidence", "exitReason", "filterID"
|
|
};
|
|
string vals[];
|
|
ArrayResize(vals, ArraySize(cols));
|
|
int i = 0;
|
|
vals[i++] = IntegerToString((long)t.ticket);
|
|
vals[i++] = IntegerToString(openT.year);
|
|
vals[i++] = IntegerToString(openT.mon);
|
|
vals[i++] = IntegerToString(openT.day);
|
|
vals[i++] = IntegerToString(openT.day_of_week);
|
|
vals[i++] = IntegerToString(openT.hour);
|
|
vals[i++] = IntegerToString(openT.min);
|
|
vals[i++] = IntegerToString(closeT.year);
|
|
vals[i++] = IntegerToString(closeT.mon);
|
|
vals[i++] = IntegerToString(closeT.day);
|
|
vals[i++] = IntegerToString(closeT.hour);
|
|
vals[i++] = IntegerToString(closeT.min);
|
|
vals[i++] = t.symbol;
|
|
vals[i++] = t.direction;
|
|
vals[i++] = DoubleToString(t.entryPrice, 8);
|
|
vals[i++] = DoubleToString(exitPrice, 8);
|
|
vals[i++] = DoubleToString(t.slPrice, 8);
|
|
vals[i++] = DoubleToString(t.tpPrice, 8);
|
|
vals[i++] = DoubleToString(t.lots, 2);
|
|
vals[i++] = DoubleToString(profit, 2);
|
|
vals[i++] = DoubleToString(riskDistance, 8);
|
|
vals[i++] = DoubleToString(rMultiple, 4);
|
|
vals[i++] = DoubleToString(t.maePoints, 8);
|
|
vals[i++] = DoubleToString(t.mfePoints, 8);
|
|
vals[i++] = DoubleToString(maeR, 4);
|
|
vals[i++] = DoubleToString(mfeR, 4);
|
|
vals[i++] = DoubleToString(t.aiConfidence, 4);
|
|
vals[i++] = DoubleToString(t.dbConfidence, 4);
|
|
vals[i++] = exitReason;
|
|
vals[i++] = t.filterID;
|
|
if(!m_dbm.BeginTransaction())
|
|
return false;
|
|
bool ok = m_dbm.InsertTradeRecord(m_tableName, cols, vals);
|
|
if(!m_dbm.CommitTransaction())
|
|
ok = false;
|
|
return ok;
|
|
}
|
|
|
|
public:
|
|
CTradeJournalManager(void) : m_dbm(NULL), m_magic(0), m_tableName("TradeJournal") {}
|
|
bool Init(CDatabaseManager *dbmPtr, ulong magic)
|
|
{
|
|
m_dbm = dbmPtr;
|
|
m_magic = magic;
|
|
if(CheckPointer(m_dbm) == POINTER_INVALID)
|
|
return false;
|
|
//--- CreateTable() needs a genuinely open handle - dbm.Init() only stores the path, it doesn't
|
|
//--- open it (OpenDatabase() does that, and Warrior_EA.mq5 doesn't call it until well after
|
|
//--- filters/patterns are registered). Opening explicitly here means this also has to run
|
|
//--- before AddFilterToSignal() - see the call site in Warrior_EA.mq5's OnInit().
|
|
if(!m_dbm.OpenDatabase())
|
|
return false;
|
|
return m_dbm.CreateTable(m_tableName, TradeJournalSchema);
|
|
}
|
|
//--- Tracking without a database. The expectancy stop (g_riskBudget.RecordTradeResult) is fed from
|
|
//--- this class's close detection, and until 2026-08-11 that feed only existed when
|
|
//--- UseDatabaseRanking was on - which ships FALSE, so the da54639 "EV = minus the cost" halt was
|
|
//--- armed (ExpectancyMinTrades=40) and never received a single closed trade on a default install.
|
|
//--- A risk rule must not be a side effect of an optional analytics toggle: this init gives the
|
|
//--- close-detection/MAE-MFE/expectancy path a life of its own, and Update() below simply skips the
|
|
//--- DB insert when there is no DB.
|
|
void InitTrackingOnly(ulong magic)
|
|
{
|
|
m_dbm = NULL;
|
|
m_magic = magic;
|
|
}
|
|
//--- call once per tick: detects this EA's own positions opening/closing on the current symbol and
|
|
//--- updates MAE/MFE for every one still open. Cheap - PositionsTotal() is typically single digits.
|
|
//--- Runs with or without a database (see InitTrackingOnly); only the journal INSERT needs one.
|
|
void Update(void)
|
|
{
|
|
int preExistingCount = ArraySize(m_tracked);
|
|
bool seen[];
|
|
ArrayResize(seen, preExistingCount);
|
|
for(int i = 0; i < preExistingCount; i++)
|
|
seen[i] = false;
|
|
int total = PositionsTotal();
|
|
for(int p = 0; p < total; p++)
|
|
{
|
|
ulong ticket = PositionGetTicket(p);
|
|
if(ticket == 0)
|
|
continue;
|
|
if(!PositionSelectByTicket(ticket))
|
|
continue;
|
|
if(PositionGetString(POSITION_SYMBOL) != _Symbol)
|
|
continue;
|
|
if((ulong)PositionGetInteger(POSITION_MAGIC) != m_magic)
|
|
continue;
|
|
double currentPrice = PositionGetDouble(POSITION_PRICE_CURRENT);
|
|
int idx = FindTracked(ticket);
|
|
if(idx < 0)
|
|
{
|
|
SJournalOpenTrack t;
|
|
t.ticket = ticket;
|
|
t.openTime = (datetime)PositionGetInteger(POSITION_TIME);
|
|
t.symbol = _Symbol;
|
|
t.direction = (PositionGetInteger(POSITION_TYPE) == POSITION_TYPE_BUY) ? "Buy" : "Sell";
|
|
t.entryPrice = PositionGetDouble(POSITION_PRICE_OPEN);
|
|
t.slPrice = PositionGetDouble(POSITION_SL);
|
|
t.tpPrice = PositionGetDouble(POSITION_TP);
|
|
t.lots = PositionGetDouble(POSITION_VOLUME);
|
|
t.riskDistance = (t.slPrice > 0.0) ? MathAbs(t.entryPrice - t.slPrice) : 0.0;
|
|
//--- same-tick snapshot OpenParams() populated right before this trade was sent - see
|
|
//--- ConfidenceBridge.mqh's declaration comments.
|
|
t.aiConfidence = MathAbs(g_AISignedConfidence);
|
|
t.dbConfidence = g_DBConfidence;
|
|
t.filterID = CurrentFilterID();
|
|
t.maePoints = 0.0;
|
|
t.mfePoints = 0.0;
|
|
int newIdx = ArraySize(m_tracked);
|
|
ArrayResize(m_tracked, newIdx + 1);
|
|
m_tracked[newIdx] = t;
|
|
}
|
|
else
|
|
{
|
|
if(idx < preExistingCount)
|
|
seen[idx] = true;
|
|
double excursion = (m_tracked[idx].direction == "Buy") ?
|
|
(currentPrice - m_tracked[idx].entryPrice) :
|
|
(m_tracked[idx].entryPrice - currentPrice);
|
|
if(excursion > m_tracked[idx].mfePoints)
|
|
m_tracked[idx].mfePoints = excursion;
|
|
if(-excursion > m_tracked[idx].maePoints)
|
|
m_tracked[idx].maePoints = -excursion;
|
|
}
|
|
}
|
|
//--- anything tracked before this pass but not seen in it closed since the last tick - resolve
|
|
//--- and record it. Walk backwards since RemoveTracked() swap-removes (changes indices >= idx).
|
|
for(int i = preExistingCount - 1; i >= 0; i--)
|
|
{
|
|
if(seen[i])
|
|
continue;
|
|
double exitPrice = 0.0, profit = 0.0;
|
|
string exitReason = "Other";
|
|
if(ResolveClose(m_tracked[i].ticket, exitPrice, profit, exitReason))
|
|
{
|
|
// MAE/MFE and the rest of this closed trade's record cannot be reconstructed after the
|
|
// fact once m_tracked[i] is removed below - if the DB insert fails (lock contention on
|
|
// the shared COMMON db, disk issue, retry exhaustion), log every field so the record is
|
|
// at least manually recoverable from the Experts journal instead of silently vanishing.
|
|
//--- FEED THE EXPECTANCY RULE. Reported in R - net profit over the money that was actually
|
|
//--- at risk - so results from different symbols, lot sizes and account balances are on one
|
|
//--- scale and can share a single mean. riskDistance is the entry-to-stop distance the trade
|
|
//--- was sized against, so lots * riskDistance * tickValue-per-point IS the amount at risk;
|
|
//--- using the account's own currency conversion via the profit figure keeps it exact rather
|
|
//--- than reconstructing tick values here.
|
|
//--- `profit` already includes swap and commission (see ResolveClose). That is deliberate and
|
|
//--- load-bearing: when the directional edge is zero, cost is the ENTIRE expectancy, so a
|
|
//--- gross-profit version of this rule would measure a strategy nobody can actually trade.
|
|
double riskAmount = RiskAmountOf(m_tracked[i]);
|
|
if(riskAmount > 0.0)
|
|
g_riskBudget.RecordTradeResult(profit / riskAmount);
|
|
if(CheckPointer(m_dbm) == POINTER_INVALID)
|
|
{
|
|
//--- tracking-only mode (no UseDatabaseRanking): the expectancy rule above is the
|
|
//--- whole point; there is no journal DB to insert into.
|
|
RemoveTracked(i);
|
|
continue;
|
|
}
|
|
if(!InsertClosedTrade(m_tracked[i], exitPrice, profit, exitReason))
|
|
PrintFormat("%s: ERROR - failed to insert closed trade into journal DB, record LOST from DB (recoverable from this log only): ticket=%I64u direction=%s entry=%.5f exit=%.5f profit=%.2f mae=%.5f mfe=%.5f reason=%s",
|
|
__FUNCTION__, m_tracked[i].ticket, m_tracked[i].direction, m_tracked[i].entryPrice, exitPrice, profit,
|
|
m_tracked[i].maePoints, m_tracked[i].mfePoints, exitReason);
|
|
RemoveTracked(i);
|
|
}
|
|
//--- else: history hasn't caught up yet this tick - leave it tracked and retry next tick
|
|
}
|
|
}
|
|
//--- forward-declared here, implemented in TradeJournalReport.mqh (kept separate - this file is
|
|
//--- the live tracking path, that one is the offline reporting/insights path; no reason for the
|
|
//--- per-tick code to pull in report-building logic it never calls).
|
|
bool GenerateReport(string &resultPath, string &errorMsg);
|
|
};
|
|
//--- CTradeJournalManager::GenerateReport() - split out, see that file's own header comment.
|
|
#include "TradeJournalReport.mqh"
|