da54639996
The daily (4%) and total (8%) rules bound how FAST an account can lose. Nothing
noticed WHETHER it was losing. A negative-expectancy signal traded at 1% inside
that envelope breaches no rule and still arrives at zero - it just takes longer,
with every limit green the whole way down. That is the realistic way this EA
destroys an account, and no existing guard could see it.
THE ARITHMETIC THIS ENFORCES. Expected value per trade is p*TP - (1-p)*SL - cost.
With no directional edge p equals SL/(SL+TP), which is also the break-even rate,
so the payoff terms cancel exactly and EV = -cost. Expected P&L is -(trades) x
cost: strictly negative, proportional to activity. Measured here: directional
precision 23-24% against a 25% break-even, flat across every confidence tier,
with 58 points of spread on SP500. Sizing, stop placement and trailing move
variance around that mean; none of them changes its sign.
So every closed position now reports its result in R (net profit over money
actually at risk) and the running mean is tested against zero. Above the
configured minimum sample, if mean + sigma*SE < 0, new entries stop.
- SIGNIFICANTLY below, not merely below. A run of losers is ordinary variance
even for a profitable system; halting on the raw mean would be the same
act-on-noise error the MI gates exist to prevent. Using the standard error
means a wide spread simply demands more trades before the rule can fire.
- NET of swap and commission (ResolveClose already sums all three). Deliberate
and load-bearing: when the edge is zero, cost IS the expectancy, so a gross
version would measure a strategy nobody can trade.
- Reported in R so symbols, lot sizes and balances share one scale and one
mean. Trades without a stop are not scored rather than assigned a guessed R.
- LATCHED across restarts, like the daily halt and for the same reason: a
latch a reattach clears is not a latch. Clearing it means deleting the risk
state file, deliberately, after looking at why.
State is appended to the risk file length-guarded, so files written before this
still load and start their sample at zero rather than misreading.
Defaults 40 trades / 2 sigma; ExpectancyMinTrades = 0 disables it.
This does not make the strategy profitable and is not meant to. It stops paying
tuition on one the results say is losing, and does it on measurement rather than
on a drawdown limit finally being reached.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
339 lines
17 KiB
MQL5
339 lines
17 KiB
MQL5
//+------------------------------------------------------------------+
|
|
//| TradeJournalManager.mqh |
|
|
//| AnimateDread |
|
|
//| https://www.mql5.com |
|
|
//+------------------------------------------------------------------+
|
|
#property copyright "AnimateDread"
|
|
#property link "https://www.mql5.com"
|
|
#include "DatabaseManager.mqh"
|
|
#include "..\Variables\ConfidenceBridge.mqh"
|
|
//--- g_riskBudget, fed one result per closed position so the expectancy rule has a sample. Include-
|
|
//--- guarded, and this file is pulled in before Money\ and Signals\ pull the same header, so the
|
|
//--- single global is defined exactly once wherever the include order lands.
|
|
#include "..\Variables\RiskBudget.mqh"
|
|
//+------------------------------------------------------------------+
|
|
//| One closed trade, as persisted to/read from the TradeJournal |
|
|
//| table. Field ORDER matters - DatabaseReadBind()/InsertTradeRecord |
|
|
//| match it positionally against the table's own column order |
|
|
//| (TradeJournalSchema below), not by name. |
|
|
//+------------------------------------------------------------------+
|
|
struct STradeJournalRecord
|
|
{
|
|
long ticket;
|
|
int openYear, openMonth, openDay, openDayOfWeek, openHour, openMinute;
|
|
int closeYear, closeMonth, closeDay, closeHour, closeMinute;
|
|
string symbol;
|
|
string direction;
|
|
double entryPrice, exitPrice, slPrice, tpPrice, lots;
|
|
double profit; // real net P&L: deal profit + swap + commission
|
|
double riskDistance; // |entryPrice-slPrice| at open; 0 if no SL was set
|
|
double rMultiple; // realized price move / riskDistance; 0 if riskDistance is 0
|
|
double maePoints, mfePoints; // worst adverse / best favorable excursion, price units, >=0
|
|
double maeR, mfeR; // same, normalized by riskDistance; 0 if riskDistance is 0
|
|
double aiConfidence, dbConfidence; // 0..1, snapshotted at entry
|
|
string exitReason; // SL/TP/Expert/Manual/StopOut/Other - from the closing deal's DEAL_REASON
|
|
string filterID; // which engine was driving trades this run (AIType name, or "Classic")
|
|
};
|
|
//--- column order matches STradeJournalRecord's field order exactly (see struct comment above)
|
|
const string TradeJournalSchema =
|
|
"ticket INTEGER, "
|
|
"openYear INTEGER, openMonth INTEGER, openDay INTEGER, openDayOfWeek INTEGER, openHour INTEGER, openMinute INTEGER, "
|
|
"closeYear INTEGER, closeMonth INTEGER, closeDay INTEGER, closeHour INTEGER, closeMinute INTEGER, "
|
|
"symbol TEXT, direction TEXT, "
|
|
"entryPrice REAL, exitPrice REAL, slPrice REAL, tpPrice REAL, lots REAL, "
|
|
"profit REAL, riskDistance REAL, rMultiple REAL, "
|
|
"maePoints REAL, mfePoints REAL, maeR REAL, mfeR REAL, "
|
|
"aiConfidence REAL, dbConfidence REAL, "
|
|
"exitReason TEXT, filterID TEXT";
|
|
//+------------------------------------------------------------------+
|
|
//| In-memory tracking for a still-open position - MAE/MFE can only |
|
|
//| be measured live, tick by tick, while the position exists; there |
|
|
//| is no post-hoc MQL5 API to recover it once the position is gone. |
|
|
//+------------------------------------------------------------------+
|
|
struct SJournalOpenTrack
|
|
{
|
|
ulong ticket;
|
|
datetime openTime;
|
|
string symbol;
|
|
string direction;
|
|
double entryPrice, slPrice, tpPrice, lots;
|
|
double riskDistance;
|
|
double aiConfidence, dbConfidence;
|
|
string filterID;
|
|
double maePoints;
|
|
double mfePoints;
|
|
};
|
|
//+------------------------------------------------------------------+
|
|
//| Owns the TradeJournal table: detects this EA's own positions |
|
|
//| opening/closing (by polling PositionsTotal() every tick rather |
|
|
//| than hooking OnTradeTransaction - simpler and robust against |
|
|
//| partial fills/multiple deals per position), tracks MAE/MFE live |
|
|
//| while a position is open, and resolves the real closing P&L/ |
|
|
//| reason from deal history (HistoryDealGetInteger(DEAL_REASON) - |
|
|
//| broker-confirmed, not a heuristic) once it closes. |
|
|
//+------------------------------------------------------------------+
|
|
class CTradeJournalManager
|
|
{
|
|
private:
|
|
CDatabaseManager *m_dbm;
|
|
ulong m_magic;
|
|
string m_tableName;
|
|
SJournalOpenTrack m_tracked[];
|
|
|
|
int FindTracked(ulong ticket)
|
|
{
|
|
for(int i = 0; i < ArraySize(m_tracked); i++)
|
|
if(m_tracked[i].ticket == ticket)
|
|
return i;
|
|
return -1;
|
|
}
|
|
void RemoveTracked(int idx)
|
|
{
|
|
int last = ArraySize(m_tracked) - 1;
|
|
if(idx < 0 || idx > last)
|
|
return;
|
|
if(idx != last)
|
|
m_tracked[idx] = m_tracked[last];
|
|
ArrayResize(m_tracked, last);
|
|
}
|
|
string CurrentFilterID(void)
|
|
{
|
|
if(AIType == AI_NONE)
|
|
return "Classic";
|
|
return EnumToString((AI_CHOICE)AIType);
|
|
}
|
|
string ExitReasonFromDealReason(long reason)
|
|
{
|
|
switch((ENUM_DEAL_REASON)reason)
|
|
{
|
|
case DEAL_REASON_SL: return "SL";
|
|
case DEAL_REASON_TP: return "TP";
|
|
case DEAL_REASON_EXPERT: return "Expert";
|
|
case DEAL_REASON_CLIENT:
|
|
case DEAL_REASON_MOBILE:
|
|
case DEAL_REASON_WEB: return "Manual";
|
|
case DEAL_REASON_SO: return "StopOut";
|
|
default: return "Other";
|
|
}
|
|
}
|
|
//--- resolves the closing deal for a position no longer in PositionsTotal() - returns false if
|
|
//--- history hasn't caught up yet (rare timing edge case); caller keeps tracking it and retries
|
|
//--- next tick rather than dropping the trade unrecorded.
|
|
//--- sums profit across every OUT/INOUT deal for this position (covers a partial close followed by
|
|
//--- a final close, however rare) rather than trusting a single deal to represent the whole
|
|
//--- position; exitPrice/exitReason are taken from the LAST (most recent) such deal, representing
|
|
//--- how the position ultimately finished.
|
|
//--- Money that was at risk on this trade, in account currency: the entry-to-stop distance converted
|
|
//--- through the symbol's own tick value, which is what the sizing used in the first place. Returns 0
|
|
//--- when the trade carried no stop or the symbol's tick data is unavailable, and the caller then
|
|
//--- simply does not score it - a trade with no stop has no R and guessing one would corrupt the mean.
|
|
double RiskAmountOf(const SJournalOpenTrack &t)
|
|
{
|
|
if(t.riskDistance <= 0.0 || t.lots <= 0.0)
|
|
return 0.0;
|
|
double tickSize = SymbolInfoDouble(t.symbol, SYMBOL_TRADE_TICK_SIZE);
|
|
double tickValue = SymbolInfoDouble(t.symbol, SYMBOL_TRADE_TICK_VALUE);
|
|
if(tickSize <= 0.0 || tickValue <= 0.0)
|
|
return 0.0;
|
|
return (t.riskDistance / tickSize) * tickValue * t.lots;
|
|
}
|
|
bool ResolveClose(ulong ticket, double &exitPrice, double &profit, string &exitReason)
|
|
{
|
|
if(!HistorySelectByPosition((long)ticket))
|
|
return false;
|
|
int deals = HistoryDealsTotal();
|
|
bool found = false;
|
|
profit = 0.0;
|
|
for(int d = 0; d < deals; d++)
|
|
{
|
|
ulong dealTicket = HistoryDealGetTicket(d);
|
|
if(dealTicket == 0)
|
|
continue;
|
|
long entry = HistoryDealGetInteger(dealTicket, DEAL_ENTRY);
|
|
if(entry != DEAL_ENTRY_OUT && entry != DEAL_ENTRY_INOUT)
|
|
continue;
|
|
profit += HistoryDealGetDouble(dealTicket, DEAL_PROFIT) +
|
|
HistoryDealGetDouble(dealTicket, DEAL_SWAP) +
|
|
HistoryDealGetDouble(dealTicket, DEAL_COMMISSION);
|
|
exitPrice = HistoryDealGetDouble(dealTicket, DEAL_PRICE);
|
|
exitReason = ExitReasonFromDealReason(HistoryDealGetInteger(dealTicket, DEAL_REASON));
|
|
found = true;
|
|
}
|
|
return found;
|
|
}
|
|
bool InsertClosedTrade(const SJournalOpenTrack &t, double exitPrice, double profit, string exitReason)
|
|
{
|
|
MqlDateTime openT, closeT;
|
|
TimeToStruct(t.openTime, openT);
|
|
TimeToStruct(TimeCurrent(), closeT);
|
|
double move = (t.direction == "Buy") ? (exitPrice - t.entryPrice) : (t.entryPrice - exitPrice);
|
|
double riskDistance = t.riskDistance;
|
|
double rMultiple = (riskDistance > 0.0) ? move / riskDistance : 0.0;
|
|
double maeR = (riskDistance > 0.0) ? t.maePoints / riskDistance : 0.0;
|
|
double mfeR = (riskDistance > 0.0) ? t.mfePoints / riskDistance : 0.0;
|
|
string cols[] = {"ticket", "openYear", "openMonth", "openDay", "openDayOfWeek", "openHour", "openMinute",
|
|
"closeYear", "closeMonth", "closeDay", "closeHour", "closeMinute",
|
|
"symbol", "direction", "entryPrice", "exitPrice", "slPrice", "tpPrice", "lots",
|
|
"profit", "riskDistance", "rMultiple", "maePoints", "mfePoints", "maeR", "mfeR",
|
|
"aiConfidence", "dbConfidence", "exitReason", "filterID"
|
|
};
|
|
string vals[];
|
|
ArrayResize(vals, ArraySize(cols));
|
|
int i = 0;
|
|
vals[i++] = IntegerToString((long)t.ticket);
|
|
vals[i++] = IntegerToString(openT.year);
|
|
vals[i++] = IntegerToString(openT.mon);
|
|
vals[i++] = IntegerToString(openT.day);
|
|
vals[i++] = IntegerToString(openT.day_of_week);
|
|
vals[i++] = IntegerToString(openT.hour);
|
|
vals[i++] = IntegerToString(openT.min);
|
|
vals[i++] = IntegerToString(closeT.year);
|
|
vals[i++] = IntegerToString(closeT.mon);
|
|
vals[i++] = IntegerToString(closeT.day);
|
|
vals[i++] = IntegerToString(closeT.hour);
|
|
vals[i++] = IntegerToString(closeT.min);
|
|
vals[i++] = t.symbol;
|
|
vals[i++] = t.direction;
|
|
vals[i++] = DoubleToString(t.entryPrice, 8);
|
|
vals[i++] = DoubleToString(exitPrice, 8);
|
|
vals[i++] = DoubleToString(t.slPrice, 8);
|
|
vals[i++] = DoubleToString(t.tpPrice, 8);
|
|
vals[i++] = DoubleToString(t.lots, 2);
|
|
vals[i++] = DoubleToString(profit, 2);
|
|
vals[i++] = DoubleToString(riskDistance, 8);
|
|
vals[i++] = DoubleToString(rMultiple, 4);
|
|
vals[i++] = DoubleToString(t.maePoints, 8);
|
|
vals[i++] = DoubleToString(t.mfePoints, 8);
|
|
vals[i++] = DoubleToString(maeR, 4);
|
|
vals[i++] = DoubleToString(mfeR, 4);
|
|
vals[i++] = DoubleToString(t.aiConfidence, 4);
|
|
vals[i++] = DoubleToString(t.dbConfidence, 4);
|
|
vals[i++] = exitReason;
|
|
vals[i++] = t.filterID;
|
|
if(!m_dbm.BeginTransaction())
|
|
return false;
|
|
bool ok = m_dbm.InsertTradeRecord(m_tableName, cols, vals);
|
|
if(!m_dbm.CommitTransaction())
|
|
ok = false;
|
|
return ok;
|
|
}
|
|
|
|
public:
|
|
CTradeJournalManager(void) : m_dbm(NULL), m_magic(0), m_tableName("TradeJournal") {}
|
|
bool Init(CDatabaseManager *dbmPtr, ulong magic)
|
|
{
|
|
m_dbm = dbmPtr;
|
|
m_magic = magic;
|
|
if(CheckPointer(m_dbm) == POINTER_INVALID)
|
|
return false;
|
|
//--- CreateTable() needs a genuinely open handle - dbm.Init() only stores the path, it doesn't
|
|
//--- open it (OpenDatabase() does that, and Warrior_EA.mq5 doesn't call it until well after
|
|
//--- filters/patterns are registered). Opening explicitly here means this also has to run
|
|
//--- before AddFilterToSignal() - see the call site in Warrior_EA.mq5's OnInit().
|
|
if(!m_dbm.OpenDatabase())
|
|
return false;
|
|
return m_dbm.CreateTable(m_tableName, TradeJournalSchema);
|
|
}
|
|
//--- call once per tick: detects this EA's own positions opening/closing on the current symbol and
|
|
//--- updates MAE/MFE for every one still open. Cheap - PositionsTotal() is typically single digits.
|
|
void Update(void)
|
|
{
|
|
if(CheckPointer(m_dbm) == POINTER_INVALID)
|
|
return;
|
|
int preExistingCount = ArraySize(m_tracked);
|
|
bool seen[];
|
|
ArrayResize(seen, preExistingCount);
|
|
for(int i = 0; i < preExistingCount; i++)
|
|
seen[i] = false;
|
|
int total = PositionsTotal();
|
|
for(int p = 0; p < total; p++)
|
|
{
|
|
ulong ticket = PositionGetTicket(p);
|
|
if(ticket == 0)
|
|
continue;
|
|
if(!PositionSelectByTicket(ticket))
|
|
continue;
|
|
if(PositionGetString(POSITION_SYMBOL) != _Symbol)
|
|
continue;
|
|
if((ulong)PositionGetInteger(POSITION_MAGIC) != m_magic)
|
|
continue;
|
|
double currentPrice = PositionGetDouble(POSITION_PRICE_CURRENT);
|
|
int idx = FindTracked(ticket);
|
|
if(idx < 0)
|
|
{
|
|
SJournalOpenTrack t;
|
|
t.ticket = ticket;
|
|
t.openTime = (datetime)PositionGetInteger(POSITION_TIME);
|
|
t.symbol = _Symbol;
|
|
t.direction = (PositionGetInteger(POSITION_TYPE) == POSITION_TYPE_BUY) ? "Buy" : "Sell";
|
|
t.entryPrice = PositionGetDouble(POSITION_PRICE_OPEN);
|
|
t.slPrice = PositionGetDouble(POSITION_SL);
|
|
t.tpPrice = PositionGetDouble(POSITION_TP);
|
|
t.lots = PositionGetDouble(POSITION_VOLUME);
|
|
t.riskDistance = (t.slPrice > 0.0) ? MathAbs(t.entryPrice - t.slPrice) : 0.0;
|
|
//--- same-tick snapshot OpenParams() populated right before this trade was sent - see
|
|
//--- ConfidenceBridge.mqh's declaration comments.
|
|
t.aiConfidence = MathAbs(g_AISignedConfidence);
|
|
t.dbConfidence = g_DBConfidence;
|
|
t.filterID = CurrentFilterID();
|
|
t.maePoints = 0.0;
|
|
t.mfePoints = 0.0;
|
|
int newIdx = ArraySize(m_tracked);
|
|
ArrayResize(m_tracked, newIdx + 1);
|
|
m_tracked[newIdx] = t;
|
|
}
|
|
else
|
|
{
|
|
if(idx < preExistingCount)
|
|
seen[idx] = true;
|
|
double excursion = (m_tracked[idx].direction == "Buy") ?
|
|
(currentPrice - m_tracked[idx].entryPrice) :
|
|
(m_tracked[idx].entryPrice - currentPrice);
|
|
if(excursion > m_tracked[idx].mfePoints)
|
|
m_tracked[idx].mfePoints = excursion;
|
|
if(-excursion > m_tracked[idx].maePoints)
|
|
m_tracked[idx].maePoints = -excursion;
|
|
}
|
|
}
|
|
//--- anything tracked before this pass but not seen in it closed since the last tick - resolve
|
|
//--- and record it. Walk backwards since RemoveTracked() swap-removes (changes indices >= idx).
|
|
for(int i = preExistingCount - 1; i >= 0; i--)
|
|
{
|
|
if(seen[i])
|
|
continue;
|
|
double exitPrice = 0.0, profit = 0.0;
|
|
string exitReason = "Other";
|
|
if(ResolveClose(m_tracked[i].ticket, exitPrice, profit, exitReason))
|
|
{
|
|
// MAE/MFE and the rest of this closed trade's record cannot be reconstructed after the
|
|
// fact once m_tracked[i] is removed below - if the DB insert fails (lock contention on
|
|
// the shared COMMON db, disk issue, retry exhaustion), log every field so the record is
|
|
// at least manually recoverable from the Experts journal instead of silently vanishing.
|
|
//--- FEED THE EXPECTANCY RULE. Reported in R - net profit over the money that was actually
|
|
//--- at risk - so results from different symbols, lot sizes and account balances are on one
|
|
//--- scale and can share a single mean. riskDistance is the entry-to-stop distance the trade
|
|
//--- was sized against, so lots * riskDistance * tickValue-per-point IS the amount at risk;
|
|
//--- using the account's own currency conversion via the profit figure keeps it exact rather
|
|
//--- than reconstructing tick values here.
|
|
//--- `profit` already includes swap and commission (see ResolveClose). That is deliberate and
|
|
//--- load-bearing: when the directional edge is zero, cost is the ENTIRE expectancy, so a
|
|
//--- gross-profit version of this rule would measure a strategy nobody can actually trade.
|
|
double riskAmount = RiskAmountOf(m_tracked[i]);
|
|
if(riskAmount > 0.0)
|
|
g_riskBudget.RecordTradeResult(profit / riskAmount);
|
|
if(!InsertClosedTrade(m_tracked[i], exitPrice, profit, exitReason))
|
|
PrintFormat("%s: ERROR - failed to insert closed trade into journal DB, record LOST from DB (recoverable from this log only): ticket=%I64u direction=%s entry=%.5f exit=%.5f profit=%.2f mae=%.5f mfe=%.5f reason=%s",
|
|
__FUNCTION__, m_tracked[i].ticket, m_tracked[i].direction, m_tracked[i].entryPrice, exitPrice, profit,
|
|
m_tracked[i].maePoints, m_tracked[i].mfePoints, exitReason);
|
|
RemoveTracked(i);
|
|
}
|
|
//--- else: history hasn't caught up yet this tick - leave it tracked and retry next tick
|
|
}
|
|
}
|
|
//--- forward-declared here, implemented in TradeJournalReport.mqh (kept separate - this file is
|
|
//--- the live tracking path, that one is the offline reporting/insights path; no reason for the
|
|
//--- per-tick code to pull in report-building logic it never calls).
|
|
bool GenerateReport(string &resultPath, string &errorMsg);
|
|
};
|
|
//--- CTradeJournalManager::GenerateReport() - split out, see that file's own header comment.
|
|
#include "TradeJournalReport.mqh"
|