forked from animatedread/Warrior_EA
f64e0f8b67
User design (2026-08-19): 'remove the enum menu that selects neural networks... individual inputs for every NN just like classic signals... the META NN should be integrated into the voting decision pipeline when enabled... as a bonus meta labelling is applied to enabled NNs.' - AI_CHOICE is GONE (tombstoned per the stale-.set doctrine). Use_MLP/Use_CONV/Use_LSTM/ Use_CONVLSTM are ordinary bools like the classic votes; the ensemble arithmetic adapts to any subset because the consensus divisor is the enabled capable weight. Two or more enabled = ensemble (|ENS1 token + joint gate, exactly the old AI_HYBRID fingerprints, so existing weight files keep loading); one = the old solo preset; none = classic-only. - Use_MetaLabeling un-couples META from the direction NNs (the old selector made them mutually exclusive). S3 ships: CSignalMETA::LiveMetaGate scores each vote-cleared entry (shared window at bar 1 + proposal descriptor: side, net vote, live geometry, spread/ATR; pattern one-hot ZEROED - ranking, not calibrated probability, documented in the body) and vetoes below the cost-adjusted break-even. Entries only; fail-open everywhere, loudly. - COEXISTENCE HAZARDS closed: VoteCapableWeight()=0 and ProspectiveVote()=false for the meta target - solo-only until today, a trained META would otherwise sit in the consensus divisor as a permanent abstainer and shrink every vote by its module weight. - CERTIFIED == TRADED: the ensemble era verdict replays the identical veto through the same g_warriorMetaGate pointer over its OOS fired bars (bar re-resolved from the row's own time; fail-open counted as fires and reported: 'metaGate: N approved, M vetoed, K unscored'). The overlay deliberately does NOT replay it (veto-filter-in-replay class, calendar-cliff precedent) - documented at the sweep site. Solo charts' own gate does not model the veto - the standing solo-gate caveat, documented at the input. - DB continuity: the pattern/journal DB fingerprint's first slot was (int)AIType; DbLegacyAiSlot() maps every legacy-expressible config to its OLD value (new 2-3 member subsets get 100+bitmask, outside the legacy range) so no existing database re-keys. filterID becomes the enabled roster via one EnabledNNSummary(). - HUD: the meta line shows the gate (armed/(trn), last P vs BE, ok/veto tally); the armed/disarmed announcement fires on state change via one latch (MetaGateArmedNow), not only when an entry happens to be proposed. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
358 lines
18 KiB
MQL5
358 lines
18 KiB
MQL5
//+------------------------------------------------------------------+
|
|
//| TradeJournalManager.mqh |
|
|
//| AnimateDread |
|
|
//| https://www.mql5.com |
|
|
//+------------------------------------------------------------------+
|
|
#property copyright "AnimateDread"
|
|
#property link "https://www.mql5.com"
|
|
#include "DatabaseManager.mqh"
|
|
#include "..\Variables\ConfidenceBridge.mqh"
|
|
//--- g_riskBudget, fed one result per closed position so the expectancy rule has a sample. Include-
|
|
//--- guarded, and this file is pulled in before Money\ and Signals\ pull the same header, so the
|
|
//--- single global is defined exactly once wherever the include order lands.
|
|
#include "..\Variables\RiskBudget.mqh"
|
|
//+------------------------------------------------------------------+
|
|
//| One closed trade, as persisted to/read from the TradeJournal |
|
|
//| table. Field ORDER matters - DatabaseReadBind()/InsertTradeRecord |
|
|
//| match it positionally against the table's own column order |
|
|
//| (TradeJournalSchema below), not by name. |
|
|
//+------------------------------------------------------------------+
|
|
struct STradeJournalRecord
|
|
{
|
|
long ticket;
|
|
int openYear, openMonth, openDay, openDayOfWeek, openHour, openMinute;
|
|
int closeYear, closeMonth, closeDay, closeHour, closeMinute;
|
|
string symbol;
|
|
string direction;
|
|
double entryPrice, exitPrice, slPrice, tpPrice, lots;
|
|
double profit; // real net P&L: deal profit + swap + commission
|
|
double riskDistance; // |entryPrice-slPrice| at open; 0 if no SL was set
|
|
double rMultiple; // realized price move / riskDistance; 0 if riskDistance is 0
|
|
double maePoints, mfePoints; // worst adverse / best favorable excursion, price units, >=0
|
|
double maeR, mfeR; // same, normalized by riskDistance; 0 if riskDistance is 0
|
|
double aiConfidence, dbConfidence; // 0..1, snapshotted at entry
|
|
string exitReason; // SL/TP/Expert/Manual/StopOut/Other - from the closing deal's DEAL_REASON
|
|
string filterID; // which engine was driving trades this run (enabled-NN roster, or "Classic")
|
|
};
|
|
//--- column order matches STradeJournalRecord's field order exactly (see struct comment above)
|
|
const string TradeJournalSchema =
|
|
"ticket INTEGER, "
|
|
"openYear INTEGER, openMonth INTEGER, openDay INTEGER, openDayOfWeek INTEGER, openHour INTEGER, openMinute INTEGER, "
|
|
"closeYear INTEGER, closeMonth INTEGER, closeDay INTEGER, closeHour INTEGER, closeMinute INTEGER, "
|
|
"symbol TEXT, direction TEXT, "
|
|
"entryPrice REAL, exitPrice REAL, slPrice REAL, tpPrice REAL, lots REAL, "
|
|
"profit REAL, riskDistance REAL, rMultiple REAL, "
|
|
"maePoints REAL, mfePoints REAL, maeR REAL, mfeR REAL, "
|
|
"aiConfidence REAL, dbConfidence REAL, "
|
|
"exitReason TEXT, filterID TEXT";
|
|
//+------------------------------------------------------------------+
|
|
//| In-memory tracking for a still-open position - MAE/MFE can only |
|
|
//| be measured live, tick by tick, while the position exists; there |
|
|
//| is no post-hoc MQL5 API to recover it once the position is gone. |
|
|
//+------------------------------------------------------------------+
|
|
struct SJournalOpenTrack
|
|
{
|
|
ulong ticket;
|
|
datetime openTime;
|
|
string symbol;
|
|
string direction;
|
|
double entryPrice, slPrice, tpPrice, lots;
|
|
double riskDistance;
|
|
double aiConfidence, dbConfidence;
|
|
string filterID;
|
|
double maePoints;
|
|
double mfePoints;
|
|
};
|
|
//+------------------------------------------------------------------+
|
|
//| Owns the TradeJournal table: detects this EA's own positions |
|
|
//| opening/closing (by polling PositionsTotal() every tick rather |
|
|
//| than hooking OnTradeTransaction - simpler and robust against |
|
|
//| partial fills/multiple deals per position), tracks MAE/MFE live |
|
|
//| while a position is open, and resolves the real closing P&L/ |
|
|
//| reason from deal history (HistoryDealGetInteger(DEAL_REASON) - |
|
|
//| broker-confirmed, not a heuristic) once it closes. |
|
|
//+------------------------------------------------------------------+
|
|
class CTradeJournalManager
|
|
{
|
|
private:
|
|
CDatabaseManager *m_dbm;
|
|
ulong m_magic;
|
|
string m_tableName;
|
|
SJournalOpenTrack m_tracked[];
|
|
|
|
int FindTracked(ulong ticket)
|
|
{
|
|
for(int i = 0; i < ArraySize(m_tracked); i++)
|
|
if(m_tracked[i].ticket == ticket)
|
|
return i;
|
|
return -1;
|
|
}
|
|
void RemoveTracked(int idx)
|
|
{
|
|
int last = ArraySize(m_tracked) - 1;
|
|
if(idx < 0 || idx > last)
|
|
return;
|
|
if(idx != last)
|
|
m_tracked[idx] = m_tracked[last];
|
|
ArrayResize(m_tracked, last);
|
|
}
|
|
//--- Since 2026-08-19 (per-NN toggles) the run label is the enabled roster, not an enum name:
|
|
//--- "MLP+LSTM", "MLP+CONV+LSTM+CONVLSTM+metaGate", "Classic". Legacy rows keep their old
|
|
//--- AI_MLP/AI_HYBRID/... labels; the column is a free-text run descriptor, nothing keys off it.
|
|
string CurrentFilterID(void)
|
|
{
|
|
return EnabledNNSummary();
|
|
}
|
|
string ExitReasonFromDealReason(long reason)
|
|
{
|
|
switch((ENUM_DEAL_REASON)reason)
|
|
{
|
|
case DEAL_REASON_SL: return "SL";
|
|
case DEAL_REASON_TP: return "TP";
|
|
case DEAL_REASON_EXPERT: return "Expert";
|
|
case DEAL_REASON_CLIENT:
|
|
case DEAL_REASON_MOBILE:
|
|
case DEAL_REASON_WEB: return "Manual";
|
|
case DEAL_REASON_SO: return "StopOut";
|
|
default: return "Other";
|
|
}
|
|
}
|
|
//--- resolves the closing deal for a position no longer in PositionsTotal() - returns false if
|
|
//--- history hasn't caught up yet (rare timing edge case); caller keeps tracking it and retries
|
|
//--- next tick rather than dropping the trade unrecorded.
|
|
//--- sums profit across every OUT/INOUT deal for this position (covers a partial close followed by
|
|
//--- a final close, however rare) rather than trusting a single deal to represent the whole
|
|
//--- position; exitPrice/exitReason are taken from the LAST (most recent) such deal, representing
|
|
//--- how the position ultimately finished.
|
|
//--- Money that was at risk on this trade, in account currency: the entry-to-stop distance converted
|
|
//--- through the symbol's own tick value, which is what the sizing used in the first place. Returns 0
|
|
//--- when the trade carried no stop or the symbol's tick data is unavailable, and the caller then
|
|
//--- simply does not score it - a trade with no stop has no R and guessing one would corrupt the mean.
|
|
double RiskAmountOf(const SJournalOpenTrack &t)
|
|
{
|
|
if(t.riskDistance <= 0.0 || t.lots <= 0.0)
|
|
return 0.0;
|
|
double tickSize = SymbolInfoDouble(t.symbol, SYMBOL_TRADE_TICK_SIZE);
|
|
double tickValue = SymbolInfoDouble(t.symbol, SYMBOL_TRADE_TICK_VALUE);
|
|
if(tickSize <= 0.0 || tickValue <= 0.0)
|
|
return 0.0;
|
|
return (t.riskDistance / tickSize) * tickValue * t.lots;
|
|
}
|
|
bool ResolveClose(ulong ticket, double &exitPrice, double &profit, string &exitReason)
|
|
{
|
|
if(!HistorySelectByPosition((long)ticket))
|
|
return false;
|
|
int deals = HistoryDealsTotal();
|
|
bool found = false;
|
|
profit = 0.0;
|
|
for(int d = 0; d < deals; d++)
|
|
{
|
|
ulong dealTicket = HistoryDealGetTicket(d);
|
|
if(dealTicket == 0)
|
|
continue;
|
|
long entry = HistoryDealGetInteger(dealTicket, DEAL_ENTRY);
|
|
if(entry != DEAL_ENTRY_OUT && entry != DEAL_ENTRY_INOUT)
|
|
continue;
|
|
profit += HistoryDealGetDouble(dealTicket, DEAL_PROFIT) +
|
|
HistoryDealGetDouble(dealTicket, DEAL_SWAP) +
|
|
HistoryDealGetDouble(dealTicket, DEAL_COMMISSION);
|
|
exitPrice = HistoryDealGetDouble(dealTicket, DEAL_PRICE);
|
|
exitReason = ExitReasonFromDealReason(HistoryDealGetInteger(dealTicket, DEAL_REASON));
|
|
found = true;
|
|
}
|
|
return found;
|
|
}
|
|
bool InsertClosedTrade(const SJournalOpenTrack &t, double exitPrice, double profit, string exitReason)
|
|
{
|
|
MqlDateTime openT, closeT;
|
|
TimeToStruct(t.openTime, openT);
|
|
TimeToStruct(TimeCurrent(), closeT);
|
|
double move = (t.direction == "Buy") ? (exitPrice - t.entryPrice) : (t.entryPrice - exitPrice);
|
|
double riskDistance = t.riskDistance;
|
|
double rMultiple = (riskDistance > 0.0) ? move / riskDistance : 0.0;
|
|
double maeR = (riskDistance > 0.0) ? t.maePoints / riskDistance : 0.0;
|
|
double mfeR = (riskDistance > 0.0) ? t.mfePoints / riskDistance : 0.0;
|
|
string cols[] = {"ticket", "openYear", "openMonth", "openDay", "openDayOfWeek", "openHour", "openMinute",
|
|
"closeYear", "closeMonth", "closeDay", "closeHour", "closeMinute",
|
|
"symbol", "direction", "entryPrice", "exitPrice", "slPrice", "tpPrice", "lots",
|
|
"profit", "riskDistance", "rMultiple", "maePoints", "mfePoints", "maeR", "mfeR",
|
|
"aiConfidence", "dbConfidence", "exitReason", "filterID"
|
|
};
|
|
string vals[];
|
|
ArrayResize(vals, ArraySize(cols));
|
|
int i = 0;
|
|
vals[i++] = IntegerToString((long)t.ticket);
|
|
vals[i++] = IntegerToString(openT.year);
|
|
vals[i++] = IntegerToString(openT.mon);
|
|
vals[i++] = IntegerToString(openT.day);
|
|
vals[i++] = IntegerToString(openT.day_of_week);
|
|
vals[i++] = IntegerToString(openT.hour);
|
|
vals[i++] = IntegerToString(openT.min);
|
|
vals[i++] = IntegerToString(closeT.year);
|
|
vals[i++] = IntegerToString(closeT.mon);
|
|
vals[i++] = IntegerToString(closeT.day);
|
|
vals[i++] = IntegerToString(closeT.hour);
|
|
vals[i++] = IntegerToString(closeT.min);
|
|
vals[i++] = t.symbol;
|
|
vals[i++] = t.direction;
|
|
vals[i++] = DoubleToString(t.entryPrice, 8);
|
|
vals[i++] = DoubleToString(exitPrice, 8);
|
|
vals[i++] = DoubleToString(t.slPrice, 8);
|
|
vals[i++] = DoubleToString(t.tpPrice, 8);
|
|
vals[i++] = DoubleToString(t.lots, 2);
|
|
vals[i++] = DoubleToString(profit, 2);
|
|
vals[i++] = DoubleToString(riskDistance, 8);
|
|
vals[i++] = DoubleToString(rMultiple, 4);
|
|
vals[i++] = DoubleToString(t.maePoints, 8);
|
|
vals[i++] = DoubleToString(t.mfePoints, 8);
|
|
vals[i++] = DoubleToString(maeR, 4);
|
|
vals[i++] = DoubleToString(mfeR, 4);
|
|
vals[i++] = DoubleToString(t.aiConfidence, 4);
|
|
vals[i++] = DoubleToString(t.dbConfidence, 4);
|
|
vals[i++] = exitReason;
|
|
vals[i++] = t.filterID;
|
|
if(!m_dbm.BeginTransaction())
|
|
return false;
|
|
bool ok = m_dbm.InsertTradeRecord(m_tableName, cols, vals);
|
|
if(!m_dbm.CommitTransaction())
|
|
ok = false;
|
|
return ok;
|
|
}
|
|
|
|
public:
|
|
CTradeJournalManager(void) : m_dbm(NULL), m_magic(0), m_tableName("TradeJournal") {}
|
|
bool Init(CDatabaseManager *dbmPtr, ulong magic)
|
|
{
|
|
m_dbm = dbmPtr;
|
|
m_magic = magic;
|
|
if(CheckPointer(m_dbm) == POINTER_INVALID)
|
|
return false;
|
|
//--- CreateTable() needs a genuinely open handle - dbm.Init() only stores the path, it doesn't
|
|
//--- open it (OpenDatabase() does that, and Warrior_EA.mq5 doesn't call it until well after
|
|
//--- filters/patterns are registered). Opening explicitly here means this also has to run
|
|
//--- before AddFilterToSignal() - see the call site in Warrior_EA.mq5's OnInit().
|
|
if(!m_dbm.OpenDatabase())
|
|
return false;
|
|
return m_dbm.CreateTable(m_tableName, TradeJournalSchema);
|
|
}
|
|
//--- Tracking without a database. The expectancy stop (g_riskBudget.RecordTradeResult) is fed from
|
|
//--- this class's close detection, and until 2026-08-11 that feed only existed when
|
|
//--- UseDatabaseRanking was on - which ships FALSE, so the da54639 "EV = minus the cost" halt was
|
|
//--- armed (ExpectancyMinTrades=40) and never received a single closed trade on a default install.
|
|
//--- A risk rule must not be a side effect of an optional analytics toggle: this init gives the
|
|
//--- close-detection/MAE-MFE/expectancy path a life of its own, and Update() below simply skips the
|
|
//--- DB insert when there is no DB.
|
|
void InitTrackingOnly(ulong magic)
|
|
{
|
|
m_dbm = NULL;
|
|
m_magic = magic;
|
|
}
|
|
//--- call once per tick: detects this EA's own positions opening/closing on the current symbol and
|
|
//--- updates MAE/MFE for every one still open. Cheap - PositionsTotal() is typically single digits.
|
|
//--- Runs with or without a database (see InitTrackingOnly); only the journal INSERT needs one.
|
|
void Update(void)
|
|
{
|
|
int preExistingCount = ArraySize(m_tracked);
|
|
bool seen[];
|
|
ArrayResize(seen, preExistingCount);
|
|
for(int i = 0; i < preExistingCount; i++)
|
|
seen[i] = false;
|
|
int total = PositionsTotal();
|
|
for(int p = 0; p < total; p++)
|
|
{
|
|
ulong ticket = PositionGetTicket(p);
|
|
if(ticket == 0)
|
|
continue;
|
|
if(!PositionSelectByTicket(ticket))
|
|
continue;
|
|
if(PositionGetString(POSITION_SYMBOL) != _Symbol)
|
|
continue;
|
|
if((ulong)PositionGetInteger(POSITION_MAGIC) != m_magic)
|
|
continue;
|
|
double currentPrice = PositionGetDouble(POSITION_PRICE_CURRENT);
|
|
int idx = FindTracked(ticket);
|
|
if(idx < 0)
|
|
{
|
|
SJournalOpenTrack t;
|
|
t.ticket = ticket;
|
|
t.openTime = (datetime)PositionGetInteger(POSITION_TIME);
|
|
t.symbol = _Symbol;
|
|
t.direction = (PositionGetInteger(POSITION_TYPE) == POSITION_TYPE_BUY) ? "Buy" : "Sell";
|
|
t.entryPrice = PositionGetDouble(POSITION_PRICE_OPEN);
|
|
t.slPrice = PositionGetDouble(POSITION_SL);
|
|
t.tpPrice = PositionGetDouble(POSITION_TP);
|
|
t.lots = PositionGetDouble(POSITION_VOLUME);
|
|
t.riskDistance = (t.slPrice > 0.0) ? MathAbs(t.entryPrice - t.slPrice) : 0.0;
|
|
//--- same-tick snapshot OpenParams() populated right before this trade was sent - see
|
|
//--- ConfidenceBridge.mqh's declaration comments.
|
|
t.aiConfidence = MathAbs(g_AISignedConfidence);
|
|
t.dbConfidence = g_DBConfidence;
|
|
t.filterID = CurrentFilterID();
|
|
t.maePoints = 0.0;
|
|
t.mfePoints = 0.0;
|
|
int newIdx = ArraySize(m_tracked);
|
|
ArrayResize(m_tracked, newIdx + 1);
|
|
m_tracked[newIdx] = t;
|
|
}
|
|
else
|
|
{
|
|
if(idx < preExistingCount)
|
|
seen[idx] = true;
|
|
double excursion = (m_tracked[idx].direction == "Buy") ?
|
|
(currentPrice - m_tracked[idx].entryPrice) :
|
|
(m_tracked[idx].entryPrice - currentPrice);
|
|
if(excursion > m_tracked[idx].mfePoints)
|
|
m_tracked[idx].mfePoints = excursion;
|
|
if(-excursion > m_tracked[idx].maePoints)
|
|
m_tracked[idx].maePoints = -excursion;
|
|
}
|
|
}
|
|
//--- anything tracked before this pass but not seen in it closed since the last tick - resolve
|
|
//--- and record it. Walk backwards since RemoveTracked() swap-removes (changes indices >= idx).
|
|
for(int i = preExistingCount - 1; i >= 0; i--)
|
|
{
|
|
if(seen[i])
|
|
continue;
|
|
double exitPrice = 0.0, profit = 0.0;
|
|
string exitReason = "Other";
|
|
if(ResolveClose(m_tracked[i].ticket, exitPrice, profit, exitReason))
|
|
{
|
|
// MAE/MFE and the rest of this closed trade's record cannot be reconstructed after the
|
|
// fact once m_tracked[i] is removed below - if the DB insert fails (lock contention on
|
|
// the shared COMMON db, disk issue, retry exhaustion), log every field so the record is
|
|
// at least manually recoverable from the Experts journal instead of silently vanishing.
|
|
//--- FEED THE EXPECTANCY RULE. Reported in R - net profit over the money that was actually
|
|
//--- at risk - so results from different symbols, lot sizes and account balances are on one
|
|
//--- scale and can share a single mean. riskDistance is the entry-to-stop distance the trade
|
|
//--- was sized against, so lots * riskDistance * tickValue-per-point IS the amount at risk;
|
|
//--- using the account's own currency conversion via the profit figure keeps it exact rather
|
|
//--- than reconstructing tick values here.
|
|
//--- `profit` already includes swap and commission (see ResolveClose). That is deliberate and
|
|
//--- load-bearing: when the directional edge is zero, cost is the ENTIRE expectancy, so a
|
|
//--- gross-profit version of this rule would measure a strategy nobody can actually trade.
|
|
double riskAmount = RiskAmountOf(m_tracked[i]);
|
|
if(riskAmount > 0.0)
|
|
g_riskBudget.RecordTradeResult(profit / riskAmount);
|
|
if(CheckPointer(m_dbm) == POINTER_INVALID)
|
|
{
|
|
//--- tracking-only mode (no UseDatabaseRanking): the expectancy rule above is the
|
|
//--- whole point; there is no journal DB to insert into.
|
|
RemoveTracked(i);
|
|
continue;
|
|
}
|
|
if(!InsertClosedTrade(m_tracked[i], exitPrice, profit, exitReason))
|
|
PrintFormat("%s: ERROR - failed to insert closed trade into journal DB, record LOST from DB (recoverable from this log only): ticket=%I64u direction=%s entry=%.5f exit=%.5f profit=%.2f mae=%.5f mfe=%.5f reason=%s",
|
|
__FUNCTION__, m_tracked[i].ticket, m_tracked[i].direction, m_tracked[i].entryPrice, exitPrice, profit,
|
|
m_tracked[i].maePoints, m_tracked[i].mfePoints, exitReason);
|
|
RemoveTracked(i);
|
|
}
|
|
//--- else: history hasn't caught up yet this tick - leave it tracked and retry next tick
|
|
}
|
|
}
|
|
//--- forward-declared here, implemented in TradeJournalReport.mqh (kept separate - this file is
|
|
//--- the live tracking path, that one is the offline reporting/insights path; no reason for the
|
|
//--- per-tick code to pull in report-building logic it never calls).
|
|
bool GenerateReport(string &resultPath, string &errorMsg);
|
|
};
|
|
//--- CTradeJournalManager::GenerateReport() - split out, see that file's own header comment.
|
|
#include "TradeJournalReport.mqh"
|