forked from animatedread/Warrior_EA
f64e0f8b67
User design (2026-08-19): 'remove the enum menu that selects neural networks... individual inputs for every NN just like classic signals... the META NN should be integrated into the voting decision pipeline when enabled... as a bonus meta labelling is applied to enabled NNs.' - AI_CHOICE is GONE (tombstoned per the stale-.set doctrine). Use_MLP/Use_CONV/Use_LSTM/ Use_CONVLSTM are ordinary bools like the classic votes; the ensemble arithmetic adapts to any subset because the consensus divisor is the enabled capable weight. Two or more enabled = ensemble (|ENS1 token + joint gate, exactly the old AI_HYBRID fingerprints, so existing weight files keep loading); one = the old solo preset; none = classic-only. - Use_MetaLabeling un-couples META from the direction NNs (the old selector made them mutually exclusive). S3 ships: CSignalMETA::LiveMetaGate scores each vote-cleared entry (shared window at bar 1 + proposal descriptor: side, net vote, live geometry, spread/ATR; pattern one-hot ZEROED - ranking, not calibrated probability, documented in the body) and vetoes below the cost-adjusted break-even. Entries only; fail-open everywhere, loudly. - COEXISTENCE HAZARDS closed: VoteCapableWeight()=0 and ProspectiveVote()=false for the meta target - solo-only until today, a trained META would otherwise sit in the consensus divisor as a permanent abstainer and shrink every vote by its module weight. - CERTIFIED == TRADED: the ensemble era verdict replays the identical veto through the same g_warriorMetaGate pointer over its OOS fired bars (bar re-resolved from the row's own time; fail-open counted as fires and reported: 'metaGate: N approved, M vetoed, K unscored'). The overlay deliberately does NOT replay it (veto-filter-in-replay class, calendar-cliff precedent) - documented at the sweep site. Solo charts' own gate does not model the veto - the standing solo-gate caveat, documented at the input. - DB continuity: the pattern/journal DB fingerprint's first slot was (int)AIType; DbLegacyAiSlot() maps every legacy-expressible config to its OLD value (new 2-3 member subsets get 100+bitmask, outside the legacy range) so no existing database re-keys. filterID becomes the enabled roster via one EnabledNNSummary(). - HUD: the meta line shows the gate (armed/(trn), last P vs BE, ok/veto tally); the armed/disarmed announcement fires on state change via one latch (MetaGateArmedNow), not only when an entry happens to be proposed. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
731 lines
36 KiB
MQL5
731 lines
36 KiB
MQL5
//+------------------------------------------------------------------+
|
|
//| Warrior_EA |
|
|
//| AnimateDread |
|
|
//| |
|
|
//+------------------------------------------------------------------+
|
|
#include "..\Expert\ExpertSignalAIBase.mqh"
|
|
#include "..\Expert\AIBase\MetaCorpus.mqh"
|
|
// wizard description start
|
|
//+------------------------------------------------------------------+
|
|
//| Description of the class |
|
|
//| Title=Signals of indicator 'Meta AI' |
|
|
//| Type=SignalAdvanced |
|
|
//| Name=Meta AI |
|
|
//| ShortName=META |
|
|
//| Class=CSignalMETA |
|
|
//| Page=signal_meta |
|
|
//+------------------------------------------------------------------+
|
|
// wizard description end
|
|
//+------------------------------------------------------------------+
|
|
//| Class CSignalMETA - stage S2 of Meta_Labeling_Design.md. |
|
|
//| |
|
|
//| The meta-labeling head: instead of asking "which way will the |
|
|
//| next bar go" (measured dead - direction-closed verdict), it asks |
|
|
//| "given that a SPECIFIC classic-pattern candidate just fired, |
|
|
//| will THAT trade reach its target before its stop, at the EA's |
|
|
//| own geometry, net of cost". One net for all 52 pattern-sides; |
|
|
//| pattern identity rides in as input features. |
|
|
//| |
|
|
//| Sample: the signal-DB corpus (built by an 18-year backtest |
|
|
//| with UseDatabaseRanking on - per-side journaling means |
|
|
//| the DB IS the candidate stream, uncensored). |
|
|
//| Label: triple-barrier win/loss of the candidate's own side |
|
|
//| from its fire bar - the side-conditional win caches |
|
|
//| the label prebuild already computes; the DB's stop- |
|
|
//| and-reverse outcome is NEVER reused as a label. |
|
|
//| Features: the shared BuildFeatureWindow() output plus a 31-wide |
|
|
//| setup descriptor appended at the input (26-slot |
|
|
//| pattern one-hot, side, tanh-squashed netVote, SL/TP |
|
|
//| in ATR, spread/ATR at fire time). Appended at the |
|
|
//| input rather than "at the head" because CNet has no |
|
|
//| concat layer; on the MLP front end the two are |
|
|
//| equivalent up to one linear layer. |
|
|
//| Head: 2 outputs, softmax+CE (== logistic/BCE); see the |
|
|
//| total==2 branches in AI\Impl\NetForward.mqh. |
|
|
//| Front end: MLP only in S2 (AddCustomLayers no-op inherited). |
|
|
//| Conv/LSTM meta variants would need the descriptor |
|
|
//| padded to whole pseudo-bars to keep their bar-major |
|
|
//| window/step geometry - deliberately out of S2 scope. |
|
|
//| |
|
|
//| S2 trains and reports (coverage x (win rate - break-even) vs the |
|
|
//| base-rate null, in Training.mqh's era-end META line). It casts |
|
|
//| NO votes: dPrevSignal never leaves its sentinel, so the base |
|
|
//| LongCondition/ShortCondition return 0. S3 (2026-08-19) is the |
|
|
//| LIVE GATE below: LiveMetaGate() scores each vote-cleared entry |
|
|
//| and vetoes those whose P(win) sits under the cost-adjusted |
|
|
//| break-even. It still casts no votes and cannot dilute the |
|
|
//| consensus (VoteCapableWeight/ProspectiveVote are 0/false for the |
|
|
//| meta target). |
|
|
//+------------------------------------------------------------------+
|
|
|
|
//--- Setup descriptor layout (AppendCandidateFeatures): 26 one-hot + side + netVote + SL + TP + spread/ATR.
|
|
//--- MetaDescWidth() returns this and the input layer is sized with it - the three MUST stay in step.
|
|
#define META_ONE_HOT_SLOTS 26
|
|
#define META_DESC_FEATURES (META_ONE_HOT_SLOTS + 5)
|
|
|
|
class CSignalMETA : public CExpertSignalAIBase
|
|
{
|
|
protected:
|
|
//--- The corpus: every journaled pattern instance, loaded ONCE per attach from the largest signal
|
|
//--- DB on disk (see LoadMetaCorpus for why largest-by-rows rather than the chart's own config
|
|
//--- fingerprint). GMT times are fixed; bar INDICES are re-resolved every era (they shift).
|
|
datetime m_corpusGmt[];
|
|
char m_corpusSide[];
|
|
double m_corpusNetVote[];
|
|
short m_corpusFamily[];
|
|
short m_corpusPattern[];
|
|
double m_corpusEntry[]; // touchable price at fire time - the offset oracle
|
|
int m_corpusCount;
|
|
bool m_corpusLoaded;
|
|
bool m_prepareReported; // first-era diagnostics print loudly, later eras verbose
|
|
bool m_datasetExported; // one export per attach (Meta_ExportDataset input)
|
|
//--- ON-CHART CANDIDATE SOURCES (the classic filters attached to this same chart). When present,
|
|
//--- the corpus is generated by SWEEPING these real ladders over the chart's own history via the
|
|
//--- StartIndex/EvalShift mechanism - no tester corpus run, no DB dependency, no GMT-offset
|
|
//--- resolution ambiguity (the sweep IS on this chart's bars). The DB loader stays as fallback.
|
|
CExpertSignalCustom *m_srcFilter[4];
|
|
int m_srcFamily[4];
|
|
int m_srcCount;
|
|
|
|
public:
|
|
void AddCandidateSource(CExpertSignalCustom *filter, const int family)
|
|
{
|
|
if(m_srcCount < 4 && CheckPointer(filter) != POINTER_INVALID)
|
|
{
|
|
m_srcFilter[m_srcCount] = filter;
|
|
m_srcFamily[m_srcCount] = family;
|
|
m_srcCount++;
|
|
}
|
|
}
|
|
CSignalMETA(void);
|
|
virtual bool InitIndicators(CIndicators *indicators) override;
|
|
virtual int MetaDescWidth(void) const override { return META_DESC_FEATURES; }
|
|
virtual bool MetaPrepareEra(const int bars) override;
|
|
virtual void AppendCandidateFeatures(const int candId) override;
|
|
//--- S3 (2026-08-19): the LIVE meta gate - contract in the base seam's comment
|
|
//--- (Expert\ExpertSignalCustom.mqh), honesty notes in the body's header below.
|
|
virtual int LiveMetaGate(const bool isLong, const double netVote, double &pWin,
|
|
double &bePct, const int barIdx) override;
|
|
|
|
protected:
|
|
bool LoadMetaCorpus(void);
|
|
bool BuildCorpusBySweep(void);
|
|
long CountDbPatternRows(const int db);
|
|
void ExportMetaDataset(void);
|
|
//--- Gate-time twin of AppendCandidateFeatures - same width, same slot order, same squash.
|
|
void AppendLiveDescriptor(const char side, const double netVote, const int barIdx);
|
|
int OneHotSlot(const int family, const int pattern) const
|
|
{
|
|
//--- MA 0-3, RSI 4-7, MACD 8-13, Ichimoku 14-25 - matches MetaFamilyPatterns' 4/4/6/12
|
|
int base = -1;
|
|
switch(family)
|
|
{
|
|
case 0: base = 0; break;
|
|
case 1: base = 4; break;
|
|
case 2: base = 8; break;
|
|
case 3: base = 14; break;
|
|
}
|
|
if(base < 0)
|
|
return -1;
|
|
int slot = base + pattern;
|
|
return (slot >= 0 && slot < META_ONE_HOT_SLOTS) ? slot : -1;
|
|
}
|
|
};
|
|
//+------------------------------------------------------------------+
|
|
//| Constructor |
|
|
//+------------------------------------------------------------------+
|
|
CSignalMETA::CSignalMETA(void) : m_corpusCount(0), m_corpusLoaded(false), m_prepareReported(false),
|
|
m_datasetExported(false), m_srcCount(0)
|
|
{
|
|
SetIdentity("Meta", "META");
|
|
//--- identity-defining, set once (feeds the |TGT:META fingerprint token and every IsMetaTarget seam)
|
|
m_trainTarget = 1;
|
|
for(int k = 0; k < 4; k++)
|
|
{
|
|
m_srcFilter[k] = NULL;
|
|
m_srcFamily[k] = -1;
|
|
}
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Create indicators and bootstrap/load the network. |
|
|
//+------------------------------------------------------------------+
|
|
bool CSignalMETA::InitIndicators(CIndicators *indicators)
|
|
{
|
|
return InitNeuralNetwork(indicators);
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Total rows across the 52 pattern tables of an open DB handle. |
|
|
//+------------------------------------------------------------------+
|
|
long CSignalMETA::CountDbPatternRows(const int db)
|
|
{
|
|
long rows = 0;
|
|
for(int f = 0; f < 4; f++)
|
|
for(int p = 0; p < MetaFamilyPatterns(f); p++)
|
|
for(int s = 0; s < 2; s++)
|
|
{
|
|
string table = MetaFamilyName(f) + "_Pattern_" + IntegerToString(p) +
|
|
(s == 0 ? "_Buy" : "_Sell");
|
|
int stmt = DatabasePrepare(db, "SELECT COUNT(*) FROM " + table);
|
|
if(stmt == INVALID_HANDLE)
|
|
continue; // table absent (family disabled in the populating run)
|
|
long c = 0;
|
|
if(DatabaseRead(stmt))
|
|
DatabaseColumnLong(stmt, 0, c);
|
|
DatabaseFinalize(stmt);
|
|
rows += c;
|
|
}
|
|
return rows;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Load the candidate corpus from the LARGEST signal DB on disk. |
|
|
//| |
|
|
//| Deliberately NOT through dbm/the chart's own config fingerprint: |
|
|
//| the DB filename hashes the journaling inputs, so a training |
|
|
//| chart whose inputs differ by one journal setting from the |
|
|
//| corpus-building tester run would open a different (empty) file |
|
|
//| and silently train on nothing - the exact procedural trap that |
|
|
//| burned four corpus-build attempts. The corpus is DATA, not |
|
|
//| config identity; the biggest coherent set of candidates on disk |
|
|
//| is the right training set, and the pick is logged so the run is |
|
|
//| auditable. All files in the folder share one semantics era - |
|
|
//| the DatabaseVersion wipe guarantees it. |
|
|
//| |
|
|
//| Read-only open: this must never write, prune, or lock the DB |
|
|
//| the journaling side owns. |
|
|
//+------------------------------------------------------------------+
|
|
bool CSignalMETA::LoadMetaCorpus(void)
|
|
{
|
|
const string folder = "Warrior_EA\\Databases\\Signals\\";
|
|
//--- Only THIS symbol's + THIS timeframe's corpora are candidates. The DB filename is
|
|
//--- <symbol>_<period>_<fingerprint>.db, and "largest on disk" without this filter would happily
|
|
//--- hand an H4 chart the (bigger) H1 corpus - whose rows then resolve onto the wrong bars - or an
|
|
//--- SP500 chart another symbol's DB entirely. Candidates journaled on a different grid are not
|
|
//--- this chart's candidates.
|
|
const string mustPrefix = _Symbol + "_" + IntegerToString((int)_Period) + "_";
|
|
string bestFile = "";
|
|
long bestRows = 0;
|
|
string fname;
|
|
long find = FileFindFirst(folder + "*.db", fname, FILE_COMMON);
|
|
if(find != INVALID_HANDLE)
|
|
{
|
|
do
|
|
{
|
|
if(StringFind(fname, mustPrefix) != 0)
|
|
continue;
|
|
int db = DatabaseOpen(folder + fname, DATABASE_OPEN_READONLY | DATABASE_OPEN_COMMON);
|
|
if(db == INVALID_HANDLE)
|
|
continue;
|
|
long rows = CountDbPatternRows(db);
|
|
DatabaseClose(db);
|
|
if(rows > bestRows)
|
|
{
|
|
bestRows = rows;
|
|
bestFile = fname;
|
|
}
|
|
}
|
|
while(FileFindNext(find, fname));
|
|
FileFindClose(find);
|
|
}
|
|
if(bestFile == "" || bestRows <= 0)
|
|
{
|
|
Print(ID + ": META CORPUS UNAVAILABLE - no signal DB matching " + mustPrefix + "*.db with pattern"
|
|
" rows found under Common\\Files\\" + folder + ". Build one for THIS symbol+timeframe first:"
|
|
" wipe the Signals folder, then run a long backtest on this chart's symbol AND timeframe"
|
|
" with UseDatabaseRanking=true and DB_MaxRowsPerTable raised (see Meta_Labeling_Design.md S1).");
|
|
return false;
|
|
}
|
|
int db = DatabaseOpen(folder + bestFile, DATABASE_OPEN_READONLY | DATABASE_OPEN_COMMON);
|
|
if(db == INVALID_HANDLE)
|
|
{
|
|
Print(ID + ": failed to reopen corpus DB " + bestFile + " (error " +
|
|
IntegerToString(GetLastError()) + ")");
|
|
return false;
|
|
}
|
|
ArrayResize(m_corpusGmt, (int)bestRows);
|
|
ArrayResize(m_corpusSide, (int)bestRows);
|
|
ArrayResize(m_corpusNetVote, (int)bestRows);
|
|
ArrayResize(m_corpusFamily, (int)bestRows);
|
|
ArrayResize(m_corpusPattern, (int)bestRows);
|
|
ArrayResize(m_corpusEntry, (int)bestRows);
|
|
m_corpusCount = 0;
|
|
for(int f = 0; f < 4; f++)
|
|
for(int p = 0; p < MetaFamilyPatterns(f); p++)
|
|
for(int s = 0; s < 2; s++)
|
|
{
|
|
string table = MetaFamilyName(f) + "_Pattern_" + IntegerToString(p) +
|
|
(s == 0 ? "_Buy" : "_Sell");
|
|
int stmt = DatabasePrepare(db, "SELECT year, month, day, hour, minutes, netVote, entryPrice"
|
|
" FROM " + table);
|
|
if(stmt == INVALID_HANDLE)
|
|
continue;
|
|
while(DatabaseRead(stmt) && m_corpusCount < (int)bestRows)
|
|
{
|
|
long y = 0, mo = 0, d = 0, h = 0, mi = 0;
|
|
double nv = 0.0, ep = 0.0;
|
|
DatabaseColumnLong(stmt, 0, y);
|
|
DatabaseColumnLong(stmt, 1, mo);
|
|
DatabaseColumnLong(stmt, 2, d);
|
|
DatabaseColumnLong(stmt, 3, h);
|
|
DatabaseColumnLong(stmt, 4, mi);
|
|
DatabaseColumnDouble(stmt, 5, nv);
|
|
DatabaseColumnDouble(stmt, 6, ep);
|
|
MqlDateTime t;
|
|
t.year = (int)y;
|
|
t.mon = (int)mo;
|
|
t.day = (int)d;
|
|
t.hour = (int)h;
|
|
t.min = (int)mi;
|
|
t.sec = 0;
|
|
m_corpusGmt[m_corpusCount] = StructToTime(t);
|
|
m_corpusSide[m_corpusCount] = (char)(s == 0 ? 1 : -1);
|
|
m_corpusNetVote[m_corpusCount] = nv;
|
|
m_corpusFamily[m_corpusCount] = (short)f;
|
|
m_corpusPattern[m_corpusCount] = (short)p;
|
|
m_corpusEntry[m_corpusCount] = ep;
|
|
m_corpusCount++;
|
|
}
|
|
DatabaseFinalize(stmt);
|
|
}
|
|
DatabaseClose(db);
|
|
m_corpusLoaded = (m_corpusCount > 0);
|
|
Print(ID + StringFormat(": meta corpus loaded from %s - %d candidates across the pattern tables"
|
|
" (largest of the DBs found; corpus rows are GMT-stamped, resolution to"
|
|
" server bars happens per era).", bestFile, m_corpusCount));
|
|
return m_corpusLoaded;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Resolve the corpus onto this era's bar grid. |
|
|
//| |
|
|
//| Rows pre-dbVersion-4.0 are GMT, BROKER time since; history is |
|
|
//| server time either way (EET-ish, DST moves |
|
|
//| it). The offset is MEASURED PER ROW, not assumed: rows are |
|
|
//| journaled at the bar's opening tick with the touchable price, so |
|
|
//| the RIGHT offset's bar open matches entryPrice to within the |
|
|
//| spread while a wrong offset lands a full hourly move away. Each |
|
|
//| row takes the offset (0..+4h) minimizing |open - entry| over |
|
|
//| offsets that land on an exact bar, then must pass a tolerance - |
|
|
//| decisive per row, and immune to DST regime changes across an |
|
|
//| 18-year corpus (the histogram printed below shows the winter/ |
|
|
//| summer split directly). |
|
|
//| |
|
|
//| The window-span filter kills the pre-2017 daily-backfill regime |
|
|
//| (measured 2026-08-13: hour-0-only rows, one per day): a |
|
|
//| candidate whose m_historyBars-deep window spans more than 4x its |
|
|
//| nominal duration is sitting on bars that are not really H1, and |
|
|
//| its geometry/labels would be silently wrong. |
|
|
//+------------------------------------------------------------------+
|
|
//+------------------------------------------------------------------+
|
|
//| Generate the candidate corpus by sweeping the REAL classic |
|
|
//| ladders over this chart's own history. |
|
|
//| |
|
|
//| Every pattern condition anchors on StartIndex() (verified across |
|
|
//| all four signal files), so EvalShift(i) makes the exact live |
|
|
//| code answer "what would you have fired at bar i" - the silent- |
|
|
//| divergence trap that justified the DB corpus does not exist on |
|
|
//| this path, and neither do the tester run, the GMT-offset |
|
|
//| ambiguity, or the DB row caps. Times/prices stored are this |
|
|
//| chart's own bar opens, so MetaPrepareEra's resolution matches at |
|
|
//| offset +0 with zero price error by construction. |
|
|
//| |
|
|
//| One-time cost at first era: bars x sources Direction() calls - |
|
|
//| a few seconds. Runs on the chart thread like the label prebuild. |
|
|
//+------------------------------------------------------------------+
|
|
bool CSignalMETA::BuildCorpusBySweep(void)
|
|
{
|
|
if(m_srcCount <= 0)
|
|
return false;
|
|
ENUM_TIMEFRAMES per = (ENUM_TIMEFRAMES)m_period;
|
|
int bars = Bars(_Symbol, per);
|
|
if(bars <= 300)
|
|
return false;
|
|
for(int s = 0; s < m_srcCount; s++)
|
|
if(!m_srcFilter[s].SweepPrepare(bars))
|
|
{
|
|
Print(ID + ": candidate sweep - filter " + m_srcFilter[s].GetFilterID() +
|
|
" could not prepare deep buffers; falling back to a DB corpus.");
|
|
return false;
|
|
}
|
|
//--- skip the indicator warm-up tail at the oldest edge of history (reads there are EMPTY/garbage
|
|
//--- and would fabricate patterns); 150 bars comfortably covers every classic period in use.
|
|
int deepest = bars - 150;
|
|
//--- Initial reserve only - the loop below GROWS the arrays on demand. The worst case is
|
|
//--- m_srcCount*2 appends per bar (every family, both sides), and STATE-model patterns really
|
|
//--- do stay active on most bars, so a fixed bars*2 cap failed both ways: it silently
|
|
//--- truncated the corpus when the guard tripped at a bar boundary (SP500 filled to exactly
|
|
//--- cap, dropping the newest bars entirely), and it overflowed mid-bar on denser symbols
|
|
//--- because the old guard only reserved room for 2 appends (array-out-of-range right here
|
|
//--- on USDJPY/XAUUSD/XTIUSD, killing the chart at attach).
|
|
int cap = bars * 2;
|
|
ArrayResize(m_corpusGmt, cap);
|
|
ArrayResize(m_corpusSide, cap);
|
|
ArrayResize(m_corpusNetVote, cap);
|
|
ArrayResize(m_corpusFamily, cap);
|
|
ArrayResize(m_corpusPattern, cap);
|
|
ArrayResize(m_corpusEntry, cap);
|
|
m_corpusCount = 0;
|
|
Print(ID + StringFormat(": sweeping %d classic ladder(s) over %d bars for candidates - the chart"
|
|
" is busy for a few seconds...", m_srcCount, deepest));
|
|
uint t0 = GetTickCount();
|
|
for(int i = deepest; i >= 2; i--)
|
|
{
|
|
if(m_corpusCount + m_srcCount * 2 > cap)
|
|
{
|
|
cap += cap / 2 + m_srcCount * 2;
|
|
ArrayResize(m_corpusGmt, cap);
|
|
ArrayResize(m_corpusSide, cap);
|
|
ArrayResize(m_corpusNetVote, cap);
|
|
ArrayResize(m_corpusFamily, cap);
|
|
ArrayResize(m_corpusPattern, cap);
|
|
ArrayResize(m_corpusEntry, cap);
|
|
}
|
|
datetime bt = iTime(_Symbol, per, i);
|
|
double bo = iOpen(_Symbol, per, i);
|
|
if(bt <= 0 || bo <= 0.0)
|
|
continue;
|
|
for(int s = 0; s < m_srcCount; s++)
|
|
{
|
|
CExpertSignalCustom *f = m_srcFilter[s];
|
|
f.EvalShift(i);
|
|
f.Direction();
|
|
f.EvalShift(0);
|
|
string pl = f.GetActivePatternLong();
|
|
string ps = f.GetActivePatternShort();
|
|
double nv = f.LastNetVote();
|
|
//--- "Pattern_N" -> N; same per-side, one-candidate-per-bar semantics as live journaling
|
|
if(pl != "NULL")
|
|
{
|
|
m_corpusGmt[m_corpusCount] = bt;
|
|
m_corpusEntry[m_corpusCount] = bo;
|
|
m_corpusSide[m_corpusCount] = 1;
|
|
m_corpusFamily[m_corpusCount] = (short)m_srcFamily[s];
|
|
m_corpusPattern[m_corpusCount] = (short)StringToInteger(StringSubstr(pl, 8));
|
|
m_corpusNetVote[m_corpusCount] = nv;
|
|
m_corpusCount++;
|
|
}
|
|
if(ps != "NULL")
|
|
{
|
|
m_corpusGmt[m_corpusCount] = bt;
|
|
m_corpusEntry[m_corpusCount] = bo;
|
|
m_corpusSide[m_corpusCount] = -1;
|
|
m_corpusFamily[m_corpusCount] = (short)m_srcFamily[s];
|
|
m_corpusPattern[m_corpusCount] = (short)StringToInteger(StringSubstr(ps, 8));
|
|
m_corpusNetVote[m_corpusCount] = nv;
|
|
m_corpusCount++;
|
|
}
|
|
}
|
|
}
|
|
//--- shrink-to-fit: the growth above can leave up to 50% slack on large symbols
|
|
ArrayResize(m_corpusGmt, m_corpusCount);
|
|
ArrayResize(m_corpusSide, m_corpusCount);
|
|
ArrayResize(m_corpusNetVote, m_corpusCount);
|
|
ArrayResize(m_corpusFamily, m_corpusCount);
|
|
ArrayResize(m_corpusPattern, m_corpusCount);
|
|
ArrayResize(m_corpusEntry, m_corpusCount);
|
|
Print(ID + StringFormat(": candidate sweep done - %d candidates from %d bars in %.1fs (no tester"
|
|
" corpus run needed; DB corpus not used).", m_corpusCount, deepest,
|
|
(GetTickCount() - t0) / 1000.0));
|
|
m_corpusLoaded = (m_corpusCount > 0);
|
|
return m_corpusLoaded;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
bool CSignalMETA::MetaPrepareEra(const int bars)
|
|
{
|
|
//--- corpus source order: the on-chart ladder sweep (self-contained, preferred), then a
|
|
//--- tester-built DB corpus as fallback for charts whose classic filters are disabled.
|
|
if(!m_corpusLoaded && !BuildCorpusBySweep() && !LoadMetaCorpus())
|
|
return false;
|
|
ENUM_TIMEFRAMES per = (ENUM_TIMEFRAMES)m_period;
|
|
ArrayResize(m_metaCandHead, bars);
|
|
ArrayInitialize(m_metaCandHead, -1);
|
|
ArrayResize(m_metaCandBar, m_corpusCount);
|
|
ArrayResize(m_metaCandSide, m_corpusCount);
|
|
ArrayResize(m_metaCandNetVote, m_corpusCount);
|
|
ArrayResize(m_metaCandFamily, m_corpusCount);
|
|
ArrayResize(m_metaCandPattern, m_corpusCount);
|
|
ArrayResize(m_metaCandNext, m_corpusCount);
|
|
m_metaCandCount = 0;
|
|
int offCount[5] = {0, 0, 0, 0, 0};
|
|
int dropNoBar = 0, dropPrice = 0, dropRegime = 0, dropRange = 0;
|
|
long spanCap = (long)PeriodSeconds(per) * (long)MathMax(m_historyBars, 1) * 4;
|
|
for(int r = 0; r < m_corpusCount; r++)
|
|
{
|
|
int bestSh = -1, bestOff = -1;
|
|
double bestDiff = DBL_MAX;
|
|
for(int off = 0; off <= 4; off++)
|
|
{
|
|
int sh = iBarShift(_Symbol, per, m_corpusGmt[r] + off * 3600, true);
|
|
if(sh < 0)
|
|
continue;
|
|
double diff = MathAbs(iOpen(_Symbol, per, sh) - m_corpusEntry[r]);
|
|
if(diff < bestDiff)
|
|
{
|
|
bestDiff = diff;
|
|
bestSh = sh;
|
|
bestOff = off;
|
|
}
|
|
}
|
|
if(bestSh < 0)
|
|
{
|
|
dropNoBar++;
|
|
continue;
|
|
}
|
|
//--- price tolerance: right-offset |diff| <= the spread; wrong-offset ~ an hourly move. 15% of
|
|
//--- the bar's ATR separates the two with a wide margin either way; the fallback (5 basis
|
|
//--- points) covers bars where the ATR indicator has no value that deep in history.
|
|
double atr = m_ATR.Main(bestSh);
|
|
double tol = (MathIsValidNumber(atr) && atr > 0.0) ? 0.15 * atr : 0.0005 * m_corpusEntry[r];
|
|
if(bestDiff > tol)
|
|
{
|
|
dropPrice++;
|
|
continue;
|
|
}
|
|
if(bestSh >= bars)
|
|
{
|
|
dropRange++;
|
|
continue;
|
|
}
|
|
datetime tSh = iTime(_Symbol, per, bestSh);
|
|
datetime tDeep = iTime(_Symbol, per, bestSh + (int)MathMax(m_historyBars, 1));
|
|
if(tSh <= 0 || tDeep <= 0 || (long)(tSh - tDeep) > spanCap)
|
|
{
|
|
dropRegime++;
|
|
continue;
|
|
}
|
|
int id = m_metaCandCount;
|
|
m_metaCandBar[id] = bestSh;
|
|
m_metaCandSide[id] = m_corpusSide[r];
|
|
m_metaCandNetVote[id] = m_corpusNetVote[r];
|
|
m_metaCandFamily[id] = m_corpusFamily[r];
|
|
m_metaCandPattern[id] = m_corpusPattern[r];
|
|
m_metaCandNext[id] = m_metaCandHead[bestSh];
|
|
m_metaCandHead[bestSh] = id;
|
|
m_metaCandCount++;
|
|
if(bestOff >= 0 && bestOff <= 4)
|
|
offCount[bestOff]++;
|
|
}
|
|
string line = StringFormat("%s: era candidate resolution - %d of %d corpus rows usable | GMT->server"
|
|
" offset histogram +0h:%d +1h:%d +2h:%d +3h:%d +4h:%d | dropped: %d no"
|
|
" exact bar, %d price mismatch (wrong offset/bad data), %d non-H1 regime"
|
|
" (pre-intraday backfill), %d beyond era grid",
|
|
ID, m_metaCandCount, m_corpusCount, offCount[0], offCount[1], offCount[2],
|
|
offCount[3], offCount[4], dropNoBar, dropPrice, dropRegime, dropRange);
|
|
if(!m_prepareReported)
|
|
{
|
|
m_prepareReported = true;
|
|
Print(line);
|
|
}
|
|
else
|
|
PrintVerbose(line);
|
|
//--- one-shot dataset export for offline pooled training - runs here because this is the first
|
|
//--- moment candidates AND labels both exist on the current bar grid (the label prebuild completed
|
|
//--- before the era start that called us).
|
|
if(Meta_ExportDataset && !m_datasetExported && m_metaCandCount > 0)
|
|
ExportMetaDataset();
|
|
return m_metaCandCount > 0;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| Dump the full training set for offline (cross-sectional pooled) |
|
|
//| work: one float32 row per resolved, LABELED candidate - |
|
|
//| [barTime int64][family int32][pattern int32][side int32] |
|
|
//| [won int32][NetInputWidth() floats: window + descriptor]. |
|
|
//| The sidecar .meta.csv carries the layout + the geometry/BE the |
|
|
//| labels were computed at, so the offline side never guesses. |
|
|
//| This is EXACTLY what pass 2 trains on - same window builder, |
|
|
//| same descriptor, same caches - so an offline model on this file |
|
|
//| and the EA's own training see byte-equivalent examples. |
|
|
//+------------------------------------------------------------------+
|
|
void CSignalMETA::ExportMetaDataset(void)
|
|
{
|
|
m_datasetExported = true;
|
|
const string base = "Warrior_EA\\MetaExport\\" + _Symbol + "_" + IntegerToString((int)m_period);
|
|
int fh = FileOpen(base + ".f32", FILE_BIN | FILE_WRITE | FILE_COMMON);
|
|
if(fh == INVALID_HANDLE)
|
|
{
|
|
Print(ID + ": dataset export FAILED - cannot open Common\\Files\\" + base + ".f32 (error " +
|
|
IntegerToString(GetLastError()) + ")");
|
|
return;
|
|
}
|
|
Print(ID + StringFormat(": exporting %d candidates to Common\\Files\\%s.f32 - one pass-1-sized"
|
|
" sweep, the chart is busy for it...", m_metaCandCount, base));
|
|
int width = NetInputWidth();
|
|
int rows = 0, skipLabel = 0, skipWindow = 0;
|
|
for(int cd = 0; cd < m_metaCandCount; cd++)
|
|
{
|
|
int idx = m_metaCandBar[cd];
|
|
if(idx < 0 || idx >= ArraySize(m_labelCacheHasValue) || !m_labelCacheHasValue[idx])
|
|
{
|
|
skipLabel++;
|
|
continue;
|
|
}
|
|
if(!BuildFeatureWindow(idx))
|
|
{
|
|
skipWindow++;
|
|
continue;
|
|
}
|
|
AppendCandidateFeatures(cd);
|
|
if(TempData.Total() != width)
|
|
{
|
|
skipWindow++;
|
|
continue;
|
|
}
|
|
FileWriteLong(fh, (long)iTime(_Symbol, (ENUM_TIMEFRAMES)m_period, idx));
|
|
FileWriteInteger(fh, (int)m_metaCandFamily[cd]);
|
|
FileWriteInteger(fh, (int)m_metaCandPattern[cd]);
|
|
FileWriteInteger(fh, (int)m_metaCandSide[cd]);
|
|
FileWriteInteger(fh, MetaCandidateWon(cd, idx) ? 1 : 0);
|
|
for(int k = 0; k < width; k++)
|
|
FileWriteFloat(fh, (float)TempData.At(k));
|
|
rows++;
|
|
}
|
|
FileClose(fh);
|
|
double slMult = 0.0, tpMult = 0.0;
|
|
BarrierMultiples(slMult, tpMult);
|
|
double bePct = (slMult + tpMult > 0.0) ? 100.0 * slMult / (slMult + tpMult) : 50.0;
|
|
int mh = FileOpen(base + ".meta.csv", FILE_CSV | FILE_WRITE | FILE_COMMON, ',');
|
|
if(mh != INVALID_HANDLE)
|
|
{
|
|
FileWrite(mh, "symbol", "period", "rows", "width", "historyBars", "featuresPerBar", "descWidth",
|
|
"slMult", "tpMult", "breakEvenPct", "horizonBars", "spreadPoints");
|
|
FileWrite(mh, _Symbol, IntegerToString((int)m_period), IntegerToString(rows),
|
|
IntegerToString(width), IntegerToString((int)m_historyBars),
|
|
IntegerToString(m_neuronsCount), IntegerToString(MetaDescWidth()),
|
|
DoubleToString(slMult, 4), DoubleToString(tpMult, 4), DoubleToString(bePct, 2),
|
|
IntegerToString(m_barrierHorizonBars), IntegerToString((int)m_symbol.Spread()));
|
|
FileClose(mh);
|
|
}
|
|
Print(ID + StringFormat(": dataset exported - %d rows (%d skipped: %d unlabeled near the era edge,"
|
|
" %d unusable windows) x %d floats | geometry %.2f/%.2f BE %.1f%% |"
|
|
" %s.f32 + .meta.csv", rows, skipLabel + skipWindow, skipLabel, skipWindow,
|
|
width, slMult, tpMult, bePct, base));
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| The setup descriptor - MUST append exactly META_DESC_FEATURES |
|
|
//| values (the input layer is sized for them; a short append fails |
|
|
//| the NetInputWidth() guard and the sample is skipped, a long one |
|
|
//| would corrupt the forward pass). |
|
|
//+------------------------------------------------------------------+
|
|
void CSignalMETA::AppendCandidateFeatures(const int candId)
|
|
{
|
|
if(candId < 0 || candId >= m_metaCandCount)
|
|
return;
|
|
int idx = m_metaCandBar[candId];
|
|
int slot = OneHotSlot(m_metaCandFamily[candId], m_metaCandPattern[candId]);
|
|
for(int k = 0; k < META_ONE_HOT_SLOTS; k++)
|
|
TempData.Add(k == slot ? 1.0 : 0.0);
|
|
TempData.Add((double)m_metaCandSide[candId]);
|
|
//--- netVote is in raw pattern-weight units (+-100ish); tanh(nv/20) keeps resolution where the
|
|
//--- votes actually live while bounding the tails. MQL5 has no MathTanh - via exp.
|
|
double e2 = MathExp(2.0 * (m_metaCandNetVote[candId] / 20.0));
|
|
TempData.Add((e2 - 1.0) / (e2 + 1.0));
|
|
//--- geometry in ATR units - constant across candidates TODAY (one pinned barrier pair), but the
|
|
//--- design's excursion-head integration makes it per-candidate later, so it rides as a feature.
|
|
double slMult = 0.0, tpMult = 0.0;
|
|
BarrierMultiples(slMult, tpMult);
|
|
TempData.Add(slMult);
|
|
TempData.Add(tpMult);
|
|
//--- spread/ATR at the fire bar (EnsureSpreadSeries copies unconditionally for the meta target)
|
|
double sprAtr = 0.0;
|
|
double atr = m_ATR.Main(idx);
|
|
if(idx >= 0 && idx < m_spreadSeriesBars && MathIsValidNumber(atr) && atr > 0.0)
|
|
sprAtr = (double)m_spreadSeries[idx] * m_symbol.Point() / atr;
|
|
TempData.Add(sprAtr);
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| S3 - THE LIVE META GATE (2026-08-19, user design: "the META NN |
|
|
//| should be integrated into the voting decision pipeline"). |
|
|
//| |
|
|
//| Scores the PROPOSED trade the consensus vote just cleared: the |
|
|
//| shared feature window at barIdx plus a descriptor of the proposal |
|
|
//| itself (side, the net vote, the live SL/TP geometry, spread/ATR). |
|
|
//| The pattern one-hot is ZEROED - the proposal is the vote, not a |
|
|
//| classic-ladder fire. On the S2 MLP the one-hot columns act as |
|
|
//| additive per-pattern bias vectors into the first layer, so |
|
|
//| zeroing them asks the trained context trunk for its pattern-free |
|
|
//| score. That is an extrapolation by one additive term (no training |
|
|
//| row ever had an empty one-hot), so the honest reading is RANKING, |
|
|
//| not calibrated probability - which is why the threshold is the |
|
|
//| cost-adjusted break-even (the least arbitrary zero-knob rule) and |
|
|
//| the gate ships default-off. The measured next lever is journaling |
|
|
//| the vote itself as a candidate family so the head trains on the |
|
|
//| exact stream it gates - see Meta_Labeling_Design.md's S3 note. |
|
|
//| |
|
|
//| FAIL-OPEN EVERYWHERE: not trained, no net, window unbuildable, |
|
|
//| width mismatch, non-finite forward - all return codes that let |
|
|
//| the trade proceed. The only -1 is a real scored veto. The BN |
|
|
//| freeze save/restore bracket makes the forward a pure function |
|
|
//| (the snapshot rule - the same bracket DisplayInference uses); the |
|
|
//| meta front-end is MLP-only, so there is no LSTM state to guard. |
|
|
//| Telemetry (m_metaGate*) is written only for LIVE queries |
|
|
//| (barIdx == 1): the ensemble verdict's historical replays must not |
|
|
//| inflate the HUD's live approve/veto counters. |
|
|
//+------------------------------------------------------------------+
|
|
int CSignalMETA::LiveMetaGate(const bool isLong, const double netVote, double &pWin,
|
|
double &bePct, const int barIdx)
|
|
{
|
|
pWin = -1.0;
|
|
bePct = CostAdjustedBreakEvenPct();
|
|
bool live = (barIdx == 1);
|
|
//--- One readiness test + transition announcement for every observer - see MetaGateArmedNow.
|
|
if(!MetaGateArmedNow())
|
|
return 0;
|
|
if(!BuildFeatureWindow(barIdx))
|
|
return 1;
|
|
AppendLiveDescriptor((char)(isLong ? 1 : -1), netVote, barIdx);
|
|
if(TempData.Total() != NetInputWidth())
|
|
return 1;
|
|
//--- Save/restore, NOT set/clear: frozen, this forward is a pure function (the snapshot rule).
|
|
bool bnWasFrozen = Net.GetBatchNormFrozen();
|
|
if(!bnWasFrozen)
|
|
Net.SetBatchNormFrozen(true);
|
|
bool fwdOk = Net.feedForward(TempData);
|
|
if(fwdOk)
|
|
Net.getResults(TempData);
|
|
if(!bnWasFrozen)
|
|
Net.SetBatchNormFrozen(false);
|
|
if(!fwdOk)
|
|
return 1;
|
|
double p = MetaWinProbability();
|
|
if(p < 0.0 || !MathIsValidNumber(p))
|
|
return 1;
|
|
pWin = p;
|
|
int verdict = (100.0 * p >= bePct) ? 2 : -1;
|
|
if(live)
|
|
{
|
|
m_metaGateLastP = p;
|
|
m_metaGateLastBe = bePct;
|
|
if(verdict > 0)
|
|
m_metaGateApproved++;
|
|
else
|
|
m_metaGateVetoed++;
|
|
}
|
|
return verdict;
|
|
}
|
|
//+------------------------------------------------------------------+
|
|
//| The gate-time twin of AppendCandidateFeatures: same width, same |
|
|
//| slot order, same squash - MUST stay in step with it and with |
|
|
//| META_DESC_FEATURES. The differences ARE the proposal: zeroed |
|
|
//| one-hot (the vote is not a classic-ladder fire), the caller's |
|
|
//| side and net vote, and spread/ATR at barIdx through the same |
|
|
//| series-or-zero rule training used (a fallback to the current |
|
|
//| quoted spread would feed the net a quantity training never saw |
|
|
//| on unavailable bars). |
|
|
//+------------------------------------------------------------------+
|
|
void CSignalMETA::AppendLiveDescriptor(const char side, const double netVote, const int barIdx)
|
|
{
|
|
for(int k = 0; k < META_ONE_HOT_SLOTS; k++)
|
|
TempData.Add(0.0);
|
|
TempData.Add((double)side);
|
|
//--- same tanh(nv/20) squash as AppendCandidateFeatures - MQL5 has no MathTanh, via exp
|
|
double e2 = MathExp(2.0 * (netVote / 20.0));
|
|
TempData.Add((e2 - 1.0) / (e2 + 1.0));
|
|
double slMult = 0.0, tpMult = 0.0;
|
|
BarrierMultiples(slMult, tpMult);
|
|
TempData.Add(slMult);
|
|
TempData.Add(tpMult);
|
|
double sprAtr = 0.0;
|
|
double atr = m_ATR.Main(barIdx);
|
|
if(barIdx >= 0 && barIdx < m_spreadSeriesBars && MathIsValidNumber(atr) && atr > 0.0)
|
|
sprAtr = (double)m_spreadSeries[barIdx] * m_symbol.Point() / atr;
|
|
TempData.Add(sprAtr);
|
|
}
|
|
//+------------------------------------------------------------------+
|