koros0111/Warrior_EA
Watch
1
0
Fork
You've already forked Warrior_EA
 0
forked from animatedread/Warrior_EA
Code Pull requests Activity
Warrior_EA/Expert/Chart/VoteArrows.mqh
AnimateDread 781ae3a702 perf(deinit): I/O-free chart cleanup, dead-panel purge, skip clean weight saves 
The 18:23 terminal close (20260825.log) killed two of six charts inside
OnDeinit: they printed "shutting down" then nothing for 5.9 s until
"Abnormal termination", stranding ~700 objects each - including the one
family no prefix sweep can reach, the control panel (CAppDialog names
its 15 objects <numeric instance id><control>, and a re-attach mints a
new id, so a killed panel is a permanent ghost; XTIUSD carried one
across sessions). The stall sat in the two file writes that preceded
all visible cleanup while the four sibling charts flooded the same
2013-era disk - the ~4x18MB-per-chart shutdown weight saves.

Three changes:

1. OnDeinit touches no file until the chart is clean. CVoteArrowStore
   splits Save() into Snapshot() (the chart scan, in memory) and
   WriteSnapshot() (the disk half, consuming). New order: status label,
   vote-arrow snapshot, prefix sweep, panel destroy - all object ops -
   then member sidecars, final sweep, timings, and only then the
   visibility file, the vote-arrow write and the weight saves.

2. PurgeOrphanedPanelObjects() at OnInit: deletes numeric-prefix
   CAppDialog ghosts by name (6 chrome + 9 buttons), qualifying a
   prefix only when >=4 of OUR button names carry it, so a foreign
   dialog sharing stock chrome names is never touched.

3. m_netDirty: set by every net mutation (both backProp sites, both
   RestoreWeights sites, online learning conservatively, panel reset),
   cleared only on a successful Net.Save. Shutdown AND the per-bar
   autosave now skip the ~18MB write when the net is provably unchanged
   - for converged ensembles that is every save - which removes the
   very flood that starved the sibling charts. .stats still writes
   every time (small; carries the vote record and calibration). A
   skipped save leaves the .nnw header dtStudied stale, which is the
   already-handled attach-after-offline-gap case.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 18:46:51 -04:00

401 lines
19 KiB
MQL5
Raw Permalink Blame History

//+------------------------------------------------------------------+
//| Warrior_EA |
//| AnimateDread |
//| |
//| PERSISTENCE FOR THE VOTE ARROWS - the filtered view's own layer. |
//+------------------------------------------------------------------+
#ifndef WARRIOR_CHART_VOTEARROWS_MQH
#define WARRIOR_CHART_VOTEARROWS_MQH
//+------------------------------------------------------------------+
//| WHY THIS EXISTS AS ITS OWN STORE. |
//| |
//| With DrawUnfilteredSignals OFF - the shipped default - the arrows |
//| on the chart are the COMBINED VOTE's (SIG_VOTE_PREFIX), drawn by |
//| the aggregate signal's overlay. They are the ones that answer |
//| "where would this EA actually have traded", because the overlay |
//| only draws a bar whose |vote| cleared Signal_ThresholdOpen. |
//| |
//| Nothing persisted them. CChartUI's .arrows sidecar is MEMBER- |
//| scoped (it scans m_view.ArrowPrefix()), so it never saw this |
//| layer at all, and the only route back was a full overlay re-sweep.|
//| That sweep replays each member's m_overlaySigSnap - an IN-MEMORY |
//| cache filled at pass-3 completion - and it re-arms only when an |
//| era ends. A DEPLOYED ensemble runs no further eras. So on a |
//| terminal restart the arrows were gone AND unrecoverable: no file |
//| to read them from, and no era ever coming to redraw them. |
//| |
//| THE THRESHOLDS ARE PART OF THE RECORD. An arrow means "the vote |
//| cleared the bar to open", so it is only true relative to the |
//| threshold it was measured against. The header stores the open and |
//| close thresholds the arrows were drawn under; on load a mismatch |
//| DISCARDS them rather than restoring a picture of a strategy the |
//| operator no longer runs. Showing stale arrows under a new |
//| threshold is worse than showing none: none is visibly empty, |
//| stale is confidently wrong. |
//+------------------------------------------------------------------+
#include "..\..\System\AtomicFile.mqh"
//--- 'WVA1'. Bump only if the record layout changes; the thresholds live IN the record, so a
//--- threshold change is a data mismatch, not a format one.
#define VOTE_ARROWS_MAGIC 0x57564131
//--- Same retention and pacing as the member arrows (see MAX_PERSISTED_ARROWS / ARROW_RESTORE_BUDGET_MS
//--- in ExpertSignalAIBase.mqh). Duplicated as its own names rather than borrowed: this layer holds one
//--- arrow per TRADE rather than one per model opinion, so it is legitimately allowed to diverge.
#define VOTE_ARROWS_MAX_KEPT 1000
#define VOTE_ARROWS_MAX_HEADER 50000
#define VOTE_ARROWS_BUDGET_MS 50
//+------------------------------------------------------------------+
//| Chart-scoped, single instance. The vote belongs to the CHART, not |
//| to any member - which is exactly why it could not live in |
//| CChartUI, whose every path is scoped to one model's prefix. |
//+------------------------------------------------------------------+
class CVoteArrowStore
{
private:
string m_key; // file basename, set once at init
double m_thresholdOpen; // what the stored arrows were measured against
double m_thresholdClose;
bool m_active; // false in tester/optimizer and in the raw view
//--- parsed-but-not-yet-drawn, consumed by AdvanceRestore()
datetime m_time[];
int m_code[];
double m_price[];
long m_tf[];
int m_index;
bool m_pending;
uint m_startMs;
int m_lastSaved;
//--- DEINIT SPLIT. Snapshot() is the chart-scan half of a save, held in memory with ZERO disk
//--- I/O, so OnDeinit can capture the arrows and purge the chart immediately; WriteSnapshot()
//--- is the disk half, run later with the heavy persistence. Measured 2026-08-25 18:23: two of
//--- six charts never reached their first cleanup step because the un-split Save() (a file
//--- write) sat ahead of it while four sibling charts flooded the same disk - "Abnormal
//--- termination" 5.9 s in, objects stranded.
long m_snapTimes[];
int m_snapCodes[];
double m_snapPrices[];
long m_snapTfs[];
int m_snapCount;
bool m_snapTaken;
string FileName(void) const { return m_key + ".votearrows"; }
public:
CVoteArrowStore(void) :
m_key(""), m_thresholdOpen(-1.0), m_thresholdClose(-1.0), m_active(false),
m_index(0), m_pending(false), m_startMs(0), m_lastSaved(0),
m_snapCount(0), m_snapTaken(false) {}
bool Active(void) const { return m_active; }
bool Pending(void) const { return m_pending; }
int LastSaved(void) const { return m_lastSaved; }
void Configure(const string key, const double thresholdOpen, const double thresholdClose,
const bool active);
void Load(void);
void AdvanceRestore(void);
void Snapshot(void);
bool WriteSnapshot(void);
bool Save(void) { Snapshot(); return WriteSnapshot(); }
void Discard(const string reason);
};
CVoteArrowStore g_voteArrows;
//+------------------------------------------------------------------+
//| Bind the store to this chart's configuration. `active` is false |
//| in the tester (throwaway charts) and in the raw view, where the |
//| per-member layer owns the chart and this one draws nothing. |
//+------------------------------------------------------------------+
void CVoteArrowStore::Configure(const string key, const double thresholdOpen, const double thresholdClose,
const bool active)
{
m_key = key;
m_thresholdOpen = thresholdOpen;
m_thresholdClose = thresholdClose;
m_active = active;
}
//+------------------------------------------------------------------+
//| Read the sidecar and QUEUE it. Deliberately draws nothing here - |
//| same reason CChartUI::LoadChartSignals defers: OnInit must not |
//| block, and this queue can hold a thousand object pairs. |
//+------------------------------------------------------------------+
void CVoteArrowStore::Load(void)
{
if(!m_active || m_key == "")
return;
string fn = FileName();
if(!FileIsExist(fn, FILE_COMMON))
return;
//--- share flags: read-only, and must not fail because another instance holds the file.
int handle = FileOpen(fn, FILE_COMMON | FILE_BIN | FILE_READ | FILE_SHARE_READ | FILE_SHARE_WRITE);
if(handle == INVALID_HANDLE)
return;
if(FileReadInteger(handle) != VOTE_ARROWS_MAGIC)
{
FileClose(handle);
return;
}
double savedOpen = FileReadDouble(handle);
double savedClose = FileReadDouble(handle);
int n = FileReadInteger(handle);
FileClose(handle);
//--- THE THRESHOLD GATE. See the header note: an arrow is a claim about a threshold, so a changed
//--- threshold makes every stored arrow a claim about a strategy that is no longer configured.
if(savedOpen != m_thresholdOpen || savedClose != m_thresholdClose)
{
Discard(StringFormat("thresholds changed (stored open %.0f / close %.0f, now open %.0f / close %.0f)"
" - those arrows describe a strategy this chart no longer runs, so they are"
" discarded rather than redrawn. They rebuild from the vote as the models run.",
savedOpen, savedClose, m_thresholdOpen, m_thresholdClose));
return;
}
if(n <= 0 || n > VOTE_ARROWS_MAX_HEADER)
{
if(n > VOTE_ARROWS_MAX_HEADER)
Print("Warrior: .votearrows header declares " + IntegerToString(n) + " arrows - refusing to restore (limit " +
IntegerToString(VOTE_ARROWS_MAX_HEADER) + "); the file looks corrupt.");
return;
}
//--- Reopen to stream the records: the header read above is cheap and lets a mismatched/corrupt
//--- file be rejected before any array is sized to it.
handle = FileOpen(fn, FILE_COMMON | FILE_BIN | FILE_READ | FILE_SHARE_READ | FILE_SHARE_WRITE);
if(handle == INVALID_HANDLE)
return;
FileReadInteger(handle);
FileReadDouble(handle);
FileReadDouble(handle);
FileReadInteger(handle);
ArrayResize(m_time, n);
ArrayResize(m_code, n);
ArrayResize(m_price, n);
ArrayResize(m_tf, n);
int parsed = 0;
for(int i = 0; i < n && !FileIsEnding(handle); i++)
{
m_time[parsed] = (datetime)FileReadLong(handle);
m_code[parsed] = FileReadInteger(handle);
m_price[parsed] = FileReadDouble(handle);
m_tf[parsed] = FileReadLong(handle);
parsed++;
}
FileClose(handle);
//--- A truncated file yields fewer records than the header claimed - keep what parsed.
if(parsed != n)
{
ArrayResize(m_time, parsed);
ArrayResize(m_code, parsed);
ArrayResize(m_price, parsed);
ArrayResize(m_tf, parsed);
}
m_index = 0;
m_pending = (parsed > 0);
m_startMs = GetTickCount();
if(m_pending)
Print("Warrior: queued " + IntegerToString(parsed) + " combined-vote arrows for progressive restore"
" (threshold to open " + DoubleToString(m_thresholdOpen, 0) + "%).");
}
//+------------------------------------------------------------------+
//| Draw a time-boxed slice. Bounds THROUGHPUT, not latency to a |
//| stop - hence the IsStopped() check inside the loop as well. |
//+------------------------------------------------------------------+
void CVoteArrowStore::AdvanceRestore(void)
{
if(!m_pending)
return;
int total = ArraySize(m_time);
uint sliceStart = GetTickCount();
int drawn = 0;
while(m_index < total)
{
//--- Budget checked every 64 objects: GetTickCount() is not free, and at microseconds per object
//--- a per-iteration clock read would dominate the work being measured.
if(drawn > 0 && (drawn & 63) == 0 && GetTickCount() - sliceStart >= VOTE_ARROWS_BUDGET_MS)
break;
if(drawn > 0 && (drawn & 63) == 0 && IsStopped())
return;
datetime t = m_time[m_index];
int code = m_code[m_index];
double price = m_price[m_index];
long tf = m_tf[m_index];
m_index++;
drawn++;
bool isBuy = (code == WARRIOR_SIG_CODE_BUY);
string nm = SIG_VOTE_PREFIX + TimeToString(t);
//--- isTrade=true: this layer IS the trade layer, and it must keep the heavier styling that
//--- distinguishes it from a member's raw opinion (see WarriorPlotSignalLevel).
WarriorPlotSignalLevel(nm, t, (ENUM_TIMEFRAMES)Period(), price, isBuy, true,
StringFormat("would trade %s @ %s | vote cleared %.0f%% | restored from the"
" previous session",
(isBuy ? "BUY" : "SELL"),
DoubleToString(price, (int)SymbolInfoInteger(_Symbol, SYMBOL_DIGITS)),
m_thresholdOpen));
//--- Both halves, or a restore made while signals are hidden brings the lines back alone.
ObjectSetInteger(0, nm, OBJPROP_TIMEFRAMES, tf);
ObjectSetInteger(0, WarriorSignalArrowName(nm), OBJPROP_TIMEFRAMES, tf);
}
ChartRedraw(0);
if(m_index < total)
return; // more slices to come
m_pending = false;
Print("Warrior: restored " + IntegerToString(total) + " combined-vote arrows in " +
IntegerToString((int)(GetTickCount() - m_startMs)) + " ms (progressive, non-blocking).");
ArrayFree(m_time);
ArrayFree(m_code);
ArrayFree(m_price);
ArrayFree(m_tf);
}
//+------------------------------------------------------------------+
//| SCAN HALF of a save: capture the drawn vote arrows into memory, |
//| zero disk I/O. See the m_snap* declaration comment - this is what |
//| lets OnDeinit purge the chart before any file write can block on |
//| a contended disk. |
//| |
//| Unlike the member layer a save does NOT clear the chart: the |
//| overlay redraws this layer from scratch on every sweep, so the |
//| file is a mirror of the chart rather than its only copy. |
//+------------------------------------------------------------------+
void CVoteArrowStore::Snapshot(void)
{
m_snapCount = 0;
m_snapTaken = true;
if(!m_active || m_key == "")
return;
int n = 0;
int total = ObjectsTotal(0);
//--- Room for the chart's arrows PLUS anything still sitting in the restore queue: a save that
//--- runs mid-restore would otherwise write only the part already drawn and silently truncate the
//--- history to the slice that happened to have finished.
int queueRemaining = m_pending ? (ArraySize(m_time) - m_index) : 0;
if(queueRemaining < 0)
queueRemaining = 0;
ArrayResize(m_snapTimes, total + queueRemaining);
ArrayResize(m_snapCodes, total + queueRemaining);
ArrayResize(m_snapPrices, total + queueRemaining);
ArrayResize(m_snapTfs, total + queueRemaining);
for(int i = 0; i < total; i++)
{
string nm = ObjectName(0, i);
if(StringFind(nm, SIG_VOTE_PREFIX) != 0)
continue;
//--- OBJ_TREND only: a mark is a line AND an arrow, and the line is the canonical half (it
//--- carries the trigger price). Counting both would write every arrow twice.
if(ObjectGetInteger(0, nm, OBJPROP_TYPE) != OBJ_TREND)
continue;
m_snapTimes[n] = (long)ObjectGetInteger(0, nm, OBJPROP_TIME, 0);
//--- Direction from the COLOUR - a line carries no arrow code. Same encoding as the member
//--- sidecar so both files stay readable by one convention.
m_snapCodes[n] = ((color)ObjectGetInteger(0, nm, OBJPROP_COLOR) == WARRIOR_SIG_BUY_COLOR)
? WARRIOR_SIG_CODE_BUY : WARRIOR_SIG_CODE_SELL;
m_snapPrices[n] = ObjectGetDouble(0, nm, OBJPROP_PRICE, 0);
m_snapTfs[n] = g_signalsVisible ? OBJ_ALL_PERIODS : OBJ_NO_PERIODS;
n++;
}
for(int q = m_index; q < m_index + queueRemaining; q++)
{
m_snapTimes[n] = (long)m_time[q];
m_snapCodes[n] = m_code[q];
m_snapPrices[n] = m_price[q];
m_snapTfs[n] = g_signalsVisible ? OBJ_ALL_PERIODS : OBJ_NO_PERIODS;
n++;
}
//--- Keep the most recent VOTE_ARROWS_MAX_KEPT by TIME (scan order is object order, not time order).
if(n > VOTE_ARROWS_MAX_KEPT)
{
long sortedTimes[];
ArrayResize(sortedTimes, n);
ArrayCopy(sortedTimes, m_snapTimes, 0, 0, n);
ArraySort(sortedTimes); // ascending
long cutoff = sortedTimes[n - VOTE_ARROWS_MAX_KEPT];
int w = 0;
for(int i = 0; i < n; i++)
{
//--- ">= cutoff" can match more than the cap when several arrows share the cut-off timestamp;
//--- the w < cap test keeps the kept set at exactly the cap in that case.
if(m_snapTimes[i] >= cutoff && w < VOTE_ARROWS_MAX_KEPT)
{
m_snapTimes[w] = m_snapTimes[i];
m_snapCodes[w] = m_snapCodes[i];
m_snapPrices[w] = m_snapPrices[i];
m_snapTfs[w] = m_snapTfs[i];
w++;
}
}
n = w;
}
m_snapCount = n;
}
//+------------------------------------------------------------------+
//| DISK HALF: write the captured snapshot, thresholds included. |
//| Consuming - the snapshot is freed either way, so a stale capture |
//| can never be replayed over a fresher chart. |
//+------------------------------------------------------------------+
bool CVoteArrowStore::WriteSnapshot(void)
{
if(!m_snapTaken)
return true;
m_snapTaken = false;
int n = m_snapCount;
m_snapCount = 0;
bool ok = true;
//--- NOTHING DRAWN IS NOT NOTHING TO SAY, but it is not a reason to destroy the record either: a
//--- chart that has not swept yet has no arrows and would otherwise wipe a good file on shutdown.
if(m_active && m_key != "" && n > 0)
{
string fn = FileName();
string tmpName = "";
int handle = AtomicWriteBegin(fn, FILE_COMMON, tmpName);
if(handle == INVALID_HANDLE)
{
Print("Warrior: ERROR - could not open " + tmpName + " to persist " + IntegerToString(n) +
" combined-vote arrows, error " + IntegerToString(GetLastError()));
ok = false;
}
else
{
ok = (FileWriteInteger(handle, VOTE_ARROWS_MAGIC) > 0);
if(ok && FileWriteDouble(handle, m_thresholdOpen) <= 0)
ok = false;
if(ok && FileWriteDouble(handle, m_thresholdClose) <= 0)
ok = false;
if(ok && FileWriteInteger(handle, n) <= 0)
ok = false;
for(int i = 0; ok && i < n; i++)
{
if(FileWriteLong(handle, m_snapTimes[i]) <= 0 || FileWriteInteger(handle, m_snapCodes[i]) <= 0 ||
FileWriteDouble(handle, m_snapPrices[i]) <= 0 || FileWriteLong(handle, m_snapTfs[i]) <= 0)
ok = false;
}
//--- Staged + atomically renamed, so an interrupted write keeps the previous arrow set rather
//--- than publishing a truncated one.
if(!AtomicWriteEnd(handle, fn, tmpName, FILE_COMMON, ok, __FUNCTION__))
ok = false;
else
if(ok)
m_lastSaved = n;
}
}
ArrayFree(m_snapTimes);
ArrayFree(m_snapCodes);
ArrayFree(m_snapPrices);
ArrayFree(m_snapTfs);
return ok;
}
//+------------------------------------------------------------------+
//| Drop the stored arrows and the queue, and say why on the journal. |
//| The chart objects go too: whatever reason invalidated the file |
//| invalidates what is drawn from it. |
//+------------------------------------------------------------------+
void CVoteArrowStore::Discard(const string reason)
{
m_pending = false;
m_index = 0;
ArrayFree(m_time);
ArrayFree(m_code);
ArrayFree(m_price);
ArrayFree(m_tf);
ObjectsDeleteAll(0, SIG_VOTE_PREFIX);
if(m_key != "" && FileIsExist(FileName(), FILE_COMMON))
FileDelete(FileName(), FILE_COMMON);
ChartRedraw(0);
Print("Warrior: combined-vote arrows cleared - " + reason);
}
#endif // WARRIOR_CHART_VOTEARROWS_MQH
//+------------------------------------------------------------------+
View git blame Copy permalink