Warrior_EA/System/AtomicFile.mqh

52 lines
2.3 KiB
MQL5
Raw Permalink Normal View History

fix: make sidecar writes atomic; extract shared AtomicFile helper FileOpen(FILE_WRITE) truncates its target on open. CNet::Save already staged the .nnw through a temp file + rename for that reason, but the three sidecars written beside it did not: .stats ExpertSignalAIBase.mqh:5918 .arrows ExpertSignalAIBase.mqh:6224 .cfg ExpertSignalAIBase.mqh:7329 Two defects followed. 1. An interrupted write published a truncated sidecar. For .cfg that is the worst case: LoadAndCompareTopologyConfiguration() reads a short file as a mismatch, which discards the trained model and restarts from era 0. 2. Windows file sharing is a mutual contract - a writer opened with no FILE_SHARE_* blocks every concurrent open regardless of the reader's flags. All three read paths carry FILE_SHARE_READ|FILE_SHARE_WRITE specifically so a tester agent can read them while a live chart runs; an exclusive writer on the same path defeated that. Extracted CNet::Save's proven pattern into System\AtomicFile.mqh (AtomicWriteBegin/AtomicWriteEnd) and routed all four writers through it. This also encodes the FileMove gotcha once instead of per call site: the destination location comes from FILE_COMMON inside the 4th arg, NOT inherited from the source, and getting it wrong moves the file to the wrong sandbox silently. Also fixed while in these functions: - SaveTopologyConfiguration had 13 copy-pasted 6-line error blocks that each returned WITHOUT FileClose(handle), leaking the handle on every write failure. Collapsed to one ok-chain that closes exactly once. The on-disk field order and types are unchanged (asserted during the rewrite) so existing .cfg files still load. - SaveChartSignals documented that pruning runs only after a successful write ("a failed write above leaves both the file AND the chart untouched") but never checked any write result, so a partial write still deleted the chart objects. Results are checked now, making the existing comment true. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:31:29 -04:00
//+------------------------------------------------------------------+
//| Warrior_EA |
//| AnimateDread |
//| |
//| Crash-safe binary file writes. |
fix: make sidecar writes atomic; extract shared AtomicFile helper FileOpen(FILE_WRITE) truncates its target on open. CNet::Save already staged the .nnw through a temp file + rename for that reason, but the three sidecars written beside it did not: .stats ExpertSignalAIBase.mqh:5918 .arrows ExpertSignalAIBase.mqh:6224 .cfg ExpertSignalAIBase.mqh:7329 Two defects followed. 1. An interrupted write published a truncated sidecar. For .cfg that is the worst case: LoadAndCompareTopologyConfiguration() reads a short file as a mismatch, which discards the trained model and restarts from era 0. 2. Windows file sharing is a mutual contract - a writer opened with no FILE_SHARE_* blocks every concurrent open regardless of the reader's flags. All three read paths carry FILE_SHARE_READ|FILE_SHARE_WRITE specifically so a tester agent can read them while a live chart runs; an exclusive writer on the same path defeated that. Extracted CNet::Save's proven pattern into System\AtomicFile.mqh (AtomicWriteBegin/AtomicWriteEnd) and routed all four writers through it. This also encodes the FileMove gotcha once instead of per call site: the destination location comes from FILE_COMMON inside the 4th arg, NOT inherited from the source, and getting it wrong moves the file to the wrong sandbox silently. Also fixed while in these functions: - SaveTopologyConfiguration had 13 copy-pasted 6-line error blocks that each returned WITHOUT FileClose(handle), leaking the handle on every write failure. Collapsed to one ok-chain that closes exactly once. The on-disk field order and types are unchanged (asserted during the rewrite) so existing .cfg files still load. - SaveChartSignals documented that pruning runs only after a successful write ("a failed write above leaves both the file AND the chart untouched") but never checked any write result, so a partial write still deleted the chart objects. Results are checked now, making the existing comment true. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:31:29 -04:00
//+------------------------------------------------------------------+
#ifndef WARRIOR_ATOMIC_FILE_MQH
#define WARRIOR_ATOMIC_FILE_MQH
//+------------------------------------------------------------------+
//| Open a temp file to stage an atomic binary write. |
fix: make sidecar writes atomic; extract shared AtomicFile helper FileOpen(FILE_WRITE) truncates its target on open. CNet::Save already staged the .nnw through a temp file + rename for that reason, but the three sidecars written beside it did not: .stats ExpertSignalAIBase.mqh:5918 .arrows ExpertSignalAIBase.mqh:6224 .cfg ExpertSignalAIBase.mqh:7329 Two defects followed. 1. An interrupted write published a truncated sidecar. For .cfg that is the worst case: LoadAndCompareTopologyConfiguration() reads a short file as a mismatch, which discards the trained model and restarts from era 0. 2. Windows file sharing is a mutual contract - a writer opened with no FILE_SHARE_* blocks every concurrent open regardless of the reader's flags. All three read paths carry FILE_SHARE_READ|FILE_SHARE_WRITE specifically so a tester agent can read them while a live chart runs; an exclusive writer on the same path defeated that. Extracted CNet::Save's proven pattern into System\AtomicFile.mqh (AtomicWriteBegin/AtomicWriteEnd) and routed all four writers through it. This also encodes the FileMove gotcha once instead of per call site: the destination location comes from FILE_COMMON inside the 4th arg, NOT inherited from the source, and getting it wrong moves the file to the wrong sandbox silently. Also fixed while in these functions: - SaveTopologyConfiguration had 13 copy-pasted 6-line error blocks that each returned WITHOUT FileClose(handle), leaking the handle on every write failure. Collapsed to one ok-chain that closes exactly once. The on-disk field order and types are unchanged (asserted during the rewrite) so existing .cfg files still load. - SaveChartSignals documented that pruning runs only after a successful write ("a failed write above leaves both the file AND the chart untouched") but never checked any write result, so a partial write still deleted the chart objects. Results are checked now, making the existing comment true. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:31:29 -04:00
//+------------------------------------------------------------------+
int AtomicWriteBegin(const string finalName, const int commonFlag, string &tmpName)
{
tmpName = finalName + ".savetmp";
ResetLastError();
refactor(stdlib): adopt Math\Stat for the deploy gate's normal tail; retire the b1/b2/lr/momentum macros The gate's NormalUpperTail was a hand-rolled Abramowitz & Stegun 26.2.17 approximation. Its own comment gave the reason - "drags a chain of headers behind it" - and that turned out to be one file: Math\Stat\Normal.mqh includes only Math.mqh, which includes nothing. Swapped for Cody's rational approximation in the library (~18 significant digits vs |error| < 7.5e-8). No past verdict changes: at the z the gate operates on, the difference is orders of magnitude below DEPLOY_FAMILY_WISE_ALPHA. Adopting it needed the four bare macros in AI\Network.mqh gone first. "#define b1 AdamBeta1" collides with an identifier in Math.mqh, so the include would have macro-expanded the library's own local and failed to compile - the same landmine that made the original author rename the approximation's coefficients to ntB1..ntB5 rather than use the reference's b1..b5. lr, b2 and momentum are the same class of hazard: single-token global macros in a 52k-line codebase. All four now resolve to the input names they always aliased, which is a pure textual identity - verified zero bare occurrences remain. Also: - SelectionSort over the buffered signals was O(n^2) with an O(n^2) count of StructToTime calls, because the comparison rebuilt both datetimes from the six int date fields every time. Now materialises the keys once and does an insertion sort; ArraySort cannot permute a struct array. IsEarlier goes with it, MakeDateTime becomes SignalTime. - Seven FileOpen sites lacked FILE_SHARE_READ|FILE_SHARE_WRITE, including AtomicWriteBegin, which stages every model save. All 43 sites now carry them - an exclusive open fails outright when another process holds the path, which here has meant a silently skipped save. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:31:36 -04:00
return FileOpen(tmpName, commonFlag | FILE_BIN | FILE_WRITE | FILE_SHARE_READ | FILE_SHARE_WRITE);
fix: make sidecar writes atomic; extract shared AtomicFile helper FileOpen(FILE_WRITE) truncates its target on open. CNet::Save already staged the .nnw through a temp file + rename for that reason, but the three sidecars written beside it did not: .stats ExpertSignalAIBase.mqh:5918 .arrows ExpertSignalAIBase.mqh:6224 .cfg ExpertSignalAIBase.mqh:7329 Two defects followed. 1. An interrupted write published a truncated sidecar. For .cfg that is the worst case: LoadAndCompareTopologyConfiguration() reads a short file as a mismatch, which discards the trained model and restarts from era 0. 2. Windows file sharing is a mutual contract - a writer opened with no FILE_SHARE_* blocks every concurrent open regardless of the reader's flags. All three read paths carry FILE_SHARE_READ|FILE_SHARE_WRITE specifically so a tester agent can read them while a live chart runs; an exclusive writer on the same path defeated that. Extracted CNet::Save's proven pattern into System\AtomicFile.mqh (AtomicWriteBegin/AtomicWriteEnd) and routed all four writers through it. This also encodes the FileMove gotcha once instead of per call site: the destination location comes from FILE_COMMON inside the 4th arg, NOT inherited from the source, and getting it wrong moves the file to the wrong sandbox silently. Also fixed while in these functions: - SaveTopologyConfiguration had 13 copy-pasted 6-line error blocks that each returned WITHOUT FileClose(handle), leaking the handle on every write failure. Collapsed to one ok-chain that closes exactly once. The on-disk field order and types are unchanged (asserted during the rewrite) so existing .cfg files still load. - SaveChartSignals documented that pruning runs only after a successful write ("a failed write above leaves both the file AND the chart untouched") but never checked any write result, so a partial write still deleted the chart objects. Results are checked now, making the existing comment true. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:31:29 -04:00
}
//+------------------------------------------------------------------+
//| Close the staged write and either publish it or discard it. |
fix: make sidecar writes atomic; extract shared AtomicFile helper FileOpen(FILE_WRITE) truncates its target on open. CNet::Save already staged the .nnw through a temp file + rename for that reason, but the three sidecars written beside it did not: .stats ExpertSignalAIBase.mqh:5918 .arrows ExpertSignalAIBase.mqh:6224 .cfg ExpertSignalAIBase.mqh:7329 Two defects followed. 1. An interrupted write published a truncated sidecar. For .cfg that is the worst case: LoadAndCompareTopologyConfiguration() reads a short file as a mismatch, which discards the trained model and restarts from era 0. 2. Windows file sharing is a mutual contract - a writer opened with no FILE_SHARE_* blocks every concurrent open regardless of the reader's flags. All three read paths carry FILE_SHARE_READ|FILE_SHARE_WRITE specifically so a tester agent can read them while a live chart runs; an exclusive writer on the same path defeated that. Extracted CNet::Save's proven pattern into System\AtomicFile.mqh (AtomicWriteBegin/AtomicWriteEnd) and routed all four writers through it. This also encodes the FileMove gotcha once instead of per call site: the destination location comes from FILE_COMMON inside the 4th arg, NOT inherited from the source, and getting it wrong moves the file to the wrong sandbox silently. Also fixed while in these functions: - SaveTopologyConfiguration had 13 copy-pasted 6-line error blocks that each returned WITHOUT FileClose(handle), leaking the handle on every write failure. Collapsed to one ok-chain that closes exactly once. The on-disk field order and types are unchanged (asserted during the rewrite) so existing .cfg files still load. - SaveChartSignals documented that pruning runs only after a successful write ("a failed write above leaves both the file AND the chart untouched") but never checked any write result, so a partial write still deleted the chart objects. Results are checked now, making the existing comment true. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:31:29 -04:00
//+------------------------------------------------------------------+
bool AtomicWriteEnd(const int handle, const string finalName, const string tmpName,
const int commonFlag, const bool ok, const string context)
{
if(handle != INVALID_HANDLE)
{
FileFlush(handle);
FileClose(handle);
}
//--- A failed/partial write must NEVER touch the real file - discard the temp, keep the last good copy.
if(!ok)
{
FileDelete(tmpName, commonFlag);
Print(context, ": write to ", tmpName, " failed - kept the existing ", finalName,
" intact (no atomic swap performed)");
return false;
}
//--- Atomic swap: FILE_REWRITE lets FileMove replace an existing destination in one operation.
ResetLastError();
if(!FileMove(tmpName, commonFlag, finalName, commonFlag | FILE_REWRITE))
{
Print(context, ": atomic rename ", tmpName, " -> ", finalName, " failed (error ",
IntegerToString(GetLastError()), ") - the temp holds the new content and the existing ",
finalName, " is unchanged");
ResetLastError();
return false;
}
return true;
}
//+------------------------------------------------------------------+
#endif // WARRIOR_ATOMIC_FILE_MQH