Tick_Forge/market/article/ARTICLE_BODY.html

<!-- MQL5.community article body. Paste in the editor's HTML mode (the title is typed in the Subject field).
     Layout: one fixed-width column (table width=700, an attribute, so it survives CSS stripping), same
     approach as the Market description. Sections are h3; paragraphs are justified, 3-5 lines, short sentences.
     Images: PNG, 700 px wide, uploaded with the Image button (placeholders below).
     Code: SRC button after MetaEditor Styler (placeholders below, files in snippets/).
     No CSS, no page wrapper, no figure tags. -->
<table width="700" cellpadding="0" cellspacing="0" border="0"><tr><td>

<h3>Introduction</h3>

<p style="text-align: justify;">A backtest shows what an Expert Advisor did on one stretch of history. It does not show why. Some Experts look perfect because part of their strategy is a memory of that history.</p>

<p style="text-align: justify;">That memory can be a list of dates, a table of times or fixed price levels. On the original data such an Expert looks brilliant. Live, there is no list for the future.</p>

<p style="text-align: justify;">There is a simple check. Change the history in a controlled way and run the Expert again. Logic survives the change. A memorised list does not.</p>

<p style="text-align: justify;">In this article we build an MQL5 tool that makes such changed history from the broker's real ticks. Then we use it to test Experts. The tool is free and open source. It never trades, uses no DLL and has no internet access.</p>

<p>&nbsp;</p>

<h3>Why not just resample the trades</h3>

<p style="text-align: justify;">A common stress test takes the trades an Expert already made. It re-orders or re-samples them many times. The result is a set of equity curves.</p>

<p style="text-align: justify;">That is the same trades, rearranged. It does not create a new market. An Expert that memorised its history passes it as easily as before.</p>

<p style="text-align: justify;">Our approach changes the price data itself. Every tick we write is a real tick from the broker. The order of the ticks, the volatility clusters, the gaps and the spreads are real. Only the date and the price level change.</p>

<p style="text-align: justify;">We do not invent random price paths. We do not simulate possible futures.</p>

<p>&nbsp;</p>

<h3>What the tool does</h3>

<p style="text-align: justify;">The tool is a utility Expert with a window. You choose a source symbol and a date range. You set a time offset and a price function. Then you press one button.</p>

<p style="text-align: justify;">The result is an ordinary custom symbol. It appears in Market Watch and in the Strategy Tester.</p>

<p style="text-align: justify;">The price function returns the new price. It can use these variables:</p>

<ul>
<li>p - the price</li>
<li>p0 - the first price in the range</li>
<li>r - p divided by p0</li>
<li>t - progress through the range, from 0 to 1</li>
<li>d - days since the start</li>
</ul>

<p style="text-align: justify;">Examples are p*2, p*0.5, p0*r^2 and p0*p0/p. The time offset is in hours or days, and a negative value moves the data earlier.</p>

<p>&nbsp;</p>

<p style="text-align: center;">[INSERT IMAGE with the Image button: images/Figure1_Tick_Forge_window.png]</p>
<p style="text-align: center;"><small>Figure 1. The window: settings on the left, whole-range preview on the right</small></p>

<p>&nbsp;</p>

<p>&nbsp;</p>

<h3>Creating the custom symbol</h3>

<p style="text-align: justify;">MQL5 has built-in functions for custom symbols. We create the symbol with <a href="https://www.mql5.com/en/docs/customsymbols/customsymbolcreate" title="CustomSymbolCreate in the MQL5 documentation">CustomSymbolCreate</a>. The source symbol is the template.</p>

<p style="text-align: justify;">If the symbol exists and the tool made it, we delete its old bars and ticks first. A real broker symbol is never overwritten.</p>

<p style="text-align: justify;">One detail matters for testing. The new symbol must accept the same order types as the source. Otherwise the tester rejects every order with "unsupported filling mode".</p>

<p style="text-align: justify;">So we copy the filling, order, expiration and execution modes from the source. We use <a href="https://www.mql5.com/en/docs/customsymbols/customsymbolsetinteger" title="CustomSymbolSetInteger in the MQL5 documentation">CustomSymbolSetInteger</a> for this.</p>

<p>&nbsp;</p>

<p style="text-align: center;">[INSERT CODE with the SRC button: snippets/Snippet1_CreateSymbol.mq5]</p>

<p>&nbsp;</p>

<p>&nbsp;</p>

<h3>Transforming the ticks</h3>

<p style="text-align: justify;">For each day we read the real ticks with CopyTicksRange. Each bid goes through the price function. The ask is rebuilt as the new bid plus the spread.</p>

<p style="text-align: justify;">By default the spread is multiplied by the same factor as the price. The relative trading cost then stays the same. The timestamp moves by the offset, in milliseconds.</p>

<p style="text-align: justify;">The M1 bars are rebuilt from the transformed ticks on the shifted clock. So bars and ticks always agree.</p>

<p style="text-align: justify;">We write the day with <a href="https://www.mql5.com/en/docs/customsymbols/customticksreplace" title="CustomTicksReplace in the MQL5 documentation">CustomTicksReplace</a> and the bars with CustomRatesReplace. If the price level changes a lot, digits, tick size and tick value are adjusted too.</p>

<p>&nbsp;</p>

<p style="text-align: center;">[INSERT CODE with the SRC button: snippets/Snippet2_TransformTick.mq5]</p>

<p>&nbsp;</p>

<p>&nbsp;</p>

<h3>The price function compiler</h3>

<p style="text-align: justify;">The price function is text typed by the user, such as p0*r^2. We must turn it into numbers for millions of ticks. Calling a general expression evaluator for every tick would be slow. It would also be unsafe, because the text comes from outside.</p>

<p style="text-align: justify;">So we write a small compiler. It reads the text once and checks it. It produces a short program in reverse-polish form. There is no eval, no file access and no side effects. A wrong function is rejected before anything is written.</p>

<p style="text-align: justify;">The parser is a recursive descent parser. It has one level for plus and minus, one for multiply and divide, one for the unary minus, one for the power, and one for numbers, variables, functions and brackets. The usual order of operations follows from this layout.</p>

<p style="text-align: justify;">At run time the program uses a small stack of 64 values. Each tick costs only a few simple operations. A division by zero, or a result that is not a valid number, gives NaN. The engine checks for it and stops with a clear message. It never writes a bad price.</p>

<p>&nbsp;</p>

<p style="text-align: center;">[INSERT CODE with the SRC button: snippets/Snippet4_ExpressionEval.mq5]</p>

<p>&nbsp;</p>

<p style="text-align: justify;">The engine also rejects a function that returns zero or a negative price anywhere in the range. A preview checks this on the whole range before you generate. So you see the problem early, and the disk stays clean.</p>

<p>&nbsp;</p>

<h3>Keeping the terminal responsive</h3>

<p style="text-align: justify;">A multi-day tick run takes minutes. If it all happens in one event, the terminal is blocked.</p>

<p style="text-align: justify;">So we split the work into steps. Each step writes one day. A timer calls the step. Between steps the window can redraw and react to the mouse.</p>

<p style="text-align: justify;">This also works in an indicator, where Sleep and MessageBox are not allowed. A fresh custom symbol rejects its first write for a short time. The step does not sleep. It waits until a ready time has passed.</p>

<p style="text-align: justify;">The overwrite question is a second click on the same button, not a dialog.</p>

<p>&nbsp;</p>

<p style="text-align: center;">[INSERT CODE with the SRC button: snippets/Snippet3_StepAndTimer.mq5]</p>

<p>&nbsp;</p>

<p>&nbsp;</p>

<h3>Building the window</h3>

<p style="text-align: justify;">The window uses the standard library class CAppDialog. This class works in Experts and indicators. It does not work in scripts. A script that creates it stops with the error "invalid program type". That is why the tool is an Expert and not a script.</p>

<p style="text-align: justify;">The window has a combo box for the source symbol, edit fields for the name, dates, offset and function, check boxes for the spread and digits options, and buttons. A second combo box offers presets such as multiply by two or squared move. The preset only fills the function field.</p>

<p style="text-align: justify;">We found two limits of the standard controls. A label and an edit field cut off long text at a fixed width, and an edit field does not scroll to the end. That hides long paths and long messages. So the long text is drawn on a canvas instead.</p>

<p style="text-align: justify;">We wrote a small class on top of CCanvas. It creates a bitmap label inside the dialog and follows the dialog when it moves. We use it for three things: the formula hint, the storage paths and the wrapped status messages.</p>

<p style="text-align: justify;">The same class draws the preview. It has two stacked panels. The source prices are on top. The result is below, with its dates shifted by the offset. Each panel has its own price axis, so a large rescale stays readable.</p>

<p style="text-align: justify;">The preview picks the bar size from the length of the range. Up to 3 days it uses M5. Up to 14 days it uses M30, up to 120 days H1, up to 700 days H4 and above that D1. It plots at most 600 points, so the whole range is always visible.</p>

<p>&nbsp;</p>

<h3>The audit set</h3>

<p style="text-align: justify;">Building each symbol by hand is slow. So the window has an Audit set button. After a confirmation it creates seven symbols from one source:</p>

<ul>
<li>a control with no change</li>
<li>shifts of 7, 14 and 28 days</li>
<li>prices multiplied by 0.5 and by 2</li>
<li>mirrored moves</li>
</ul>

<p style="text-align: justify;">The confirmation states the number of symbols and days first. Nothing is created silently.</p>

<p style="text-align: justify;">Each symbol uses the disk space of one generated symbol. Real ticks need far more space than M1 bars. A Delete audit set button removes the set again. It touches only symbols the tool created.</p>

<p style="text-align: justify;">The set also writes a small manifest file to MQL5\Files\TickForge. It lists the symbols of the set.</p>

<p style="text-align: justify;">A shift of whole weeks is the key test. Every tick keeps its weekday and its time of day. Only the date changes.</p>

<p style="text-align: justify;">An Expert built on prices, weekdays and times of day should trade much as before. An Expert that depends on specific dates will not.</p>

<p>&nbsp;</p>

<p style="text-align: center;">[INSERT IMAGE with the Image button: images/Figure2_Shift_7_days.png]</p>
<p style="text-align: center;"><small>Figure 2. Same prices, seven days later: the dates move, the shape is identical</small></p>

<p>&nbsp;</p>

<p>&nbsp;</p>

<h3>Why the tester cannot be started from MQL5</h3>

<p style="text-align: justify;">An MQL5 program cannot launch the Strategy Tester. The only automatic route is to start the terminal with a configuration file. That needs a process call, which means a DLL or a shell call. We use neither.</p>

<p style="text-align: justify;">So you run your Expert on each symbol in the tester yourself. Start with the control. Use identical settings for every run.</p>

<p style="text-align: justify;">To automate it, the repository has a free Python script. It reads the manifest and writes a tester configuration for each symbol. It starts the terminal you name, collects the reports and writes one comparison page.</p>

<p style="text-align: justify;">The terminal can be started with a <a href="https://www.metatrader5.com/en/terminal/help/start_advanced/start" title="Starting MetaTrader 5 with a configuration file">configuration file</a>. Use a testing terminal. Never use one that is logged in to a live account.</p>

<p>&nbsp;</p>

<h3>A step-by-step test run</h3>

<p style="text-align: justify;">Here is the full procedure for one Expert. We assume you have the Expert and its settings file, and a testing terminal. First, open the tool on a chart. Choose the source symbol and a range of at least a few months. A short range gives too few trades to compare.</p>

<ol>
<li>Choose M1 bars only for long ranges, or real ticks for a precise test. Real ticks need much more disk space and time.</li>
<li>Press Audit set and confirm. Wait until the status says DONE.</li>
<li>Close MetaTrader normally with File and Exit. The terminal saves its list of custom symbols only on a normal exit.</li>
<li>Start the terminal again and open the Strategy Tester. Select your Expert and load its settings.</li>
<li>Select the control symbol, which ends in _C. Use the model that matches the data: Every tick based on real ticks for real ticks, or a bar model for M1 bars.</li>
<li>Set the same deposit, leverage and chart period for every run. Run the test and save the report.</li>
<li>Repeat for the other symbols. For a shifted symbol, move the test dates by the same shift. For +7 days, start and end one week later.</li>
</ol>

<p style="text-align: justify;">Then compare the reports. Look at the number of trades first. Then look at the net profit, the profit factor and the drawdown. The Python script does these steps for you and writes the comparison.</p>

<p style="text-align: justify;">If the control and the 7-day shift differ a lot, do not stop at the numbers. Open the list of trades. Find the entries that exist in one run and not in the other. Check whether they fall on particular dates. That is where the explanation is.</p>

<p>&nbsp;</p>

<h3>Reading the results</h3>

<p style="text-align: justify;">The report shows trade count, net profit, profit factor and drawdown for each symbol. It also shows how many of the control's entries reappear at the shifted time.</p>

<p style="text-align: justify;">The flags are rules of thumb, not statistics. One detail is easy to miss. Shifted data cannot lie after today. So the report compares only the overlapping part.</p>

<p style="text-align: justify;">To check the chain, we wrote a simple demo Expert. We ran it on 57 days of gold M1 data. It has two modes.</p>

<ul>
<li>Mode one trades one fixed rule every weekday.</li>
<li>Mode two trades the same rule only on a hard-coded list of dates. The list holds the days that won on the control run. That is what a memorised schedule looks like.</li>
</ul>

<p style="text-align: justify;">The results for the weekday rule:</p>

<ul>
<li>The control made 39 trades with a net of -2046.00.</li>
<li>The 7-day shift gave the same 39 trades and the same net.</li>
<li>No flag fired.</li>
</ul>

<p style="text-align: justify;">The results for the fitted date list:</p>

<ul>
<li>The control shows +929.50. The same rule loses money on every weekday.</li>
<li>Under a 7-day shift only 30% of the entries reappear. The net is -691.40.</li>
<li>Under 14 days the net is -705.00. Under 28 days it is -579.30.</li>
<li>The report flags all three.</li>
</ul>

<p style="text-align: justify;">The demo Expert is only a teaching example. It is not a statement about any product.</p>

<p style="text-align: justify;">The price level tests work the same way. Multiply the prices by 0.5 or by 2 and run again. Fixed price levels, and distances that fit only one price, stop working.</p>

<p>&nbsp;</p>

<p style="text-align: center;">[INSERT IMAGE with the Image button: images/Figure3_Prices_x0_5.png]</p>
<p style="text-align: center;"><small>Figure 3. Every price multiplied by 0.5</small></p>

<p>&nbsp;</p>

<p>&nbsp;</p>

<p style="text-align: center;">[INSERT IMAGE with the Image button: images/Figure4_Mirrored_moves.png]</p>
<p style="text-align: center;"><small>Figure 4. Mirrored moves over a long range</small></p>

<p>&nbsp;</p>

<p>&nbsp;</p>

<h3>Checking that the data is right</h3>

<p style="text-align: justify;">A test tool is only useful if its data is correct. So we checked the generated symbols against the source. The first check used gold on the test terminal. The range was two days in October 2026. The offset was +24 hours and the function was p*2. The mode was real ticks.</p>

<ul>
<li>The tool wrote 1,470,186 ticks and 2,748 M1 bars.</li>
<li>The first minute held 445 ticks in the source and 445 in the result.</li>
<li>The first bid went from 4000.13 to 8000.26. The ask went from 4000.41 to 8000.82, so the spread doubled with the price.</li>
<li>Every timestamp moved by exactly 24 hours.</li>
</ul>

<p style="text-align: justify;">We then ran a test Expert in the Strategy Tester on the new symbol, with the model Every tick based on real ticks. The tester replayed 1,470,184 of the 1,470,186 ticks. The first tick was at 00:00:00 on the shifted day with a bid of 8000.26. The average spread was 0.538.</p>

<p style="text-align: justify;">The second check used M1 bars only. The function was p0*r^2 with a shift of -6 hours over eight days. The tool wrote 8,299 bars. An open price of 4136.40 became 4130.59. The first price p0 was 4142.22, so r is about 0.9986 and the square gives exactly that value.</p>

<p style="text-align: justify;">These checks do not prove that every case is right. They show that the pipeline does what the text says on real data. We repeat the check with the control symbol and the function p, which must give the same numbers as the source apart from the shift.</p>

<p>&nbsp;</p>

<h3>Limits</h3>

<p style="text-align: justify;">This is a stress test. It is not a forecast and not a trading result.</p>

<p style="text-align: justify;">A pass means the Expert did not depend on what we changed. It does not prove that the Expert is safe or profitable.</p>

<p style="text-align: justify;">A failed test is a question, not a verdict. Honest Experts can depend on sessions, weekdays or round price levels. Look at which trades changed, and why.</p>

<p style="text-align: justify;">The custom symbol keeps the contract settings of the source symbol. Its name differs from the real one. An Expert that runs only on one exact symbol name will refuse to run. That is also information.</p>

<p style="text-align: justify;">Tick history depth depends on the broker. Use M1 bars for older periods.</p>

<p style="text-align: justify;">Some practical points save time:</p>

<ul>
<li>Close MetaTrader normally after generating. A forced close can lose the list of custom symbols.</li>
<li>If the tester rejects every order, check that the symbol accepts the same filling mode as the source.</li>
<li>Shifted data cannot lie after today. End the range earlier for a full comparison of the larger shifts.</li>
<li>The broker symbol name differs between brokers, for example XAUUSD or GOLD.pro. Choose it from the list.</li>
</ul>

<p>&nbsp;</p>

<h3>Conclusion</h3>

<p style="text-align: justify;">We built a free tool that makes changed history from real broker ticks. It has a time shift and a price function.</p>

<p style="text-align: justify;">We saw how the symbol is created and how the ticks are transformed. We saw how a timer keeps the terminal responsive. We saw how a set of test symbols exposes dependence on dates or price levels.</p>

<p style="text-align: justify;">You now have working MQL5 code and a repeatable protocol. Run a control. Run a shift of whole weeks. Run a price rescale. Then compare.</p>

<p style="text-align: justify;">The source code is in the <a href="https://forge.mql5.io/joshuadunnink/Tick_Forge" title="Tick Forge source code on the MQL5 forge">Tick Forge repository</a>. The compiled tool is also available in the MQL5 Market as a free utility.</p>

<p>&nbsp;</p>

<h3>Files used in the article</h3>

<ul>
<li><a href="https://forge.mql5.io/joshuadunnink/Tick_Forge/src/branch/main/market/Tick_Forge.mq5" title="Single-file Tick Forge Expert on the MQL5 forge">Tick_Forge.mq5</a> - the utility Expert with the window, in one file.</li>
<li><a href="https://forge.mql5.io/joshuadunnink/Tick_Forge/src/branch/main/TF_Engine.mqh" title="Engine source on the MQL5 forge">TF_Engine.mqh</a>, <a href="https://forge.mql5.io/joshuadunnink/Tick_Forge/src/branch/main/TF_Gui.mqh" title="Window source on the MQL5 forge">TF_Gui.mqh</a> and <a href="https://forge.mql5.io/joshuadunnink/Tick_Forge/src/branch/main/TF_Expr.mqh" title="Price function compiler source on the MQL5 forge">TF_Expr.mqh</a> - the same program split into the engine, the window and the price function compiler.</li>
<li><a href="https://forge.mql5.io/joshuadunnink/Tick_Forge/src/branch/main/tools/tf_audit.py" title="Audit runner and report script on the MQL5 forge">tf_audit.py</a> - the runner and report script.</li>
<li><a href="https://forge.mql5.io/joshuadunnink/Tick_Forge/src/branch/main/tools/TF_DemoEA.mq5" title="Demo Expert source on the MQL5 forge">TF_DemoEA.mq5</a> - the demo Expert used in the test.</li>
<li><a href="https://forge.mql5.io/joshuadunnink/Tick_Forge/src/branch/main/examples" title="Example reports on the MQL5 forge">examples folder</a> - the two example reports and the manifest from the demo runs.</li>
</ul>

</td></tr></table>