Tick Forge

Build custom backtesting symbols in MetaTrader 5 from your broker's own history: shift the timestamps, rescale or reshape the prices with a typed function, and test your Expert Advisor on the result in the normal Strategy Tester.

Free and open source (MIT). No DLL. No internet access. It reads your terminal's downloaded history and writes one custom symbol - nothing else.

Why

An EA that works on one stretch of price history may only work because of that stretch's price level, volatility or calendar. Tick Forge lets you ask the question directly:

• Does it still work if the same market is 6 hours earlier or 3 days later? (time offset)
• Does it still work if gold costs 1,000 instead of 4,000? (price scale)
• What if the moves were twice as big, or the trend ramped up over the period? (price function)

Run the original and the changed data side by side. If the result only exists on the original data, you have learned something before it cost you money.

Quick start

1. Install (below), attach Tick Forge to a chart.
2. Pick a source symbol. Leave the name (<symbol>_TF) or type your own.
3. Pick a date range (or Use full range), a data type, a time offset and a price function. Press Preview.
4. Press GENERATE SYMBOL. When it says DONE, the symbol is in Market Watch (group "Tick Forge").
5. In the Strategy Tester choose the new symbol. Real-tick data pairs with the model Every tick based on real ticks, M1-only data with Every tick / 1 minute OHLC / Open prices only.

Do a control run first: generate with function p and offset 0, test your EA on it, and only then test the changed data. Differences are then caused by your change, not by the pipeline.

Install

From the MQL5 Market: install the free product, then drag Tick Forge from the Navigator onto a chart.

From source:

1. Copy Tick_Forge.mq5, TF_Engine.mqh and TF_Expr.mqh into one folder under MQL5\Experts\Tick_Forge\ of your terminal (File > Open Data Folder). (market/Tick_Forge.mq5 is the same program as one file with the two headers inlined; that is what the Market compiles.)
2. Compile Tick_Forge.mq5 in MetaEditor (F7).
3. Drag Tick Forge from the Navigator onto any chart. It is a utility Expert: it never trades, so AutoTrading can stay off. (MetaTrader's window library only works in Experts and indicators, which is why it is not a script.) Closing the window removes it from the chart.

The window needs a chart of at least about 1070 x 640 pixels; if the chart is smaller you get a message on the chart. It does nothing when loaded in the Strategy Tester.

Close MetaTrader normally after generating (File > Exit, not a task kill): the terminal saves the list of custom symbols on a normal exit.

Using the window

Field	What it does
Source symbol	Any symbol in your Market Watch. Its history is the raw material.
Custom symbol name	Name of the symbol to create. An existing custom symbol asks before it is overwritten. A real broker symbol is refused.
Broker data available	First and last M1 bar your broker serves. "Scan tick availability" lists the months that have ticks.
From / to	Date range (inclusive). "Use full range" fills the whole M1 history.
Data type	Real ticks: every broker tick is transformed and written, M1 bars are rebuilt from them. Use with the tester model "Every tick based on real ticks". M1 bars only: the tester synthesizes ticks (models "Every tick", "1 minute OHLC", "Open prices only").
Time offset	Moves the raw timestamps by hours or days. Weekends and sessions move with the data (the clock is shifted, not re-aligned).
Price function	The new price as a function of the old one. See below.
Spread follows the price factor	Spread is multiplied by the same factor as the price, so the relative trading cost stays the same. Untick to keep the broker's absolute spread.
Add digits when prices shrink	If prices become much smaller, digits and tick size are increased so the spread is still resolvable.
Preview button	Draws the whole date range: the source prices on top, the result below (after the time shift and the function), each on its own price axis, with dates along the bottom. Check the shape before generating.
Bars / Ticks stored in	Where the generated data sits on disk (read-only fields you can copy).

Price functions

The function returns the new price.

Variable	Meaning
p (or x)	original price
p0	first price in the range
r	p / p0, the relative price
t	progress through the range, 0 to 1
d	days since the start of the range

Operators + - * / ^, parentheses, unary minus. Functions abs sqrt ln log10 exp sin cos pow(a,b) min(a,b) max(a,b). Constants pi, e.

Goal	Function
Unchanged (control run)	p
Everything costs twice as much	p*2
Prices rescaled to a lower level	p*0.5
Moves squared (extreme), same start price	p0*r^2
Multiplier ramps from 1 to 2 over the range	p*(1+t)
Add a fixed amount	p+1000
Mirror the moves around the start price	p0*p0/p

A function that returns zero, a negative number or an invalid value for any price stops the run with an error - nothing is written past that point. Always do a Preview first, and always run a control (p, no offset) through the same pipeline next to your changed data.

Things to know

• Only data your terminal can obtain is used. Tick history is downloaded from the broker on demand and its depth varies by broker; "Scan tick availability" shows what exists. Use "M1 bars only" for older periods.
• The custom symbol keeps the contract (size, margin, swap, sessions) of the source symbol.
• Custom symbols are stored under <data folder>\bases\Custom\. Delete one from Market Watch > Symbols > Custom.
• This is a research tool. A backtest on altered data is a stress test, not a prediction.

Audit set and the runner

The Audit set (7 symbols) button builds <name>_C (control), _S7, _S14, _S28 (shifted 7/14/28 days), _L50 (prices x0.5), _L200 (x2) and _MIR (mirrored moves) in one go, after a confirmation that states the number of symbols and days. Delete audit set removes them (only symbols Tick Forge created). Each symbol uses the disk space of one generated symbol; real ticks use far more than M1 bars. It also writes MQL5\Files\TickForgeudit_<name>.json.

MQL5 cannot start the Strategy Tester, so tools/tf_audit.py does that part on a terminal you choose and writes a comparison page (trades, net, profit factor, drawdown, share of control entries reproduced at the shifted time, flags):

python tools/tf_audit.py run --manifest audit_<name>.json --expert Folder\MyEA --terminal "<testing terminal64.exe>" --data "<its data folder>" --period M15 --model 1 --set my.set
python tools/tf_audit.py report --manifest audit_<name>.json --reports <dir with the collected reports>

Use a testing install, never a terminal logged in to a live-money account. The script refuses to start if that terminal is running and never kills anything. examples/ holds two reports made with tools/TF_DemoEA.mq5, a deliberately simple demo Expert: one trading a rule every weekday, one trading the same rule only on a fitted list of dates. Shifted data that would lie after today cannot be tested; the report then compares the overlapping part only.

Developer tools (tools/)

TF_Dev.mq5 is a developer Expert that runs the same window plus headless generation (InpAuto), a form-driven run (InpGuiRun) and screenshot mode (InpScreenshot); tf_run.py drives it on a testing terminal. build_market.py builds and compiles the single-file Market versions (Expert and indicator), make_icon.py redraws the logos, tf_run.py / shots.py drive a testing terminal headless, tf_tester_check.py + TF_TestEA.mq5 check that the Strategy Tester replays a generated symbol. Never point them at a live-money terminal.

Licence

MIT - see LICENSE. You may use, change and redistribute it, including commercially, as long as the copyright notice stays. Contributions welcome.

Author: Joshua Dunnink

Ready-made binary

release/Tick_Forge.ex5 is the compiled 1.03 Expert (utility) build of market/Tick_Forge.mq5; copy it to MQL5\Experts\ of your terminal and attach it to a chart. release/Tick_Forge_Indicator.ex5 is the same window as an indicator (copy to MQL5\Indicators\). Both compile with 0 errors and 0 warnings; you can also compile the source yourself.