A high-performance, memory-free JSON parser for MQL5, based on a flat tape architecture.
Single-pass iterative state machine (as is, without extra loops, "pure" o(n)): no recursion, no dynamic memory fragmentation, no overhead from function calls.
---
## Main Features
- **Tape-based zero-alloc model**: the entire JSON is parsed into a single contiguous `long[]` array
- **Single flat loop**: one `switch` over a token enum, no helper function calls during parsing, strictly O(n)
- **Two parsing engines**: a pure-MQL5 engine (`Parse()`, no external dependencies) and an optional SIMD engine that runs through a companion C++ DLL (`ParseSimd()`, AVX2, see [SIMD via DLL](#simd-via-dll) below) — same tape output either way, pick whichever fits your deployment
- **JIT perfect-hash key lookup**: object access starts as linear probing (`StrEquals`); once a node passes `TSN_JSON_JIT_MIN_REF_TO_HASING` accesses, a minimal perfect hash table is built for it on the fly (`k = n`, no slack), promoting that node's lookups to O(1). No FNV hashing is computed upfront during parsing.
- **Handle-based navigation**: `CJsonNode` is a lightweight struct (pointer + two ints), copying is free
- **Iterator support**: `CJsonIteratorObj` and `CJsonIteratorArray` for clean traversal
- **Mutable DOM API**: `CDomNodeBase` builds a real, editable node tree from the tape (`ToDom`) — add/remove keys, change a value's type in place, slice arrays Python-style (`PySlicing`), serialize back to JSON (`CJsonDomSerializer`), and load/dump native MQL5 structs directly (`Load()` / `GetAs()`)
- **SAX streaming parser**: `CJsonParserStream` + `ITsnJsonParserStream` feed the tape incrementally in fixed-size chunks (e.g. 64 bytes at a time) and fire `OnStartObj`/`OnKey`/`OnInteger`/... callbacks as it goes — no need to hold the whole document, or the whole tape, in memory at once
- **JSON Builder**: `CJsonBuilder`/`CJsonBuilderStr` for constructing valid JSON (or an escaped JSON string literal) directly, without an intermediate DOM
- **JSON Pointer (RFC 6901)**: `Query("/a/0/b~1c")`-style lookups on both `CJsonNode` (tape) and `CDomNodeBase` (DOM) via `CJsonPointerResolver`
- **JSONPath-like query VM**: a small register-based virtual machine (`CSLDomQueryVm`) compiles an assembly-like query language (`INS_ACCESO_KEY`, `INS_COMPARE_*`, `INS_ITER_ASSING`, custom `INS_CALL`-able functions, ...) to filter/traverse a DOM tree — think `$.libros[?(@.valor > 100)]` compiled down to a tiny bytecode loop
- **Three input formats, one API**: `AssingFile` auto-detects by extension — `.json` (plain text, needs `Parse()`), `.jsonasm` (assembly-like text dump of the tape, needs `ParseAssembly()`), `.jsontc` (precompiled binary tape, ready to use immediately, no parsing step)
- **Compiled tape cache (`.jsontc`)**: `SaveCompiled` persists the tape, the source JSON (or a reference to its file), and every perfect-hash table already built — so a reload via `AssingFile` skips parsing entirely and keeps any JIT-hashed nodes already O(1)
- **104 unit tests** in `Wf/` covering node access, the builder, DOM mutation, `.jsonasm`/`.jsontc` round-trips and malformed-input handling — runnable as a standalone script inside MetaTrader
### Parse and navigate
```mql5
#include "Src\\JsonNode.mqh"
TSN::CJsonParser parser;
parser.Assing("{\"symbol\":\"EURUSD\",\"bid\":1.2345,\"ask\":1.2347,\"active\":true}");
parser.CorrectPadding();
parser.Parse();
TSN::CJsonNode root = parser.GetRoot();
string symbol = root["symbol"].ToString();
double bid = root["bid"].ToDouble(0.0);
bool active = root["active"].ToBool(false);
```
### Build json as json string or valid json with CJsonBuilder
```mql5
TSN::CJsonBuilderStr build;
build.PutChar('"');
build.Obj();
build.Key("Valor").ValS("└");
build.Key("asas").ValS("\r\n\v\t aaaaa");
uchar data[4] = {'h', 'o', 'l', 'a'};
build.KeyWV("valora").ValU(data);
build.EndObj();
build.PutChar('"');
Print(build.Build());
/*
"{\"Valor\":\"\\u2514\",\"asas\":\"\\r\\n\\v\\t aaaaa\",\"valora\":\"hola\"}"
*/
```
### Iterate an object
```mql5
TSN::CJsonIteratorObj it = root.BeginObj();
while(it.IsValid())
{
PrintFormat("%s : %s", it.Key(), it.Val().ToString());
it.Next();
}
```
### Iterate an array
```mql5
TSN::CJsonNode arr = root["prices"];
TSN::CJsonIteratorArray it = arr.BeginArr();
while(it.IsValid())
{
Print(it.Val().ToDouble(0.0));
it.Next();
}
```
### Parse from file
```mql5
TSN::CJsonParser parser;
parser.AssingFile("data.json", false);
parser.CorrectPadding();
parser.Parse();
```
### DOM-API (Full mutable)
```mql5
void OnStart()
{
//---
string json = "{\"ae\":[[10,10],[20,20]],\"comida\":\"\\ud83d\\ude00\",\"trapo\":10.05,\"a\":true,\"invalid\":\"\\xFF\",\"precios\":[10,20,30,40,50],\"va\":1"
",\"interpolado\":\"${{valor}}\"}";
//Print(json);
//---
TSN::CJsonParser parser;
parser.Assing(json);
parser.CorrectPadding();
parser.Parse();
parser.PrintCintaTypes(0, WHOLE_ARRAY);
//---
TSN::CDomNodeManager manager;
TSN::CJsonDomSerializer serializer;
TSN::CDomNodeBase* root = new TSN::CDomNodeBase(&manager);
parser.GetRoot().ToDom(root, &manager);
//---
string kes[];
root.GetKeys(kes);
ArrayPrint(kes);
//---
Print(root.Size());
Print(root["trapo"].ToDouble(0.0));
Print(root["ae"].At(1).At(1).ToInt(0));
Print(root["comida"].ToString());
// Agregar y cambiar tipo
root["sumando"] = 20.0;
Print(root.Size());
Print(root["sumando"].ToDouble(0.0));
root["sumando"] = true;
Print(root["sumando"].ToBool(false));
//--- Estadisticas
Print("Numero de arrays creados: ", manager.m_node_size);
Print("Numero de objetos creados: ", manager.m_objs_size);
Print("Numero de strgins creados: ", manager.m_string_stack_size);
//--- Otro json
TSN::CDomNodeManager manager2;
TSN::CDomNodeBase* ptr = new TSN::CDomNodeBase(&manager2); // vacio
ptr.NewObj(128);
ptr["valores"] = 20.0;
//---
root.Set("nueva_key", ptr); // No copia profunda si no PUNTERO..
ptr["coma"] = true;
TSN::CDomNodeBase* out[];
const int t = root["precios"].PySlicing(out, 0, -1, -1, -1);
for(int i = 0; i < t; i++)
Print("valor: ", out[i].ToInt(0)); // al revez 50,40,30,20,10
//---
Print(serializer.Serialize(root));
Print(CharArrayToString(serializer.m_buf, 0, serializer.m_buf_s));
//---
delete root;
delete ptr;
}
/*
2026.07.20 09:19:07.290 Dom (EURUSD,H1)
{"a":true,"invalid":"\\xFF","precios":[10,20,30,40,50],"va":1,"comida":"\ud83d\ude00","interpolado":"${{valor}}",
"sumando":true,"nueva_key":{"valores":20.00000000,"coma":true},"ae":[[10,10],[20,20]],"trapo":10.05000000}
*/
```
### DOM API From and ToJson
```mql5
struct Libro
{
string name;
string titulo;
double valor;
//---
Libro() {}
Libro(TSN::CDomNodeBase& obj)
{
name = obj["name"].ToString();
titulo = obj["titulo"].ToString();
valor = obj["valor"].ToDouble(0.00);
}
static void ToTsnDomNode(TSN::CDomNodeBase& obj, const Libro& other)
{
obj.NewObj(16);
obj["name"] = other.name;
obj["titulo"] = other.titulo;
obj["valor"] = other.valor;
}
};
void OnStart()
{
//---
TSN::CDomNodeManager manager;
Libro l;
l.name = "hola";
l.titulo = "titulo";
l.valor = 1.0;
// Load
TSN::CJsonDomSerializer ser;
TSN::CDomNodeBase node(&manager);
node.Load(l);
Print(ser.Serialize(&node));
Print(CharArrayToString(ser.m_buf, 0, ser.m_buf_s)); // {"name":"hola","titulo":"titulo","valor":1.00000000}
//---
TSN::CJsonBuilder builder;
builder.Obj();
builder.KeyWV("name").ValSWV("hola");
builder.KeyWV("titulo").ValSWV("como");
builder.KeyWV("valor").Val(10.0490);
builder.EndObj();
//---
TSN::CJsonParser parser;
ArrayCopy(parser.m_raw, builder.m_buf, 0, 0, builder.m_pos);
parser.CalcLen();
parser.CorrectPadding();
parser.Parse();
// Dom building
TSN::CDomNodeBase node2(&manager); // le pasamos su manager
parser.GetRoot().ToDom(&node2, &manager);
//---
Libro l2 = node2.GetAs();
Print(l2.name); // DomSer (EURUSD,H1) hola
Print(l2.titulo); // DomSer (EURUSD,H1) como
Print(l2.valor); // DomSer (EURUSD,H1) 10.049
}
```
### Json Pointer (DOM and CJsonNode API)
```mql5
//---
TSN::CJsonParser parser;
parser.Assing(g_json);
parser.CorrectPadding();
parser.Parse();
TSN::CJsonNode root = parser.GetRoot();
//---
Print("--- CJsonNode ---");
Print(root.Query("/config~1general/riesgo~0maximo").ToDouble(0.0));
Print(root.Query("/raro~1~0mix~0~11").ToString(""));
Print(root.Query("/ordenes/2/tags/2").ToString(""));
//---
TSN::CDomNodeManager manager;
TSN::CDomNodeBase node(&manager);
root.ToDom(&node, &manager);
//---
CJsonPointerResolver resolver;
resolver.Base(&node);
Print("--- CJsonPointerResolver ---");
Print(resolver.Query("/config~1general/riesgo~0maximo").ToDouble(0.0));
Print(resolver.Query("/raro~1~0mix~0~11").ToString(""));
Print(resolver.Query("/ordenes/2/tags/2").ToString(""));
/*
Pointer (EURUSD,M1) --- CJsonNode ---
Pointer (EURUSD,M1) 2.5
Pointer (EURUSD,M1) combinacion rara
Pointer (EURUSD,M1) riesgo/alto
Pointer (EURUSD,M1) --- CJsonPointerResolver ---
Pointer (EURUSD,M1) 2.5
Pointer (EURUSD,M1) combinacion rara
Pointer (EURUSD,M1) riesgo/alto
*/
```
### Stream-With SAX Parse
```mql5
class CTestStream : public TSN::ITsnJsonParserStream
{
public:
CTestStream(void) {}
~CTestStream(void) {}
void OnStartDocument() override final { Print("Inicio de documento"); }
void OnEndDocument() override final { Print("Fin de documento"); }
void OnStartObj() override final { Print("Inicio de objeto: ", m_ctx.CurrentStack()); }
void OnEndObJ() override final { Print("Fin de objeto: ", m_ctx.CurrentStack()); }
void OnStartArr() override final { Print("Inicio de array: ", m_ctx.CurrentStack()); }
void OnEndArr() override final { Print("Fin de array: ", m_ctx.CurrentStack()); }
void OnInteger(long v) override final { Print("Numero: ", v); }
void OnReal(double v) override final { Print("Real: ", v); }
void OnString(int s, int len) override final { Print("Texto: ", m_ctx.Unescape(s, s + len - 1)); }
void OnBool(bool v) override final { Print("Boleano: ", v); }
void OnNull() override final { Print("Null"); }
void OnKey(int s, int len) override final { Print("Clave: ", m_ctx.Unescape(s, s + len - 1)); }
};
//+------------------------------------------------------------------+
void OnStart()
{
//---
TSN::CJsonParserStream stream;
stream.SetCaller(new CTestStream(), true);
//---
string json = "{\"ae\":[[10,10],[20,20]],\"comida\":\"\\ud83d\\ude00\",\"trapo\":10.05,\"a\":true,\"invalid\":\"\\xFF\",\"precios\":[10,20,30,40,50],\"va\":1"
",\"interpolado\":\"${{valor}}\"}";
uchar raw[];
const int l = StringToCharArray(json, raw);
int p = 0;
// Simular pasadas de 64 bytes
while(p < l) // tien que ser menor, en un caso real quizas ahsta on end document
{
//--- le damos datos
// Tambien antes de dar los datos odemos llamar a CheckLimpieza
// Para que mueva pos a posicion 0 (reutlziacion) en caso se pueda si no seguir acomulando
// Ahora en este caso vamos sumando dado que m_len neceista crecer.... luego quizas se le peude reinicar..
stream.m_len += ArrayCopy(stream.m_raw, raw, stream.m_pos, p, 64); //
// intermante va parseando aumentado m_pos
// Otro si no da el documento para otro el bulce se dentecia aunqeu array copy ya menaja eso intemante
// si nos pamaos normaliza asi que no hay problema pero eso un aviso..
stream.ParseSreamSax();
// Aqui por ejemplo se peuden hacer mas cosas.. tu lo llamas cuando quieres
p += 64; // ya le dimos 64
}
}
```
### Json Patch full Fast parse and VM (register-based)
```mql5
void JsonPathSize(TSN::CSLDomQueryVm* state)
{
TSN::CDomNodeBase* ptr = state.FReadNode(0);
state.FPushInteger(ptr.Size());
}
void OnStart()
{
//---
const string json = "{\n"
"\"libros\":{\n"
"\"libro1\": {\"valor\":542,\"sumando\":[],\"comando\":30},\n"
"\"libro2\": {\"valor\":19,\"sumando\":[30],\"comando\":2},\n"
"\"libro3\": {\"valor\":452,\"sumando\":[20,50,65,89,419],\"comando\":7}\n"
" }\n"
"}";
//---
TSN::CJsonParser parser;
parser.Assing(json);
parser.CorrectPadding();
parser.Parse();
//---
TSN::CDomNodeManager manager;
TSN::CDomNodeBase node(&manager);
parser.GetRoot().ToDom(&node, &manager);
//---
TSN::CSLDomQueryVm vm;
vm.SetFunction("size", JsonPathSize);
// $.libros[?(@.comando * 20 < @.valor && @.sumando.size()].sumando[*]
const string asm =
"%reg 0 = 20 \n"
"%reg 1 = 1\n"
"INS_ACCESO_KEY_F K\"libros\"\n"
"INS_INICIAR_ITERACION 1 ; Solo se permiten objetos en esta iteracion\n"
"@Iteracion: ; Jump aqui\n"
" INS_ITER_ASSING 'Final' ; En caso falle salta a Final y terminamos .. \n"
" INS_ACCESO_KEY 'Iteracion' K\"comando\" ; Accedemos a comando \n"
" INS_SAVE 256 'Iteracion' ; Lo guardamos en el registro 256 si falla reset\n"
" INS_M_MUL 256 0 256 'Iteracion' ; Mulñtiplicamos el registro 256 por el 0 y lo guardamos en 256\n"
" INS_ACCESO_KEY 'Iteracion' K\"valor\" ; Accede a valor\n"
" INS_SAVE 257 'Iteracion'; Guardamos su valor actual\n"
" INS_COMPARE_MENOR 256 257 1 'Iteracion' ; Comparamos registros\n"
" INS_ACCESO_KEY 'Iteracion' K\"sumando\" ; Accedemos a sumando \n"
" INS_SAVE_NO_RESET 256 'Iteracion'; guadcamos en sumand\n"
// Nota que aqui el resultado se pondra apartir del 258 reg dado qeu el 257 se pone auto
// el numero de parametros escritos...
" INS_CALL 256 1 'size'; LLamamos a la funcino size y sus parametros emppizan en 256 y tiene 1 parametro a leer\n"
// Nota que esta verirficacion quizas no haga falta.. aunque si esperamos exactamentwe N parameotrs
// o digamos 1 quizas si si no se nos desalinea... nuestra lecutra.. auqnue quizas nos podemos saltar a cierta
// direccion daod que sabemos que tenemos n parmaeotrs en el reg 257
// A futuro si veo que se requiere separar o casos mas complejos como iterar sobre los parametros obtenidos
// ahi si si usariosm offsets... ahi quizas lo agrego no es complejo es cosa de cambiar layort y nueva sintaxis de asm
// pero ya eso creo que tiraraimso ya a una vm casi de un lenguaje solo faltura stack frames y ya por que en teoria
// ahora mismo podemos tener varaibles, bucles, con los ifs, etc...
" INS_COMPARE_EQ 257 1 1 'Iteracion' ; Verificamos que el numero de retornanos es 1\n"
" INS_COMPARE_IS 258 'Iteracion' 1 ; Usamos is para comprar su valor\n"
" ; Si llegamos hasta aqui pasamos todo entonces copy all\n"
" INS_COPY_ALL 'Iteracion' \n"
" INS_JUMP 'Iteracion'; Ahora nos dirigimos devuelva a iteracion \n"
"@Final: \n"
" INS_SALIDA ; salimos"
;
//---
vm.Assing(asm);
vm.CorrectPadding();
vm.CompileAsm();
vm.LockValues();
vm.BaseNode(&node);
//---
vm.Summary();
//---
TSN::CDomNodeBase* elements[];
vm.Run(elements);
//---
const int t = vm.m_el_s;
for(int i = 0; i < t; i++)
{
Print(elements[i].ToInt(0));
}
//---
/* Output:
JPath (EURUSD,M1) 20
JPath (EURUSD,M1) 50
JPath (EURUSD,M1) 65
JPath (EURUSD,M1) 89
JPath (EURUSD,M1) 419
*/
}
```
### Compile and reload a `.jsontc` tape cache
`SaveCompiled` writes the parsed tape (plus the source JSON and every perfect-hash table already built) to a binary `.jsontc` file. Reloading it via `AssingFile` skips parsing entirely:
```mql5
//--- Compile once
TSN::CJsonParser parser;
parser.Assing(my_json_string);
parser.CorrectPadding();
parser.Parse();
parser.GetRoot()["symbol"]; // (optional) warm up perfect-hash tables before saving
parser.SaveCompiled("data.jsontc", false);
//--- Reload later — no Parse()/ParseAssembly() needed, tape and hash tables are restored as-is
TSN::CJsonParser parser2;
parser2.AssingFile("data.jsontc", false);
TSN::CJsonNode root = parser2.GetRoot();
```
There's also an overload that stores a reference to an external JSON file instead of embedding it:
```mql5
parser.SaveCompiled("data.jsontc", false, "data.json", false);
```
---
## SIMD via DLL
For MQL5, `Parse()` is fast but limited to what a script/EA can do in pure MQL5. `ParseSimd()` offloads phase 1 (finding every structural character: `{`, `}`, `[`, `]`, `:`, `,`, string bounds) to a small companion C++ DLL that uses AVX2 (SWAR bit-tricks for backslash/quote detection, `pclmulqdq` for the in-string prefix-xor, `popcnt`/`tzcnt` to walk set bits) — phase 2 (building the tape) still runs the same flat state machine. Same `long[]` tape as `Parse()`; only the structural scan changes.
```mql5
#define JSONPARSERBYLEO_DLL
#define JSONPARSERBYLEO_DLL_AVX2
#include "Src\\JsonNode.mqh"
TSN::CJsonParser parser;
parser.Assing(my_json_string);
parser.CorrectPadding(); // pads to a 64-byte boundary when JSONPARSERBYLEO_DLL is defined
if(parser.ParseSimd())
{
TSN::CJsonNode root = parser.GetRoot();
}
```
**Building the DLL:** source is in `Src/DLL/` (`Parser.cpp` + CMake project), requires MSVC + a AVX2-capable CPU. Build it
(`CMakeLists.txt` targets `x64-windows-static`, C++23, `/arch:AVX2`) and drop the resulting `JsonParserByLeo-Avx2.dll` into your terminal's
`MQL5/Libraries` folder. Only needed if you want the SIMD path — the pure-MQL5 `Parse()` has no external dependency.
---
## Performance
JsonParserByLeo is a fast JSON library. You can see the results of my "mini-investigation" of JSON libraries where
I tested and benchmarked the parsing and node access capabilities of the most popular MQL5 libraries.
I also included parsing only for libraries from other languages.
- [Click here](./Test/Ben/README.md)
---
## Repository Structure
```
JsonParserByLeo/
├── BenOther/ # Reference benchmarks in other languages (C++/simdjson, Python, Rust)
├── Src/ # Full MQL5 source (Parser, Node, DOM, SAX, Builder, Pointer, JsonPath VM)
│ └── DLL/ # Optional companion C++ DLL: AVX2 structural scan (Parser.cpp + CMake project)
├── Test/ # Manual tests and benchmarks (Test/Ben/), run as MQL5 scripts
└── Wf/ # 104 automated unit tests (Wf/Test.mq5), run as a standalone script
```
---
## License
**[Read Full License](./LICENSE)**
By downloading or using this repository, you accept the license terms.
---
## Requirements
See [dependencies.json](./dependencies.json) for the full list.
---
## Installation
```bash
cd "C:\Users\YOUR_USER\AppData\Roaming\MetaQuotes\Terminal\YOUR_ID\MQL5\Shared Projects"
tsndep install "https://forge.mql5.io/nique_372/JsonParserByLeo.git"
```
Requires the `tsndep` package, available on [PyPI](https://pypi.org/project/tsndep). It automatically downloads and installs all declared dependencies.
---
## Quick Start
**1. Include the library:**
```mql5
#include "..\\JsonParserByLeo\\Src\\JsonNode.mqh"
```
**2. Parse and access:**
```mql5
TSN::CJsonParser parser;
parser.Assing(my_json_string);
parser.CorrectPadding();
if(parser.Parse())
{
TSN::CJsonNode root = parser.GetRoot();
double price = root["price"].ToDouble(0.0);
}
```
**3. Re-parse without re-copying (benchmark pattern):**
```mql5
parser.Assing(raw_string); // copy once
parser.CorrectPadding();
for(int i = 0; i < 1000; i++)
parser.Parse(); // re-parse in-place
```
---
## Roadmap
- Add suport for JSONPath [In progres 100% ASM ready] [Added 24/7/26]
- Add suport for (deserialize\serialize native structs\objs) [Added 23/7/26]
- Add Optimizer and Compiler for JsonPath queryes
---
## Contact
- **Platform:** [MQL5 Community](https://www.mql5.com/es/users/nique_372)
- **Profile:** https://www.mql5.com/es/users/nique_372
- **Articles:** https://www.mql5.com/es/users/nique_372/publications