Fast JSON parser from the TSN ecosystem
  • MQL5 93.4%
  • MQL4 6.6%
Vai al file
Nique_372 82bfb131d3
2026-07-21 10:38:56 -05:00
Src 2026-07-21 10:38:19 -05:00
Test 2026-07-20 12:04:54 -05:00
Wf 2026-07-21 10:38:19 -05:00
dependencies.json 2026-07-19 07:41:52 -05:00
JsonParserByLeo.mqproj Generated by MQL5 Wizard 2026-06-02 15:28:54 -05:00
LICENSE Añadir LICENSE 2026-06-03 02:26:21 +00:00
README.md 2026-07-21 10:38:56 -05:00

A high-performance, memory-free JSON parser for MQL5, based on a flat tape architecture.
Single-pass iterative state machine (as is, without extra loops, "pure" o(n)): no recursion, no dynamic memory fragmentation, no overhead from function calls.


Main Features

  • Tape-based zero-alloc model: the entire JSON is parsed into a single contiguous long[] array
  • Single flat loop: one switch over a token enum, no helper function calls during parsing, strictly O(n)
  • JIT perfect-hash key lookup: object access starts as linear probing (StrEquals); once a node passes TSN_JSON_JIT_MIN_REF_TO_HASING accesses, a minimal perfect hash table is built for it on the fly (k = n, no slack), promoting that node's lookups to O(1). No FNV hashing is computed upfront during parsing.
  • Handle-based navigation: CJsonNode is a lightweight struct (pointer + two ints), copying is free
  • Iterator support: CJsonIteratorObj and CJsonIteratorArray for clean traversal
  • Three input formats, one API: AssingFile auto-detects by extension — .json (plain text, needs Parse()), .jsonasm (assembly-like text dump of the tape, needs ParseAssembly()), .jsontc (precompiled binary tape, ready to use immediately, no parsing step)
  • Compiled tape cache (.jsontc): SaveCompiled persists the tape, the source JSON (or a reference to its file), and every perfect-hash table already built — so a reload via AssingFile skips parsing entirely and keeps any JIT-hashed nodes already O(1)

Parse and navigate

#include "Src\\JsonNode.mqh"

TSN::CJsonParser parser;
parser.Assing("{\"symbol\":\"EURUSD\",\"bid\":1.2345,\"ask\":1.2347,\"active\":true}");
parser.CorrectPadding();
parser.Parse();

TSN::CJsonNode root = parser.GetRoot();
string symbol = root["symbol"].ToString();
double bid = root["bid"].ToDouble(0.0);
bool active = root["active"].ToBool(false);

Build json as json string or valid json with CJsonBuilder

  TSN::CJsonBuilderStr build;
  build.PutChar('"');
  build.Obj();
  build.Key("Valor").ValS("└");
  build.Key("asas").ValS("\r\n\v\t aaaaa");
  uchar data[4] = {'h', 'o', 'l', 'a'};
  build.KeyWV("valora").ValU(data);
  build.EndObj();
  build.PutChar('"');
  Print(build.Build()); 
/*
"{\"Valor\":\"\\u2514\",\"asas\":\"\\r\\n\\v\\t aaaaa\",\"valora\":\"hola\"}"
*/  

Iterate an object

TSN::CJsonIteratorObj it = root.BeginObj();
while(it.IsValid())
 {
  PrintFormat("%s : %s", it.Key(), it.Val().ToString());
  it.Next();
 }

Iterate an array

TSN::CJsonNode arr = root["prices"];
TSN::CJsonIteratorArray it = arr.BeginArr();
while(it.IsValid())
 {
  Print(it.Val().ToDouble(0.0));
  it.Next();
 }

Parse from file

TSN::CJsonParser parser;
parser.AssingFile("data.json", false);
parser.CorrectPadding();
parser.Parse();

DOM-API (Full mutable)

void OnStart()
 {
//---
  string json = "{\"ae\":[[10,10],[20,20]],\"comida\":\"\\ud83d\\ude00\",\"trapo\":10.05,\"a\":true,\"invalid\":\"\\xFF\",\"precios\":[10,20,30,40,50],\"va\":1"
                ",\"interpolado\":\"${{valor}}\"}";
//Print(json);
//---
  TSN::CJsonParser parser;
  parser.Assing(json);
  parser.CorrectPadding();
  parser.Parse();
  parser.PrintCintaTypes(0, WHOLE_ARRAY);

//---
  TSN::CDomNodeBase* root;
  TSN::CDomNodeManager manager;
  TSN::CJsonDomSerializer serializer;
  parser.GetRoot().ToDom(root, &manager);

//---
  string kes[];
  root.GetKeys(kes);
  ArrayPrint(kes);

//---
  Print(root.Size());
  Print(root["trapo"].ToDouble(0.0));
  Print(root["ae"].At(1).At(1).ToInt(0));
  Print(root["comida"].ToString());

// Agregar y cambiar tipo
  root["sumando"] = 20.0;
  Print(root.Size());
  Print(root["sumando"].ToDouble(0.0));
  root["sumando"] = true;
  Print(root["sumando"].ToBool(false));

//--- Estadisticas
  Print("Numero de arrays creados: ", manager.m_node_size);
  Print("Numero de objetos creados: ", manager.m_objs_size);
  Print("Numero de strgins creados: ", manager.m_string_stack_size);

//--- Otro json
  TSN::CDomNodeManager manager2;
  TSN::CDomNodeBase* ptr = new TSN::CDomNodeBase(&manager2); // vacio
  ptr.NewObj(128);
  ptr["valores"] = 20.0;

//---
  root.Set("nueva_key", ptr); // No copia profunda si no PUNTERO..
  ptr["coma"] = true; 

//---
  Print(serializer.Serialize(root));
  Print(CharArrayToString(serializer.m_buf, 0, serializer.m_buf_s));

//---
  delete root;
  delete ptr;
 }
 
/*
2026.07.20 09:19:07.290	Dom (EURUSD,H1)	
{"a":true,"invalid":"\\xFF","precios":[10,20,30,40,50],"va":1,"comida":"\ud83d\ude00","interpolado":"${{valor}}",
"sumando":true,"nueva_key":{"valores":20.00000000,"coma":true},"ae":[[10,10],[20,20]],"trapo":10.05000000}
*/ 

Compile and reload a .jsontc tape cache

SaveCompiled writes the parsed tape (plus the source JSON and every perfect-hash table already built) to a binary .jsontc file. Reloading it via AssingFile skips parsing entirely:

//--- Compile once
TSN::CJsonParser parser;
parser.Assing(my_json_string);
parser.CorrectPadding();
parser.Parse();
parser.GetRoot()["symbol"]; // (optional) warm up perfect-hash tables before saving
parser.SaveCompiled("data.jsontc", false);

//--- Reload later — no Parse()/ParseAssembly() needed, tape and hash tables are restored as-is
TSN::CJsonParser parser2;
parser2.AssingFile("data.jsontc", false);
TSN::CJsonNode root = parser2.GetRoot();

There's also an overload that stores a reference to an external JSON file instead of embedding it:

parser.SaveCompiled("data.jsontc", false, "data.json", false);

Performance

Benchmark: twitter.json (616.7 KB), 1000 iterations. Same file, Iterations, and Machine (Laptop)

Note: Here we measure how quickly the JSON is parsed.
All benchmarks are located in: Test\Ben\

Parser Language Time (ms, total / 1000 iter)
simdjson::dom::parser (reused parser) C++ 381
simdjson.Parser (reused parser) Python 475
JsonParserByLeo ASM (One copy of array - JSONASM File R460kb~) MQL5 651-653 Stable
JsonParserByLeo (One copy of array) MQL5 662 - 667
fast_json v3.4 (One copy of array) MQL5 1229 - 1300
YamlParserByLeo (One copy of array) MQL5 1300-1304
Claude Opus 4.8 (Effort=Max) (Best Ai now) (Extended reasoning activated) Generating code (Fast json lib) (One copy of array) MQL5 1312-1313 Stable
orjson Python 1863
ding9736/MQL5-JsonLib (One copy of string) (no dom building only tape parsing) MQL5 2145-2163
ryml (Pure parse time) C++ 2846.55
MQL5 Lite (goose) Metaeditor AI Model MQL5 3242-3250
simdjson.Parser + as_dict Python 4309
simdjson.loads Python 4955
ujson Python 5322
json (stdlib) Python 5860
JAson (One copy of array) MQL5 ~(19070-19220)
ToyJson3 (One copy of string) Only tokenization MQL5 ~(24600) Stable
CJsonNode (One copy of array) MQL5 ~(76720-80470)

Performance in acceses

Benchmark: test.json, 1000 iterations. Same file, Iterations, and Machine (Laptop)

Note: Here we measure how fast access to a node: root["key..."].
All benchmarks are located in: Test\All\

Parser Language Time (microseconds, total / 1000 iter)
JsonParserByLeo MQL5 161-163
fast_json v3.4 MQL5 223 Stable
Claude Opus 4.8 (Effort=Max) (Best Ai now) (Extended reasoning activated) Generating code (Fast json lib) (One copy of array) MQL5 595-605

Performance notes

  • Machine: (Laptop Lenovo ~2017, Windows 10 build 19045, Intel Core i5-8250U @ 1.60GHz, AVX2, GMT-5).
  • MQL5 run: MetaTrader 5 x64 build 5836
  • Python/C++ runs: Same machine (py = 3.10.9) and C++ (CXX23~)

Repository Structure

JsonParserByLeo/
├── Src/  # Full code (Defines, Node, Parser)
├── Test/ # Test and Benchmarks (vs)
└── Wf/   # 100+ Unit test 

License

Read Full License
By downloading or using this repository, you accept the license terms.


Requirements

See dependencies.json for the full list.


Installation

cd "C:\Users\YOUR_USER\AppData\Roaming\MetaQuotes\Terminal\YOUR_ID\MQL5\Shared Projects"
tsndep install "https://forge.mql5.io/nique_372/JsonParserByLeo.git"

Requires the tsndep package, available on PyPI. It automatically downloads and installs all declared dependencies.


Quick Start

1. Include the library:

#include "..\\JsonParserByLeo\\Src\\JsonNode.mqh"

2. Parse and access:

TSN::CJsonParser parser;
parser.Assing(my_json_string);
parser.CorrectPadding();
if(parser.Parse())
 {
  TSN::CJsonNode root = parser.GetRoot();
  double price = root["price"].ToDouble(0.0);
 }

3. Re-parse without re-copying (benchmark pattern):

parser.Assing(raw_string);   // copy once
parser.CorrectPadding();
for(int i = 0; i < 1000; i++)
  parser.Parse();             // re-parse in-place

Roadmap

  • Add suport for JSONPath
  • Add suport for (deserialize\serialize native structs\objs via interfaces)

Contact