A high-performance, memory-free JSON parser for MQL5, based on a flat tape architecture.<br/> Single-pass iterative state machine (as is, without extra loops, "pure" o(n)): no recursion, no dynamic memory fragmentation, no overhead from function calls.
2026-06-03 09:20:57 -05:00
</p>
---
## Main Features
- **Tape-based zero-alloc model**: the entire JSON is parsed into a single contiguous `long[]` array
- **Single flat loop**: one `switch` over a token enum, no helper function calls during parsing, strictly O(n)
- **Two parsing engines**: a pure-MQL5 engine (`Parse()`, no external dependencies) and an optional SIMD engine that runs through a companion C++ DLL (`ParseSimd()`, AVX2, see [SIMD via DLL](#simd-via-dll) below) — same tape output either way, pick whichever fits your deployment
2026-06-20 17:39:08 -05:00
- **JIT perfect-hash key lookup**: object access starts as linear probing (`StrEquals`); once a node passes `TSN_JSON_JIT_MIN_REF_TO_HASING` accesses, a minimal perfect hash table is built for it on the fly (`k = n`, no slack), promoting that node's lookups to O(1). No FNV hashing is computed upfront during parsing.
2026-06-03 09:20:57 -05:00
- **Handle-based navigation**: `CJsonNode` is a lightweight struct (pointer + two ints), copying is free
- **Iterator support**: `CJsonIteratorObj` and `CJsonIteratorArray` for clean traversal
- **Mutable DOM API**: `CDomNodeBase` builds a real, editable node tree from the tape (`ToDom`) — add/remove keys, change a value's type in place, slice arrays Python-style (`PySlicing`), serialize back to JSON (`CJsonDomSerializer`), and load/dump native MQL5 structs directly (`Load<T>()` / `GetAs<T>()`)
- **SAX streaming parser**: `CJsonParserStream` + `ITsnJsonParserStream` feed the tape incrementally in fixed-size chunks (e.g. 64 bytes at a time) and fire `OnStartObj`/`OnKey`/`OnInteger`/... callbacks as it goes — no need to hold the whole document, or the whole tape, in memory at once
- **JSON Builder**: `CJsonBuilder`/`CJsonBuilderStr` for constructing valid JSON (or an escaped JSON string literal) directly, without an intermediate DOM
- **JSON Pointer (RFC 6901)**: `Query("/a/0/b~1c")`-style lookups on both `CJsonNode` (tape) and `CDomNodeBase` (DOM) via `CJsonPointerResolver`
- **JSONPath-like query VM**: a small register-based virtual machine (`CSLDomQueryVm`) compiles an assembly-like query language (`INS_ACCESO_KEY`, `INS_COMPARE_*`, `INS_ITER_ASSING`, custom `INS_CALL`-able functions, ...) to filter/traverse a DOM tree — think `$.libros[?(@.valor > 100)]` compiled down to a tiny bytecode loop
2026-06-20 17:39:08 -05:00
- **Three input formats, one API**: `AssingFile` auto-detects by extension — `.json` (plain text, needs `Parse()`), `.jsonasm` (assembly-like text dump of the tape, needs `ParseAssembly()`), `.jsontc` (precompiled binary tape, ready to use immediately, no parsing step)
- **Compiled tape cache (`.jsontc`)**: `SaveCompiled` persists the tape, the source JSON (or a reference to its file), and every perfect-hash table already built — so a reload via `AssingFile` skips parsing entirely and keeps any JIT-hashed nodes already O(1)
- **104 unit tests** in `Wf/` covering node access, the builder, DOM mutation, `.jsonasm`/`.jsontc` round-trips and malformed-input handling — runnable as a standalone script inside MetaTrader
`SaveCompiled` writes the parsed tape (plus the source JSON and every perfect-hash table already built) to a binary `.jsontc` file. Reloading it via `AssingFile` skips parsing entirely:
```mql5
//--- Compile once
TSN::CJsonParser parser;
parser.Assing(my_json_string);
2026-07-19 15:56:36 -05:00
parser.CorrectPadding();
2026-06-20 17:39:08 -05:00
parser.Parse();
parser.GetRoot()["symbol"]; // (optional) warm up perfect-hash tables before saving
parser.SaveCompiled("data.jsontc", false);
//--- Reload later — no Parse()/ParseAssembly() needed, tape and hash tables are restored as-is
TSN::CJsonParser parser2;
parser2.AssingFile("data.jsontc", false);
TSN::CJsonNode root = parser2.GetRoot();
```
There's also an overload that stores a reference to an external JSON file instead of embedding it:
For MQL5, `Parse()` is fast but limited to what a script/EA can do in pure MQL5. `ParseSimd()` offloads phase 1 (finding every structural character: `{`, `}`, `[`, `]`, `:`, `,`, string bounds) to a small companion C++ DLL that uses AVX2 (SWAR bit-tricks for backslash/quote detection, `pclmulqdq` for the in-string prefix-xor, `popcnt`/`tzcnt` to walk set bits) — phase 2 (building the tape) still runs the same flat state machine. Same `long[]` tape as `Parse()`; only the structural scan changes.
```mql5
#define JSONPARSERBYLEO_DLL
#define JSONPARSERBYLEO_DLL_AVX2
#include "Src\\JsonNode.mqh"
TSN::CJsonParser parser;
parser.Assing(my_json_string);
parser.CorrectPadding(); // pads to a 64-byte boundary when JSONPARSERBYLEO_DLL is defined
if(parser.ParseSimd())
{
TSN::CJsonNode root = parser.GetRoot();
}
```
2026-08-28 17:44:15 -05:00
**Building the DLL:** source is in `Src/DLL/` (`Parser.cpp` + CMake project), requires MSVC + a AVX2-capable CPU. Build it
(`CMakeLists.txt` targets `x64-windows-static`, C++23, `/arch:AVX2`) and drop the resulting `JsonParserByLeo-Avx2.dll` into your terminal's
`MQL5/Libraries` folder. Only needed if you want the SIMD path — the pure-MQL5 `Parse()` has no external dependency.
Requires the `tsndep` package, available on [PyPI](https://pypi.org/project/tsndep). It automatically downloads and installs all declared dependencies.
---
## Quick Start
**1. Include the library:**
```mql5
#include "..\\JsonParserByLeo\\Src\\JsonNode.mqh"
```
**2. Parse and access:**
```mql5
TSN::CJsonParser parser;
parser.Assing(my_json_string);
2026-07-19 15:56:36 -05:00
parser.CorrectPadding();
2026-06-03 09:20:57 -05:00
if(parser.Parse())
{
TSN::CJsonNode root = parser.GetRoot();
double price = root["price"].ToDouble(0.0);
}
```
**3. Re-parse without re-copying (benchmark pattern):**
