| <p align="center">
|
| <img src="https://img.shields.io/badge/Language-MQL5-1B6CA8?style=flat-square"/>
|
| <img src="https://img.shields.io/badge/Platform-MetaTrader%205-0D1B2A?style=flat-square"/>
|
| <img src="https://img.shields.io/badge/Author-nique__372-C9D6DF?style=flat-square&logoColor=white"/>
|
| <img src="https://img.shields.io/badge/MQL5.com-nique__372-1B6CA8?style=flat-square"/>
|
| </p>
|
|
|
| <p align="center">
|
| A high-performance, memory-free JSON parser for MQL5, based on a flat tape architecture.<br/> Single-pass iterative state machine (as is, without extra loops, "pure" o(n)): no recursion, no dynamic memory fragmentation, no overhead from function calls.
|
| </p>
|
|
|
| ---
|
|
|
| ## Main Features
|
|
|
| - **Tape-based zero-alloc model**: the entire JSON is parsed into a single contiguous `long[]` array (numbers, booleans and nulls are materialized eagerly; only keys and strings are stored lazily as offset + length)
|
| - **Single flat loop**: one `switch` over a token enum, no helper function calls during parsing, strictly O(n), with branch-less padding sentinels to keep the hot path as short as possible
|
| - **Two parsing engines**: a pure-MQL5 engine (`Parse()`, no external dependencies) and an optional SIMD engine that runs through a companion C++ DLL (`ParseSimd()`, AVX2, see [SIMD via DLL](#simd-via-dll) below) — same tape output either way, pick whichever fits your deployment
|
| - **Strict validation**: numbers are fully validated (leading zeros, exponent and decimal form), `true`/`false`/`null` are checked completely (not just skipped), strings/keys must be closed, and grammar order is enforced by the state machine. Detailed guarantees in [SPEC.md](./SPEC.md)
|
| - **Safe number handling**: full 64-bit unsigned support (`ulong`, no precision loss on large ids), and any integer or real the parser can't represent exactly is kept as a string (`ToString()`) so you can use a big-number library instead of silently losing digits
|
| - **JIT perfect-hash key lookup**: object access starts as a linear scan; once an object (with enough keys) has been accessed enough times, a minimal perfect hash table is built for it on the fly (powered by [PerfectHashByLeo](https://forge.mql5.io/nique_372/PerfectHashByLeo)), promoting that object's lookups to O(1). Nothing is hashed upfront during parsing, small objects/arrays never trigger it, and the thresholds are configurable via defines (`TSN_NODEBASEREAD_MIN_JIT_ATTEMPS_TO_PHASH`, `TSN_NODEBASEREAD_MIN_JIT_VALUES_TO_HASH`) or disabled entirely with `Options()`
|
| - **Handle-based navigation**: `CJsonNode` is a lightweight struct (pointer + two ints), copying is free
|
| - **Iterator support**: `CJsonIteratorObj` and `CJsonIteratorArray` for clean traversal
|
| - **On-demand unescape**: keys and strings are unescaped only when you read them (`node["\n"]` works, emojis and `\uXXXX` surrogates included); invalid escapes are reported as errors instead of being silently ignored
|
| - **Mutable DOM API**: `CDomNodeBase` builds a real, editable node tree from the tape (`ToDom`) — add/remove keys, change a value's type in place, slice arrays Python-style (`PySlicing`), serialize back to JSON (`CJsonDomSerializer`), and load/dump native MQL5 structs directly (`Load<T>()` / `GetAs<T>()`)
|
| - **SAX streaming parser**: `CJsonParserStream` + `ITsnJsonParserStream` feed the tape incrementally in fixed-size chunks (e.g. 64 bytes at a time) and fire `OnStartObj`/`OnKey`/`OnInteger`/... callbacks as it goes — no need to hold the whole document, or the whole tape, in memory at once. Numbers too big to convert safely are delivered raw through `OnBigInteger`/`OnBigDecimal`
|
| - **JSON Builder**: `CJsonBuilder`/`CJsonBuilderStr` for constructing valid JSON (or an escaped JSON string literal) directly, without an intermediate DOM
|
| - **JSON Minify**: whitespace stripping for existing documents
|
| - **JSON Pointer (RFC 6901)**: `Query("/a/0/b~1c")`-style lookups on both `CJsonNode` (tape) and `CDomNodeBase` (DOM), precompiled once with `BSFLPointerCompiler`
|
| - **JSONPath-like query VM**: a small register-based virtual machine (`CSLDomQueryVm`) compiles an assembly-like query language (`INS_ACCESO_KEY`, `INS_COMPARE_*`, `INS_ITER_ASSING`, custom `INS_CALL`-able functions, ...) to filter/traverse a DOM tree — think `$.libros[?(@.valor > 100)]` compiled down to a tiny bytecode loop
|
| - **Two input formats, one API**: `AssingFile` auto-detects by extension — `.json` (plain text, needs `Parse()`) and `.jsontc` (precompiled binary tape, ready to use immediately, no parsing step)
|
| - **Compiled tape cache (`.jsontc`)**: `SaveCompiled` persists the tape, the source JSON (or a reference to its file), and every perfect-hash table already built — so a reload via `AssingFile` skips parsing entirely and keeps any JIT-hashed nodes already O(1)
|
| - **Modular packages**: include only what you need — `JsonNode.mqh` (parser + node + DOM), `Json.mqh` (+ serializer, SAX, builders, pointer, minify) or `JsonFull.mqh` (+ JsonPath VM). See [Src/README.md](./Src/README.md)
|
| - **100+ unit tests** in `Wf/` covering node access, the builder, DOM mutation, `.jsontc` round-trips and malformed-input handling — runnable as a standalone script inside MetaTrader
|
|
|
| ### Parse and navigate
|
|
|
| ```mql5
|
| #include "Src\\JsonNode\\JsonNode.mqh"
|
|
|
| TSN::CJsonParser parser;
|
| parser.Assing("{\"symbol\":\"EURUSD\",\"bid\":1.2345,\"ask\":1.2347,\"active\":true}");
|
| parser.CorrectPadding();
|
| parser.Parse();
|
|
|
| TSN::CJsonNode root = parser.GetRoot();
|
| string symbol = root["symbol"].ToString();
|
| double bid = root["bid"].ToDouble(0.0);
|
| bool active = root["active"].ToBool(false);
|
| ```
|
|
|
| ### Build json as json string or valid json with CJsonBuilder
|
|
|
| ```mql5
|
| TSN::CJsonBuilderStr build;
|
| build.PutChar('"');
|
| build.Obj();
|
| build.Key("Valor").ValS("└");
|
| build.Key("asas").ValS("\r\n\v\t aaaaa");
|
| uchar data[4] = {'h', 'o', 'l', 'a'};
|
| build.KeyWV("valora").ValU(data);
|
| build.EndObj();
|
| build.PutChar('"');
|
| Print(build.Build());
|
| /*
|
| "{\"Valor\":\"\\u2514\",\"asas\":\"\\r\\n\\v\\t aaaaa\",\"valora\":\"hola\"}"
|
| */
|
| ```
|
|
|
| ### Iterate an object
|
|
|
| ```mql5
|
| TSN::CJsonIteratorObj it = root.BeginObj();
|
| while(it.IsValid())
|
| {
|
| PrintFormat("%s : %s", it.Key(), it.Val().ToString());
|
| it.Next();
|
| }
|
| ```
|
|
|
| ### Iterate an array
|
|
|
| ```mql5
|
| TSN::CJsonNode arr = root["prices"];
|
| TSN::CJsonIteratorArray it = arr.BeginArr();
|
| while(it.IsValid())
|
| {
|
| Print(it.Val().ToDouble(0.0));
|
| it.Next();
|
| }
|
| ```
|
|
|
| ### Parse from file
|
|
|
| ```mql5
|
| TSN::CJsonParser parser;
|
| parser.AssingFile("data.json", false);
|
| parser.CorrectPadding();
|
| parser.Parse();
|
| ```
|
|
|
| ### DOM-API (Full mutable)
|
|
|
| ```mql5
|
| void OnStart()
|
| {
|
| //---
|
| string json = "{\"ae\":[[10,10],[20,20]],\"comida\":\"\\ud83d\\ude00\",\"trapo\":10.05,\"a\":true,\"invalid\":\"\\xFF\",\"precios\":[10,20,30,40,50],\"va\":1"
|
| ",\"interpolado\":\"${{valor}}\"}";
|
| //Print(json);
|
| //---
|
| TSN::CJsonParser parser;
|
| parser.Assing(json);
|
| parser.CorrectPadding();
|
| parser.Parse();
|
| parser.PrintCintaTypes(0, WHOLE_ARRAY);
|
|
|
| //---
|
| TSN::CDomNodeManager manager;
|
| TSN::CJsonDomSerializer serializer;
|
| TSN::CDomNodeBase* root = new TSN::CDomNodeBase(&manager);
|
| parser.GetRoot().ToDom(root, &manager);
|
|
|
| //---
|
| string kes[];
|
| root.GetKeys(kes);
|
| ArrayPrint(kes);
|
|
|
| //---
|
| Print(root.Size());
|
| Print(root["trapo"].ToDouble(0.0));
|
| Print(root["ae"].At(1).At(1).ToInt(0));
|
| Print(root["comida"].ToString());
|
|
|
| // Agregar y cambiar tipo
|
| root["sumando"] = 20.0;
|
| Print(root.Size());
|
| Print(root["sumando"].ToDouble(0.0));
|
| root["sumando"] = true;
|
| Print(root["sumando"].ToBool(false));
|
|
|
| //--- Estadisticas
|
| Print("Numero de arrays creados: ", manager.m_node_size);
|
| Print("Numero de objetos creados: ", manager.m_objs_size);
|
| Print("Numero de strgins creados: ", manager.m_string_stack_size);
|
|
|
| //--- Otro json
|
| TSN::CDomNodeManager manager2;
|
| TSN::CDomNodeBase* ptr = new TSN::CDomNodeBase(&manager2); // vacio
|
| ptr.NewObj(128);
|
| ptr["valores"] = 20.0;
|
|
|
| //---
|
| root.Set("nueva_key", ptr); // No copia profunda si no PUNTERO..
|
| ptr["coma"] = true;
|
|
|
| TSN::CDomNodeBase* out[];
|
| const int t = root["precios"].PySlicing(out, 0, -1, -1, -1);
|
| for(int i = 0; i < t; i++)
|
| Print("valor: ", out[i].ToInt(0)); // al revez 50,40,30,20,10
|
|
|
| //---
|
| Print(serializer.Serialize(root));
|
| Print(CharArrayToString(serializer.m_buf, 0, serializer.m_buf_s));
|
|
|
| //---
|
| delete root;
|
| delete ptr;
|
| }
|
|
|
| /*
|
| 2026.07.20 09:19:07.290 Dom (EURUSD,H1)
|
| {"a":true,"invalid":"\\xFF","precios":[10,20,30,40,50],"va":1,"comida":"\ud83d\ude00","interpolado":"${{valor}}",
|
| "sumando":true,"nueva_key":{"valores":20.00000000,"coma":true},"ae":[[10,10],[20,20]],"trapo":10.05000000}
|
| */
|
| ```
|
|
|
| ### DOM API From and ToJson
|
|
|
| ```mql5
|
| struct Libro
|
| {
|
| string name;
|
| string titulo;
|
| double valor;
|
|
|
|
|
| //---
|
| Libro() {}
|
| Libro(TSN::CDomNodeBase& obj)
|
| {
|
| name = obj["name"].ToString();
|
| titulo = obj["titulo"].ToString();
|
| valor = obj["valor"].ToDouble(0.00);
|
| }
|
| static void ToTsnDomNode(TSN::CDomNodeBase& obj, const Libro& other)
|
| {
|
| obj.NewObj(16);
|
| obj["name"] = other.name;
|
| obj["titulo"] = other.titulo;
|
| obj["valor"] = other.valor;
|
| }
|
| };
|
|
|
| void OnStart()
|
| {
|
| //---
|
| TSN::CDomNodeManager manager;
|
| Libro l;
|
| l.name = "hola";
|
| l.titulo = "titulo";
|
| l.valor = 1.0;
|
|
|
| // Load
|
| TSN::CJsonDomSerializer ser;
|
| TSN::CDomNodeBase node(&manager);
|
| node.Load<Libro>(l);
|
| Print(ser.Serialize(&node));
|
| Print(CharArrayToString(ser.m_buf, 0, ser.m_buf_s)); // {"name":"hola","titulo":"titulo","valor":1.00000000}
|
|
|
|
|
| //---
|
| TSN::CJsonBuilder builder;
|
| builder.Obj();
|
| builder.KeyWV("name").ValSWV("hola");
|
| builder.KeyWV("titulo").ValSWV("como");
|
| builder.KeyWV("valor").Val(10.0490);
|
| builder.EndObj();
|
|
|
| //---
|
| TSN::CJsonParser parser;
|
| ArrayCopy(parser.m_raw, builder.m_buf, 0, 0, builder.m_pos);
|
| parser.CalcLen();
|
| parser.CorrectPadding();
|
| parser.Parse();
|
|
|
| // Dom building
|
| TSN::CDomNodeBase node2(&manager); // le pasamos su manager
|
| parser.GetRoot().ToDom(&node2, &manager);
|
|
|
| //---
|
| Libro l2 = node2.GetAs<Libro>();
|
| Print(l2.name); // DomSer (EURUSD,H1) hola
|
| Print(l2.titulo); // DomSer (EURUSD,H1) como
|
| Print(l2.valor); // DomSer (EURUSD,H1) 10.049
|
| }
|
| ```
|
|
|
| ### Json Pointer (DOM and CJsonNode API)
|
|
|
| ```mql5
|
| //---
|
| TSN::CJsonParser parser;
|
| parser.Assing(g_json);
|
| parser.CorrectPadding();
|
| parser.Parse();
|
|
|
| //---
|
| TSN::BSFLPointerCompiler compiler;
|
|
|
| //---
|
| TSN::BSFLPointerQuery query("/config~1general/riesgo~0maximo", compiler);
|
|
|
| //---
|
| TSN::CJsonNode root = parser.GetRoot();
|
|
|
| //---
|
| Print("--- CJsonNode ---");
|
| Print(root[query].ToDouble(0.0));
|
| query.Set("/raro~1~0mix~0~11", compiler);
|
| Print(root[query].ToString(""));
|
| query.Set("/ordenes/2/tags/2", compiler);
|
| query.CintaTypesInfo();
|
| Print(root[query].ToString(""));
|
|
|
| //---
|
| TSN::CDomNodeManager manager;
|
| TSN::CDomNodeBase node(&manager);
|
| root.ToDom(&node, &manager);
|
|
|
| Print("--- CJsonPointerResolver ---");
|
| Print(node[query].ToString());
|
|
|
| /*
|
| Pointer (EURUSD,H1) --- CJsonNode ---
|
| Pointer (EURUSD,H1) 2.5
|
| Pointer (EURUSD,H1) combinacion rara
|
| Pointer (EURUSD,H1) riesgo/alto
|
| Pointer (EURUSD,H1) riesgo/alto
|
| */
|
| ```
|
|
|
| ### Stream-With SAX Parse
|
|
|
| ```mql5
|
| class CTestStream : public TSN::ITsnJsonParserStream
|
| {
|
| private:
|
| string ToStr(const uchar& in[], int i, int len)
|
| {
|
| static string str;
|
| return TSN::CBSFLUnescape::MakeString<TSN::CJsonParser>(in, i, len, str) ? str : NULL;
|
| }
|
| public:
|
| CTestStream(void) {}
|
| ~CTestStream(void) {}
|
| void OnStartDocument() override final { Print("Inicio de documento"); }
|
| void OnEndDocument() override final { Print("Fin de documento"); }
|
| void OnStartObj() override final { Print("Inicio de objeto: ", m_ctx.CurrentStack()); }
|
| void OnEndObJ() override final { Print("Fin de objeto: ", m_ctx.CurrentStack()); }
|
| void OnStartArr() override final { Print("Inicio de array: ", m_ctx.CurrentStack()); }
|
| void OnEndArr() override final { Print("Fin de array: ", m_ctx.CurrentStack()); }
|
| void OnInteger(long v) override final { Print("Numero: ", v); }
|
| void OnReal(double v) override final { Print("Real: ", v); }
|
| void OnString(int s, int len) override final { Print("Texto: ", ToStr(m_ctx.m_raw, s, len)); }
|
| void OnBool(bool v) override final { Print("Boleano: ", v); }
|
| void OnNull() override final { Print("Null"); }
|
| void OnKey(int s, int len) override final { Print("Clave: ", ToStr(m_ctx.m_raw, s, len)); }
|
| void OnError() { Print(EnumToString(m_ctx.LastErr())); }
|
| void OnBigDecimal(int s, int len) override final { Print("Big decimal: ", CharArrayToString(m_ctx.m_raw, s, len)); }
|
| void OnBigInteger(int s, int len) override final { Print("Big integer: ", CharArrayToString(m_ctx.m_raw, s, len)); }
|
| };
|
| //+------------------------------------------------------------------+
|
| void OnStart()
|
| {
|
| //---
|
| TSN::CJsonParserStream stream;
|
| stream.SetCaller(new CTestStream(), true);
|
| //---
|
| string json = "{\"ae\":[[10,10],[20,20]],\"comida\":\"\\ud83d\\ude00\",\"trapo\":10.05,\"a\":true,\"invalid\":\"\\xFF\",\"precios\":[10,20,30,40,50],\"va\":1"
|
| ",\"interpolado\":\"${{valor}}\"}";
|
| uchar raw[];
|
| const int l = StringToCharArray(json, raw);
|
| int p = 0;
|
| // Simular pasadas de 64 bytes
|
| while(p < l) // tien que ser menor, en un caso real quizas ahsta on end document
|
| {
|
| //--- le damos datos
|
| // Tambien antes de dar los datos odemos llamar a CheckLimpieza
|
| // Para que mueva pos a posicion 0 (reutlziacion) en caso se pueda si no seguir acomulando
|
| // Ahora en este caso vamos sumando dado que m_len neceista crecer.... luego quizas se le peude reinicar..
|
| stream.m_len += ArrayCopy(stream.m_raw, raw, stream.m_pos, p, 64); //
|
| // intermante va parseando aumentado m_pos
|
| // Otro si no da el documento para otro el bulce se dentecia aunqeu array copy ya menaja eso intemante
|
| // si nos pamaos normaliza asi que no hay problema pero eso un aviso..
|
| stream.ParseSreamSax();
|
| // Aqui por ejemplo se peuden hacer mas cosas.. tu lo llamas cuando quieres
|
| p += 64; // ya le dimos 64
|
| }
|
| }
|
| ```
|
|
|
| ### Json Patch full Fast parse and VM (register-based)
|
|
|
|
|
| ```mql5
|
| void JsonPathSize(TSN::CSLDomQueryVm* state)
|
| {
|
| TSN::CDomNodeBase* ptr = state.FReadNode(0);
|
| state.FPushInteger(ptr.Size());
|
| }
|
| void OnStart()
|
| {
|
|
|
| //---
|
| const string json = "{\n"
|
| "\"libros\":{\n"
|
| "\"libro1\": {\"valor\":542,\"sumando\":[],\"comando\":30},\n"
|
| "\"libro2\": {\"valor\":19,\"sumando\":[30],\"comando\":2},\n"
|
| "\"libro3\": {\"valor\":452,\"sumando\":[20,50,65,89,419],\"comando\":7}\n"
|
| " }\n"
|
| "}";
|
|
|
| //---
|
| TSN::CJsonParser parser;
|
| parser.Assing(json);
|
| parser.CorrectPadding();
|
| parser.Parse();
|
|
|
| //---
|
| TSN::CDomNodeManager manager;
|
| TSN::CDomNodeBase node(&manager);
|
| parser.GetRoot().ToDom(&node, &manager);
|
|
|
| //---
|
| TSN::CSLDomQueryVm vm;
|
| vm.SetFunction("size", JsonPathSize);
|
|
|
| // $.libros[?(@.comando * 20 < @.valor && @.sumando.size()].sumando[*]
|
| const string asm =
|
| "%reg 0 = 20 \n"
|
| "%reg 1 = 1\n"
|
| "INS_ACCESO_KEY_F K\"libros\"\n"
|
| "INS_INICIAR_ITERACION 1 ; Solo se permiten objetos en esta iteracion\n"
|
| "@Iteracion: ; Jump aqui\n"
|
| " INS_ITER_ASSING 'Final' ; En caso falle salta a Final y terminamos .. \n"
|
| " INS_ACCESO_KEY 'Iteracion' K\"comando\" ; Accedemos a comando \n"
|
| " INS_SAVE 256 'Iteracion' ; Lo guardamos en el registro 256 si falla reset\n"
|
| " INS_M_MUL 256 0 256 'Iteracion' ; Mulñtiplicamos el registro 256 por el 0 y lo guardamos en 256\n"
|
| " INS_ACCESO_KEY 'Iteracion' K\"valor\" ; Accede a valor\n"
|
| " INS_SAVE 257 'Iteracion'; Guardamos su valor actual\n"
|
| " INS_COMPARE_MENOR 256 257 1 'Iteracion' ; Comparamos registros\n"
|
| " INS_ACCESO_KEY 'Iteracion' K\"sumando\" ; Accedemos a sumando \n"
|
| " INS_SAVE_NO_RESET 256 'Iteracion'; guadcamos en sumand\n"
|
| // Nota que aqui el resultado se pondra apartir del 258 reg dado qeu el 257 se pone auto
|
| // el numero de parametros escritos...
|
| " INS_CALL 256 1 'size'; LLamamos a la funcino size y sus parametros emppizan en 256 y tiene 1 parametro a leer\n"
|
| // Nota que esta verirficacion quizas no haga falta.. aunque si esperamos exactamentwe N parameotrs
|
| // o digamos 1 quizas si si no se nos desalinea... nuestra lecutra.. auqnue quizas nos podemos saltar a cierta
|
| // direccion daod que sabemos que tenemos n parmaeotrs en el reg 257
|
| // A futuro si veo que se requiere separar o casos mas complejos como iterar sobre los parametros obtenidos
|
| // ahi si si usariosm offsets... ahi quizas lo agrego no es complejo es cosa de cambiar layort y nueva sintaxis de asm
|
| // pero ya eso creo que tiraraimso ya a una vm casi de un lenguaje solo faltura stack frames y ya por que en teoria
|
| // ahora mismo podemos tener varaibles, bucles, con los ifs, etc...
|
| " INS_COMPARE_EQ 257 1 1 'Iteracion' ; Verificamos que el numero de retornanos es 1\n"
|
| " INS_COMPARE_IS 258 'Iteracion' 1 ; Usamos is para comprar su valor\n"
|
| " ; Si llegamos hasta aqui pasamos todo entonces copy all\n"
|
| " INS_COPY_ALL 'Iteracion' \n"
|
| " INS_JUMP 'Iteracion'; Ahora nos dirigimos devuelva a iteracion \n"
|
| "@Final: \n"
|
| " INS_SALIDA ; salimos"
|
| ;
|
|
|
| //---
|
| vm.Assing(asm);
|
| vm.CorrectPadding();
|
| vm.CompileAsm();
|
| vm.LockValues();
|
| vm.BaseNode(&node);
|
|
|
| //---
|
| vm.Summary();
|
|
|
| //---
|
| TSN::CDomNodeBase* elements[];
|
| vm.Run(elements);
|
|
|
| //---
|
| const int t = vm.m_el_s;
|
| for(int i = 0; i < t; i++)
|
| {
|
| Print(elements[i].ToInt(0));
|
| }
|
|
|
| //---
|
| /* Output:
|
| JPath (EURUSD,M1) 20
|
| JPath (EURUSD,M1) 50
|
| JPath (EURUSD,M1) 65
|
| JPath (EURUSD,M1) 89
|
| JPath (EURUSD,M1) 419
|
| */
|
| }
|
| ```
|
|
|
| ### Compile and reload a `.jsontc` tape cache
|
|
|
| `SaveCompiled` writes the parsed tape (plus the source JSON and every perfect-hash table already built) to a binary `.jsontc` file. Reloading it via `AssingFile` skips parsing entirely:
|
|
|
| ```mql5
|
| //--- Compile once
|
| TSN::CJsonParser parser;
|
| parser.Assing(my_json_string);
|
| parser.CorrectPadding();
|
| parser.Parse();
|
| parser.GetRoot()["symbol"]; // (optional) warm up perfect-hash tables before saving
|
| parser.SaveCompiled("data.jsontc", false);
|
|
|
| //--- Reload later — no Parse()/ParseAssembly() needed, tape and hash tables are restored as-is
|
| TSN::CJsonParser parser2;
|
| parser2.AssingFile("data.jsontc", false);
|
| TSN::CJsonNode root = parser2.GetRoot();
|
| ```
|
|
|
| There's also an overload that stores a reference to an external JSON file instead of embedding it:
|
|
|
| ```mql5
|
| parser.SaveCompiled("data.jsontc", false, "data.json", false);
|
| ```
|
|
|
| ---
|
|
|
| ## SIMD via DLL
|
|
|
| For MQL5, `Parse()` is fast but limited to what a script/EA can do in pure MQL5. `ParseSimd()` offloads phase 1 (finding every structural character: `{`, `}`, `[`, `]`, `:`, `,`, string bounds) to a small companion C++ DLL that uses AVX2 (SWAR bit-tricks for backslash/quote detection, `pclmulqdq` for the in-string prefix-xor, `popcnt`/`tzcnt` to walk set bits) — phase 2 (building the tape) still runs the same flat state machine. Same `long[]` tape as `Parse()`; only the structural scan changes.
|
|
|
| ```mql5
|
| #define JSONPARSERBYLEO_DLL
|
| #define JSONPARSERBYLEO_DLL_AVX2
|
| #include "Src\\JsonNode\\JsonNode.mqh"
|
|
|
| TSN::CJsonParser parser;
|
| parser.Assing(my_json_string);
|
| parser.CorrectPaddingSimd(); // pads to a 64-byte boundary when JSONPARSERBYLEO_DLL is defined
|
| if(parser.ParseSimd())
|
| {
|
| TSN::CJsonNode root = parser.GetRoot();
|
| }
|
| ```
|
|
|
| **Building the DLL:** source is in `Src/DLL/` (`Parser.cpp` + CMake project), requires MSVC + a AVX2-capable CPU. Build it
|
| (`CMakeLists.txt` targets `x64-windows-static`, C++23, `/arch:AVX2`) and drop the resulting `JsonParserByLeo-Avx2.dll` into your terminal's
|
| `MQL5/Libraries` folder. Only needed if you want the SIMD path — the pure-MQL5 `Parse()` has no external dependency.
|
|
|
| ---
|
|
|
| ## Performance
|
|
|
| JsonParserByLeo is a fast JSON library. You can see the results of my "mini-investigation" of JSON libraries where
|
| I tested and benchmarked the parsing and node access capabilities of the most popular MQL5 libraries.
|
| I also included parsing only for libraries from other languages.
|
| - [Click here](./Test/Ben/README.md)
|
|
|
|
|
| ---
|
|
|
| ## Repository Structure
|
|
|
| ```
|
| JsonParserByLeo/
|
| ├── BenOther/ # Reference benchmarks in other languages (C++/simdjson, Python, Rust)
|
| ├── Src/ # Full MQL5 source (Parser, Node, DOM, SAX, Builder, Pointer, Minify, JsonPath VM)
|
| │ └── DLL/ # Optional companion C++ DLL: AVX2 structural scan (Parser.cpp + CMake project)
|
| ├── SPEC.md # Guarantees, edge cases and design philosophy
|
| ├── CHANGELOG.md # Version history
|
| ├── Test/ # Manual tests and benchmarks (Test/Ben/), run as MQL5 scripts
|
| └── Wf/ # 104 automated unit tests (Wf/Test.mq5), run as a standalone script
|
| ```
|
|
|
| ---
|
|
|
| ## License
|
|
|
| **[Read Full License](./LICENSE)**
|
| By downloading or using this repository, you accept the license terms.
|
|
|
| ---
|
|
|
| ## Requirements
|
|
|
| See [dependencies.json](./dependencies.json) for the full list.
|
|
|
| ---
|
|
|
| ## Installation
|
|
|
| ```bash
|
| cd "C:\Users\YOUR_USER\AppData\Roaming\MetaQuotes\Terminal\YOUR_ID\MQL5\Shared Projects"
|
| tsndep install "https://forge.mql5.io/nique_372/JsonParserByLeo.git"
|
| ```
|
|
|
| Requires the `tsndep` package, available on [PyPI](https://pypi.org/project/tsndep). It automatically downloads and installs all declared dependencies.
|
|
|
| ---
|
|
|
| ## Quick Start
|
|
|
| **1. Include the library:**
|
|
|
| ```mql5
|
| #include "..\\JsonParserByLeo\\Src\\JsonNode\\JsonNode.mqh"
|
| ```
|
|
|
| **2. Parse and access:**
|
|
|
| ```mql5
|
| TSN::CJsonParser parser;
|
| parser.Assing(my_json_string);
|
| parser.CorrectPadding();
|
| if(!parser.Parse())
|
| {
|
| //--- error handling
|
| // error info method
|
| parser.ErrorInfo();
|
| // or manualy logged (obligatory use CatchErr To capture more specific errors that could not be fully captured.)
|
| parser.CatchErr();
|
| Print("Last json err = ", EnumToString(parser.LastErr()), " = ", parser.LastErr());
|
| }
|
|
|
| TSN::CJsonNode root = parser.GetRoot();
|
| double price = root["price"].ToDouble(0.0);
|
| ```
|
|
|
| **3. Re-parse without re-copying (benchmark pattern):**
|
|
|
| ```mql5
|
| parser.Assing(raw_string); // copy once
|
| parser.CorrectPadding();
|
| for(int i = 0; i < 1000; i++)
|
| parser.Parse(); // re-parse in-place
|
| ```
|
|
|
| ---
|
|
|
| ## Roadmap
|
|
|
| - Add suport for JSONPath [In progres 100% ASM ready] [Added 24/7/26]
|
| - Add suport for (deserialize\serialize native structs\objs) [Added 23/7/26]
|
| - Add Optimizer and Compiler for JsonPath queryes
|
|
|
| ---
|
|
|
| ## Contact
|
|
|
| - **Platform:** [MQL5 Community](https://www.mql5.com/es/users/nique_372)
|
| - **Profile:** https://www.mql5.com/es/users/nique_372
|
| - **Articles:** https://www.mql5.com/es/users/nique_372/publications |
