//+------------------------------------------------------------------+ //| AiCode.mqh | //| Parser de archivos .set usando formato TAPE (estilo simdjson): | //| toda la estructura parseada vive en UN SOLO array plano | //| long m_tape[], sin objetos CSetParam/CSetValue2. Elimina el | //| overhead de construccion/gestion de objetos por linea. | //| | //| FORMATO DEL TAPE: | //| tape[0] = cantidad total de nodos (lineas parseadas) | //| Por cada nodo, a partir de tape[1], en secuencia: | //| [Key slot] (keyStart<<32)|keyLen | //| [Metadata slot] bit63=isOptimizable, bit62=enabled | //| [Value slot(s)] 1 valor si Simple, 4 valores (Valor/Start/ | //| Step/Stop) seguidos si Optimizable. | //| Cada valor individual ocupa: | //| - 1 slot si STRING o BOOL | //| - 2 slots si DOUBLE (tag + bits crudos) | //| tag = 4 bits altos del primer slot del | //| valor (0=STRING,1=DOUBLE,2=BOOL) | //| | //| Uso identico al de siempre para el caller: | //| CSetFileParser setfile; | //| setfile.AssignString(g_setfile); | //| if(setfile.Parse()) | //| { | //| int x = setfile["MaxOrders_18"].Valor().ToInt(); | //| int st = setfile["MaxOrders_18"].Step().ToInt(); | //| } | //+------------------------------------------------------------------+ #property strict //==================================================================== // Constantes de formato del tape //==================================================================== #define TAPE_TAG_STRING 0 #define TAPE_TAG_DOUBLE 1 #define TAPE_TAG_BOOL 2 #define TAPE_META_OPTIM_BIT 0x8000000000000000 #define TAPE_META_ENABLED_BIT 0x4000000000000000 class CSetFileParser; // fwd decl //==================================================================== // Union a nivel de archivo (scope global) para reinterpretar double // <-> long bit a bit sin perdida de precision. Se declara aca afuera // de cualquier funcion/metodo porque MQL5 solo permite declaraciones // de tipo (union/struct/class) en scope global, namespace o de clase // -- no dentro del cuerpo de una funcion. //==================================================================== union UDoubleLong { double d; long l; }; //==================================================================== // CTapeValue: "vista" liviana (no objeto persistente) sobre un valor // dentro del tape. Se construye al vuelo cuando el usuario llama // Valor()/Start()/Step()/Stop() -- no vive en el tape en si, solo // referencia la posicion (slotIndex) donde esta el valor. //==================================================================== class CTapeValue { private: CSetFileParser *m_owner; int m_slotIndex; // -1 si no existe (dummy) public: CTapeValue() : m_owner(NULL), m_slotIndex(-1) {} void Bind(CSetFileParser *owner, int slotIndex) { m_owner = owner; m_slotIndex = slotIndex; } // Implementadas despues de CSetFileParser (necesitan su definicion) double ToDouble() const; int ToInt() const; long ToLong() const; bool ToBool() const; string ToString() const; bool IsEmpty() const { return m_slotIndex < 0; } }; //==================================================================== // CTapeNode: "vista" liviana sobre un nodo (linea) del tape. Se // construye al vuelo en operator[]/At() -- no persiste como objeto, // solo guarda el indice del primer slot del nodo (el Key slot). //==================================================================== class CTapeNode { private: CSetFileParser *m_owner; int m_nodeSlot; // indice del Key slot de este nodo, -1 si no encontrado bool m_found; public: CTapeNode() : m_owner(NULL), m_nodeSlot(-1), m_found(false) {} void Bind(CSetFileParser *owner, int nodeSlot, bool found) { m_owner = owner; m_nodeSlot = nodeSlot; m_found = found; } bool Found() const { return m_found; } // Implementadas despues de CSetFileParser string Key() const; bool IsOptimizable() const; bool IsEnabled() const; CTapeValue Valor() const; CTapeValue Start() const; CTapeValue Step() const; CTapeValue Stop() const; }; //==================================================================== // CSetFileParser: parser que escribe TODO a un unico array long[] // (el tape), sin crear objetos CSetParam/CSetValue2 por linea. //==================================================================== class CSetFileParser { private: uchar m_buf[]; int m_len; long m_tape[]; // el tape completo int m_tapeLen; // cantidad de slots usados (incluye tape[0]) int m_nodeCount; // = tape[0], cacheado para no releer int m_lastTapeSize; // tamaño real que ocupo el ultimo Parse() exitoso, // usado para pre-dimensionar exacto en la // proxima llamada sobre el mismo buffer CTapeNode m_notFoundNode; bool IsSpaceByte(uchar c) const { return (c == ' ' || c == '\t' || c == '\r' || c == '\n'); } bool IsNumericRange(int start, int end) const { if(start >= end) return false; bool seenDigit = false; bool seenDot = false; for(int i = start; i < end; i++) { uchar c = m_buf[i]; if(c == '-' || c == '+') { if(i != start) return false; continue; } if(c == '.') { if(seenDot) return false; seenDot = true; continue; } if(c < '0' || c > '9') return false; seenDigit = true; } return seenDigit; } bool RangeEqualsLiteral(int start, int end, const string lit) const { int n = StringLen(lit); if(end - start != n) return false; for(int i = 0; i < n; i++) if(m_buf[start + i] != (uchar)StringGetCharacter(lit, i)) return false; return true; } void TrimRange(int &start, int &end) const { while(start < end && IsSpaceByte(m_buf[start])) start++; while(end > start && IsSpaceByte(m_buf[end - 1])) end--; } //----------------------------------------------------------------- // Resuelve double directo sobre bytes (sin CharArrayToString). //----------------------------------------------------------------- double ParseDoubleRange(int start, int end) const { if(start >= end) return 0.0; int i = start; bool neg = false; if(m_buf[i] == '-') { neg = true; i++; } else if(m_buf[i] == '+') { i++; } double intPart = 0.0; while(i < end && m_buf[i] >= '0' && m_buf[i] <= '9') { intPart = intPart * 10.0 + (m_buf[i] - '0'); i++; } double fracPart = 0.0; if(i < end && m_buf[i] == '.') { i++; double scale = 0.1; while(i < end && m_buf[i] >= '0' && m_buf[i] <= '9') { fracPart += (m_buf[i] - '0') * scale; scale *= 0.1; i++; } } double result = intPart + fracPart; return neg ? -result : result; } bool ParseBoolRange(int start, int end) const { if(end - start == 1 && m_buf[start] == '1') return true; if(end - start == 4 && m_buf[start]=='t' && m_buf[start+1]=='r' && m_buf[start+2]=='u' && m_buf[start+3]=='e') return true; return false; } //----------------------------------------------------------------- // Reinterpreta double <-> long bit a bit via el union global // UDoubleLong (declarado a nivel de archivo, ver arriba). //----------------------------------------------------------------- long DoubleToBits(double d) const { UDoubleLong u; u.d = d; return u.l; } double BitsToDouble(long bits) const { UDoubleLong u; u.l = bits; return u.d; } //----------------------------------------------------------------- // Asegura espacio en el tape para 'extraSlots' mas, creciendo x1.5 // si hace falta (poco frecuente si pre-dimensionamos bien). //----------------------------------------------------------------- void EnsureTapeCapacity(int extraSlots) { int need = m_tapeLen + extraSlots; if(ArraySize(m_tape) >= need) return; int newCap = ArraySize(m_tape); if(newCap == 0) newCap = 64; while(newCap < need) newCap = newCap + newCap/2 + 64; ArrayResize(m_tape, newCap); } //----------------------------------------------------------------- // Escribe al tape UN valor individual (string/double/bool) en la // posicion actual (m_tapeLen), avanzando 1 o 2 slots segun tipo. // Determina el tipo mirando el rango [vs,ve): si es puramente // numerico -> DOUBLE; si es 'true'/'false'/'1' con longitud exacta // que matchea bool -> BOOL; si no, STRING (offset+len). // // NOTA sobre clasificacion Simple: para valores Simple (no // optimizables) mantenemos SIEMPRE tag=STRING, porque el fallback // debe preservar el texto crudo tal cual (incluye casos mixtos // como "1002032a" que no son 100% numericos). Solo los campos dentro // de un Optimizable valido (donde ya se valido que son numericos o // booleanos) se guardan como DOUBLE/BOOL. //----------------------------------------------------------------- void WriteStringSlot(int start, int end) { EnsureTapeCapacity(1); int len = end - start; long slot = ((long)TAPE_TAG_STRING << 60) | ((long)start << 24) | (long)len; m_tape[m_tapeLen] = slot; m_tapeLen++; } void WriteDoubleSlot(double d) { EnsureTapeCapacity(2); m_tape[m_tapeLen] = ((long)TAPE_TAG_DOUBLE << 60); m_tape[m_tapeLen + 1] = DoubleToBits(d); m_tapeLen += 2; } void WriteBoolSlot(bool v) { EnsureTapeCapacity(1); long slot = ((long)TAPE_TAG_BOOL << 60) | (v ? 1 : 0); m_tape[m_tapeLen] = slot; m_tapeLen++; } //----------------------------------------------------------------- // Parsea el "value part" [vs,ve) y escribe al tape: metadata + // 1 valor (Simple) o 4 valores (Optimizable). Misma logica de // deteccion de siempre (5 campos validos = optimizable). //----------------------------------------------------------------- void ParseAndWriteValue(int vs, int ve) { int fs[5], fe[5]; int nFields = 0; int p = vs; int segStart = vs; while(p < ve - 1 && nFields < 5) { if(m_buf[p] == '|' && m_buf[p + 1] == '|') { fs[nFields] = segStart; fe[nFields] = p; nFields++; p += 2; segStart = p; continue; } p++; } if(nFields < 5) { fs[nFields] = segStart; fe[nFields] = ve; nFields++; } if(nFields != 5) { // Simple: metadata (optim=0) + 1 valor STRING EnsureTapeCapacity(1); m_tape[m_tapeLen] = 0; // metadata: no optimizable, no enabled m_tapeLen++; WriteStringSlot(vs, ve); return; } int t0s=fs[0], t0e=fe[0]; TrimRange(t0s, t0e); int t1s=fs[1], t1e=fe[1]; TrimRange(t1s, t1e); int t2s=fs[2], t2e=fe[2]; TrimRange(t2s, t2e); int t3s=fs[3], t3e=fe[3]; TrimRange(t3s, t3e); int t4s=fs[4], t4e=fe[4]; TrimRange(t4s, t4e); bool f4isY = RangeEqualsLiteral(t4s, t4e, "Y"); bool f4isN = RangeEqualsLiteral(t4s, t4e, "N"); if(!f4isY && !f4isN) { EnsureTapeCapacity(1); m_tape[m_tapeLen] = 0; m_tapeLen++; WriteStringSlot(vs, ve); return; } bool f1Bool = RangeEqualsLiteral(t1s,t1e,"true") || RangeEqualsLiteral(t1s,t1e,"false"); bool f3Bool = RangeEqualsLiteral(t3s,t3e,"true") || RangeEqualsLiteral(t3s,t3e,"false"); bool f1Num = IsNumericRange(t1s, t1e); bool f3Num = IsNumericRange(t3s, t3e); bool validPair = (f1Bool && f3Bool) || (f1Num && f3Num); if(!validPair) { EnsureTapeCapacity(1); m_tape[m_tapeLen] = 0; m_tapeLen++; WriteStringSlot(vs, ve); return; } bool asBool = f1Bool; // si no, asNum (f1Num) // metadata: optimizable=1, enabled=f4isY EnsureTapeCapacity(1); long meta = TAPE_META_OPTIM_BIT; if(f4isY) meta |= TAPE_META_ENABLED_BIT; m_tape[m_tapeLen] = meta; m_tapeLen++; // Valor (campo 0): puede ser numerico o string libre -- el // campo 0 (el "valor" real) NO se valida como numerico/bool, // solo start/step lo son. Para preservar el texto tal cual si // no es puramente numerico, lo escribimos como STRING salvo // que matchee el mismo tipo que start/stop (heuristica: si // asBool, intentamos bool; si asNum, intentamos double; si no // matchea ninguno, STRING). if(asBool && (RangeEqualsLiteral(t0s,t0e,"true") || RangeEqualsLiteral(t0s,t0e,"false"))) WriteBoolSlot(RangeEqualsLiteral(t0s,t0e,"true")); else if(!asBool && IsNumericRange(t0s,t0e)) WriteDoubleSlot(ParseDoubleRange(t0s,t0e)); else WriteStringSlot(t0s, t0e); // Start, Step, Stop: start y stop ya sabemos que matchean el // tipo (validado arriba). Step no se valido -- igual heuristica // que el valor. if(asBool) { WriteBoolSlot(RangeEqualsLiteral(t1s,t1e,"true")); if(RangeEqualsLiteral(t2s,t2e,"true") || RangeEqualsLiteral(t2s,t2e,"false")) WriteBoolSlot(RangeEqualsLiteral(t2s,t2e,"true")); else if(IsNumericRange(t2s,t2e)) WriteDoubleSlot(ParseDoubleRange(t2s,t2e)); else WriteStringSlot(t2s,t2e); WriteBoolSlot(RangeEqualsLiteral(t3s,t3e,"true")); } else { WriteDoubleSlot(ParseDoubleRange(t1s,t1e)); if(IsNumericRange(t2s,t2e)) WriteDoubleSlot(ParseDoubleRange(t2s,t2e)); else if(RangeEqualsLiteral(t2s,t2e,"true") || RangeEqualsLiteral(t2s,t2e,"false")) WriteBoolSlot(RangeEqualsLiteral(t2s,t2e,"true")); else WriteStringSlot(t2s,t2e); WriteDoubleSlot(ParseDoubleRange(t3s,t3e)); } } public: CSetFileParser() : m_len(0), m_tapeLen(0), m_nodeCount(0), m_lastTapeSize(0) { m_notFoundNode.Bind(GetPointer(this), -1, false); } void Assign(const uchar &data[]) { int n = ArraySize(data); ArrayResize(m_buf, n); ArrayCopy(m_buf, data, 0, 0, n); m_len = n; } void AssignString(const string raw) { StringToCharArray(raw, m_buf, 0, WHOLE_ARRAY, CP_UTF8); int n = ArraySize(m_buf); if(n > 0 && m_buf[n - 1] == 0) n--; m_len = n; if(ArraySize(m_buf) != n) ArrayResize(m_buf, n); } bool AssignFile(const string path, int commonFlag = 0) { int handle = FileOpen(path, FILE_READ | FILE_BIN | commonFlag); if(handle == INVALID_HANDLE) { Print("CSetFileParser: no se pudo abrir '", path, "' err=", GetLastError()); return false; } ulong size = FileSize(handle); ArrayResize(m_buf, (int)size); if(size > 0) FileReadArray(handle, m_buf, 0, (int)size); m_len = (int)size; FileClose(handle); return true; } //----------------------------------------------------------------- // Parse: escanea el buffer y escribe TODO al tape (long[] plano), // sin crear ningun objeto CSetParam/CSetValue2. //----------------------------------------------------------------- bool Parse() { if(m_len <= 0) return false; m_tapeLen = 1; // slot 0 reservado para el contador de nodos int approxLines = 1; for(int i = 0; i < m_len; i++) if(m_buf[i] == '\n') approxLines++; // Pre-dimensionamos el tape. Si ya conocemos el tamaño real que // uso el Parse() anterior sobre este mismo buffer (m_lastTapeSize), // usamos ese +margen -- exacto, cero resizes en llamadas repetidas // (relevante si Parse() se llama muchas veces sobre el mismo // archivo, como en un benchmark o un re-parseo). Si es la // primera vez, usamos una estimacion generosa (7 slots/linea, // cubre el caso Optimizable-con-numeros que es el mas pesado: // 2 header + 4 valores*2 = 10, un poco por debajo a proposito // porque no todas las lineas son optimizables, pero mucho mas // realista que 4). int wantedInitial = (m_lastTapeSize > 0) ? (m_lastTapeSize + 16) : (approxLines * 7 + 32); EnsureTapeCapacity(wantedInitial - 1); // -1 porque ya contamos el slot[0] int nodeCount = 0; int i = 0; while(i < m_len) { int lineStart = i; int eq = -1; while(i < m_len && m_buf[i] != '\n') { if(eq < 0 && m_buf[i] == '=') eq = i; i++; } int lineEnd = i; if(i < m_len) i++; int realEnd = lineEnd; if(realEnd > lineStart && m_buf[realEnd - 1] == '\r') realEnd--; int s = lineStart; while(s < realEnd && IsSpaceByte(m_buf[s])) s++; if(s >= realEnd) continue; if(m_buf[s] == ';') continue; if(eq < 0 || eq >= realEnd) continue; int keyStart = s; int keyEnd = eq; while(keyEnd > keyStart && IsSpaceByte(m_buf[keyEnd - 1])) keyEnd--; // Key slot EnsureTapeCapacity(1); long keyLen = (long)(keyEnd - keyStart); m_tape[m_tapeLen] = ((long)keyStart << 32) | keyLen; m_tapeLen++; // metadata + value slot(s) ParseAndWriteValue(eq + 1, realEnd); nodeCount++; } m_tape[0] = nodeCount; m_nodeCount = nodeCount; m_lastTapeSize = m_tapeLen; return true; } //----------------------------------------------------------------- // Recorre el tape linealmente desde el nodo 'startSlot' (indice // del Key slot de un nodo) y devuelve el indice del Key slot del // SIGUIENTE nodo (o m_tapeLen si es el ultimo). Necesario porque // el tamaño de cada nodo es variable. //----------------------------------------------------------------- int NextNodeSlot(int nodeSlot) const { int i = nodeSlot + 1; // metadata slot long meta = m_tape[i]; i++; bool isOptim = (meta & TAPE_META_OPTIM_BIT) != 0; int valuesToSkip = isOptim ? 4 : 1; for(int v = 0; v < valuesToSkip; v++) { long slot = m_tape[i]; int tag = (int)((slot >> 60) & 0xF); i += (tag == TAPE_TAG_DOUBLE) ? 2 : 1; } return i; } CTapeNode operator[](const string key) { CTapeNode node; int slot = 1; for(int n = 0; n < m_nodeCount; n++) { long keySlotVal = m_tape[slot]; int keyOff = (int)(keySlotVal >> 32); int keyLen = (int)(keySlotVal & 0xFFFFFFFF); if(RangeEqualsLiteralPublic(keyOff, keyOff + keyLen, key)) { node.Bind(GetPointer(this), slot, true); return node; } slot = NextNodeSlot(slot); } node.Bind(GetPointer(this), -1, false); return node; } bool RangeEqualsLiteralPublic(int start, int end, const string key) const { return RangeEqualsLiteral(start, end, key); } int Count() const { return m_nodeCount; } CTapeNode At(int index) { CTapeNode node; int slot = 1; for(int n = 0; n < index && n < m_nodeCount; n++) slot = NextNodeSlot(slot); if(index < 0 || index >= m_nodeCount) { node.Bind(GetPointer(this), -1, false); return node; } node.Bind(GetPointer(this), slot, true); return node; } //----------------------------------------------------------------- // API interna usada por CTapeNode/CTapeValue. //----------------------------------------------------------------- long TapeAt(int idx) const { return m_tape[idx]; } string ResolveString(int start, int len) const { if(len <= 0) return ""; return CharArrayToString(m_buf, start, len, CP_ACP); } double GetBitsAsDouble(long bits) const { return BitsToDouble(bits); } }; //==================================================================== // Implementaciones de CTapeNode (necesitan CSetFileParser completa) //==================================================================== string CTapeNode::Key() const { if(!m_found || m_owner == NULL) return ""; long keySlotVal = m_owner.TapeAt(m_nodeSlot); int keyOff = (int)(keySlotVal >> 32); int keyLen = (int)(keySlotVal & 0xFFFFFFFF); return m_owner.ResolveString(keyOff, keyLen); } bool CTapeNode::IsOptimizable() const { if(!m_found || m_owner == NULL) return false; long meta = m_owner.TapeAt(m_nodeSlot + 1); return (meta & TAPE_META_OPTIM_BIT) != 0; } bool CTapeNode::IsEnabled() const { if(!m_found || m_owner == NULL) return false; long meta = m_owner.TapeAt(m_nodeSlot + 1); return (meta & TAPE_META_ENABLED_BIT) != 0; } CTapeValue CTapeNode::Valor() const { CTapeValue v; if(!m_found) { v.Bind(NULL, -1); return v; } v.Bind(m_owner, m_nodeSlot + 2); // justo despues de key+metadata return v; } CTapeValue CTapeNode::Start() const { CTapeValue v; if(!m_found || !IsOptimizable()) { v.Bind(NULL, -1); return v; } int slot = m_nodeSlot + 2; long s0 = m_owner.TapeAt(slot); int tag0 = (int)((s0 >> 60) & 0xF); slot += (tag0 == TAPE_TAG_DOUBLE) ? 2 : 1; // saltar Valor v.Bind(m_owner, slot); return v; } CTapeValue CTapeNode::Step() const { CTapeValue v; if(!m_found || !IsOptimizable()) { v.Bind(NULL, -1); return v; } int slot = m_nodeSlot + 2; for(int k = 0; k < 2; k++) // saltar Valor y Start { long s = m_owner.TapeAt(slot); int tag = (int)((s >> 60) & 0xF); slot += (tag == TAPE_TAG_DOUBLE) ? 2 : 1; } v.Bind(m_owner, slot); return v; } CTapeValue CTapeNode::Stop() const { CTapeValue v; if(!m_found || !IsOptimizable()) { v.Bind(NULL, -1); return v; } int slot = m_nodeSlot + 2; for(int k = 0; k < 3; k++) // saltar Valor, Start, Step { long s = m_owner.TapeAt(slot); int tag = (int)((s >> 60) & 0xF); slot += (tag == TAPE_TAG_DOUBLE) ? 2 : 1; } v.Bind(m_owner, slot); return v; } //==================================================================== // Implementaciones de CTapeValue //==================================================================== double CTapeValue::ToDouble() const { if(m_slotIndex < 0 || m_owner == NULL) return 0.0; long slot = m_owner.TapeAt(m_slotIndex); int tag = (int)((slot >> 60) & 0xF); if(tag == TAPE_TAG_DOUBLE) { long bits = m_owner.TapeAt(m_slotIndex + 1); return m_owner.GetBitsAsDouble(bits); } if(tag == TAPE_TAG_BOOL) return (slot & 1) ? 1.0 : 0.0; // STRING: parsear el prefijo numerico (comportamiento heredado) int off = (int)((slot >> 24) & 0xFFFFFF); int len = (int)(slot & 0xFFFFFF); string s = m_owner.ResolveString(off, len); return StringToDouble(s); } int CTapeValue::ToInt() const { return (int)ToDouble(); } long CTapeValue::ToLong() const { return (long)ToDouble(); } bool CTapeValue::ToBool() const { if(m_slotIndex < 0 || m_owner == NULL) return false; long slot = m_owner.TapeAt(m_slotIndex); int tag = (int)((slot >> 60) & 0xF); if(tag == TAPE_TAG_BOOL) return (slot & 1) != 0; if(tag == TAPE_TAG_DOUBLE) { long bits = m_owner.TapeAt(m_slotIndex + 1); return m_owner.GetBitsAsDouble(bits) != 0.0; } int off = (int)((slot >> 24) & 0xFFFFFF); int len = (int)(slot & 0xFFFFFF); string s = m_owner.ResolveString(off, len); return (s == "true" || s == "1"); } string CTapeValue::ToString() const { if(m_slotIndex < 0 || m_owner == NULL) return ""; long slot = m_owner.TapeAt(m_slotIndex); int tag = (int)((slot >> 60) & 0xF); if(tag == TAPE_TAG_STRING) { int off = (int)((slot >> 24) & 0xFFFFFF); int len = (int)(slot & 0xFFFFFF); return m_owner.ResolveString(off, len); } if(tag == TAPE_TAG_BOOL) return (slot & 1) ? "true" : "false"; if(tag == TAPE_TAG_DOUBLE) { long bits = m_owner.TapeAt(m_slotIndex + 1); return DoubleToString(m_owner.GetBitsAsDouble(bits), 8); } return ""; }