//+------------------------------------------------------------------+ //| FracDiff.mqh | //| Fixed-width fractional differentiation (Lopez de Prado AFML c.5)| //+------------------------------------------------------------------+ //| WHY THIS EXISTS | //| | //| FeatureBuilder.mqh does not feed non-stationary prices. It feeds | //| (close-open)/atr, (close-close[20])/atr, (close-SMA20)/atr and | //| volume/volume_base. Those are INTEGER-order differences: d = 1. | //| They are stationary and they are memory-less. | //| | //| So this module does not fix a stationarity bug - there isn't one. | //| It fixes OVER-differencing: at d = 1 the correlation between the | //| transformed series and the price level is ~0, so every feature | //| the network sees has been stripped of where price actually IS. | //| Fractional d keeps that level information and still passes ADF. | //| | //| THE RECURRENCE | //| w_0 = 1, w_k = -w_{k-1} * (d - k + 1) / k | //| X_t^(d) = sum_{k=0}^{L} w_k * X_{t-k} | //| Truncated at the first |w_k| < tau, so every output uses the same | //| L taps and the transform is translation-invariant. | //| | //| *** THE 1024-BAR TRAP *** | //| The stdlib series wrappers (CiClose/CiOpen/... via CopyBuffer) | //| return 0.0 IN SILENCE past shift 1023. An FFD window is routinely | //| WIDER than that: at tau = 1e-5, d = 0.4 needs 1458 taps and | //| d = 0.1 needs 4076. Reading those through m_Close.GetData() would | //| multiply real weights by silent zeros and produce a confident, | //| wrong number that no guard catches. | //| Therefore this module calls CopyClose/CopyHigh/... DIRECTLY and | //| never touches the indicator series. Callers must do the same. | //| | //| Tap budget (from research/fracdiff.py, measured): | //| d tau=1e-5 tau=1e-4 tau=1e-3 | //| 0.10 4076 503 62 | //| 0.30 2275 388 66 | //| 0.40 1458 282 55 | //| 0.50 927 200 44 | //| tau = 1e-4 keeps every d >= 0.1 inside ~500 bars, which is the | //| setting this module defaults to. | //+------------------------------------------------------------------+ #ifndef FRACDIFF_MQH #define FRACDIFF_MQH #define FFD_MAX_TAPS 4096 #define FFD_DEFAULT_TAU 1e-4 //+------------------------------------------------------------------+ //| Weight vector, lag-ordered: w[0] multiplies X_t, w[1] X_{t-1}, ...| //| Returns the tap count, or 0 on a bad d. | //+------------------------------------------------------------------+ int FFDWeights(const double d, double &w[], const double tau = FFD_DEFAULT_TAU) { if(d < 0.0 || !MathIsValidNumber(d)) return 0; ArrayResize(w, FFD_MAX_TAPS); w[0] = 1.0; int n = 1; for(int k = 1; k < FFD_MAX_TAPS; k++) { double wk = -w[k - 1] * (d - k + 1.0) / (double)k; if(MathAbs(wk) < tau) break; w[k] = wk; n++; } ArrayResize(w, n); return n; } //+------------------------------------------------------------------+ //| CFracDiff - caches one weight vector and applies it to a series. | //| | //| Series convention is MQL5's: index 0 = newest. shift is the bar | //| to transform; the taps walk toward OLDER bars (increasing index), | //| so nothing here can read the future. | //+------------------------------------------------------------------+ class CFracDiff { private: double m_w[]; int m_taps; double m_d; double m_tau; string m_symbol; ENUM_TIMEFRAMES m_period; public: CFracDiff(void) : m_taps(0), m_d(0.0), m_tau(FFD_DEFAULT_TAU), m_symbol(NULL), m_period(PERIOD_CURRENT) { } bool Init(const string symbol, const ENUM_TIMEFRAMES period, const double d, const double tau = FFD_DEFAULT_TAU) { m_symbol = symbol; m_period = period; m_d = d; m_tau = tau; m_taps = FFDWeights(d, m_w, tau); if(m_taps <= 0) { Print(__FUNCTION__ + ": bad d=" + DoubleToString(d, 4)); return false; } //--- A d so small the window exceeds what we will ever load is a config //--- error, not a runtime degradation: fail loudly at Init rather than //--- return a silently short-windowed transform for the rest of the run. if(m_taps > FFD_MAX_TAPS - 1) { Print(__FUNCTION__ + ": d=" + DoubleToString(d, 4) + " needs " + IntegerToString(m_taps) + " taps - raise tau or raise d"); return false; } return true; } int Taps(void) const { return m_taps; } double D(void) const { return m_d; } //--- Bars of warm-up a caller must leave before the oldest usable output. int WarmupBars(void) const { return m_taps; } //+---------------------------------------------------------------+ //| Apply to an arbitrary pre-loaded array (index 0 = newest). | //| Returns false if the array cannot cover shift + taps. | //+---------------------------------------------------------------+ bool ApplyArray(const double &src[], const int shift, double &out) const { if(m_taps <= 0) return false; int need = shift + m_taps; if(shift < 0 || need > ArraySize(src)) return false; double acc = 0.0; for(int k = 0; k < m_taps; k++) { double v = src[shift + k]; //--- == EMPTY_VALUE explicitly: DBL_MAX passes MathIsValidNumber and //--- would dominate the sum while still looking like a real reading. //--- Same reasoning as the price guards in FeatureBuilder.mqh. if(!MathIsValidNumber(v) || v == EMPTY_VALUE || v <= 0.0) return false; acc += m_w[k] * MathLog(v); // LOG price: scale-free across the fleet } out = acc; return true; } //+---------------------------------------------------------------+ //| Load straight from the terminal and transform one bar. | //| | //| CopyClose, NOT CiClose::GetData - see the 1024-bar trap above. | //+---------------------------------------------------------------+ bool CloseAt(const int shift, double &out) const { if(m_taps <= 0 || shift < 0) return false; double buf[]; int need = m_taps; if(CopyClose(m_symbol, m_period, shift, need, buf) != need) return false; //--- CopyClose with a start/count returns OLDEST-first; flip to series //--- order so index 0 is the bar at `shift`. ArraySetAsSeries(buf, true); return ApplyArray(buf, 0, out); } //+---------------------------------------------------------------+ //| Same, for tick volume (log1p: CFD bars can legitimately be 0). | //+---------------------------------------------------------------+ bool VolumeAt(const int shift, double &out) const { if(m_taps <= 0 || shift < 0) return false; long vbuf[]; int need = m_taps; if(CopyTickVolume(m_symbol, m_period, shift, need, vbuf) != need) return false; ArraySetAsSeries(vbuf, true); double acc = 0.0; for(int k = 0; k < m_taps; k++) acc += m_w[k] * MathLog(1.0 + (double)vbuf[k]); out = acc; return true; } }; #endif // FRACDIFF_MQH