TICC/README.md

TICC

Toeplitz Inverse Covariance Clustering in native MQL5. Estimates a sparse precision matrix per market regime, so a regime is defined by which symbols are conditionally dependent and at what lag, not by how volatile the market is.

Code for the article Toeplitz Inverse Covariance Clustering in MQL5. The article explains the method and reports the measured results. This repo is the source it describes.

What is here

A correlation matrix over FX majors is mostly the dollar leg showing up five times. Inverting it gives the precision matrix, whose off-diagonal entries are partial correlations: the relationship between two symbols once every other symbol is accounted for. An L1 penalty drives unsupported entries to exact zero, so the estimate reads as a graph instead of a heatmap.

Each bar is stacked into a vector of n symbols over w bars, so the precision matrix covers lead and lag as well as same-bar structure. The block-Toeplitz constraint forces the model to be time invariant inside the window. For n = 5 and w = 5 that collapses 625 matrix entries to 115 free parameters.

The solver is ADMM. The theta step reduces to one scalar quadratic per eigenvalue, so there is no matrix inversion anywhere and the result is positive definite by construction. The Z step averages each Toeplitz group and soft-thresholds it, which is the exact proximal operator, not a projection applied afterwards.

Regimes are fitted by alternating the solver with a Viterbi pass over the whole label path, penalising switches by beta. Labels are therefore persistent rather than flipping on single-bar noise.

Linear algebra uses the MQL5 matrix and vector types. ALGLIB is used only to seed the initial clustering. No Python, no DLLs.

Layout

MQL5/Include/TICC/TICC_Types.mqh        config, regime and report structs, file paths
MQL5/Include/TICC/TICC_Math.mqh         block-Toeplitz group map, penalty taper, eigen helpers
MQL5/Include/TICC/TICC_GraphLasso.mqh   Toeplitz graphical lasso by ADMM
MQL5/Include/TICC/TICC_Engine.mqh       k-means seeding, Viterbi assignment, EM loop, persistence
MQL5/Include/TICC/TICC_Data.mqh         timestamp alignment, log returns, standardisation
MQL5/Include/TICC/TICC_Live.mqh         chart-side causal labelling
MQL5/Scripts/TICC/TICC_SolverTest.mq5   solver checks against a known sparsity pattern
MQL5/Scripts/TICC/TICC_Fit.mq5          fits regimes, model selection, held-out diagnostics
MQL5/Indicators/TICC/TICC_Regime.mq5    non-repainting regime ribbon
MQL5/Indicators/TICC/TICC_GraphPanel.mq5  dependency graph of the active regime
MQL5/Files/TICC/ticc_c_k6_b100.txt      fitted model the indicators default to

Copy the folders into your terminal's MQL5 directory and compile.

Running it

Run in this order. Each step assumes the previous one passed.

1. TICC_SolverTest.mq5 on any chart. Needs no market data, takes a few seconds, prints twenty checks. If any fail, stop here.
2. TICC_Fit.mq5 with InpSaveModel set to a file name. Put every basket symbol in Market Watch first, since it pulls history for all of them. Writes to MQL5\Files\TICC\, then reloads and verifies the copy reproduces the same labels.
3. TICC_Regime.mq5 on a chart, InpModelFile set to that file. The model carries its own basket, so the chart symbol does not have to be one of them.
4. TICC_GraphPanel.mq5 on the same chart, same model file, same commit lag.

The shipped model ticc_c_k6_b100.txt lets you skip step 2 and attach the indicators directly.

Settings behind the shipped model

EURUSD, GBPUSD, USDJPY, AUDUSD, USDCHF on H1, w = 5, K = 6, lambda = 0.01, taper sigma = 2.5, beta = 100, commit lag 5, 120-bar causal rolling standardisation, two-thirds training split.

InpBars counts back from whenever you run it, so refitting on a different day gives a different window. ALGLIB seeds k-means from MathRand, so set InpSeed if you want a fit to reproduce.

Non-repainting

The batch fit runs a backward pass and therefore sees the future. The indicators do not use it. They freeze the fitted regimes and run the recursion forward only, committing a label once the path behind it can no longer be revised. The cost is that the label arrives InpCommitLag bars late, so the newest bars stay unpainted. That gap is real and painting it would be repainting.

Things that cost me time

Four MQL5 behaviours worth knowing, all of which fail silently:

• EigenSymmetricDC rejects a matrix that is asymmetric in the last bit. A covariance built as X^T X always is, because the two dot products accumulate in different orders. Symmetrise before every call. Hand-built test matrices pass, so this only shows up on real data.
• ALGLIB seeds its k-means from MathRand on first use. Without an explicit MathSrand a fit will not reproduce even on identical data.
• A convergence test that can never be met turns the iteration cap into a hidden hyperparameter that decides the answer.
• An indicator that latches its recalculation guard before doing the work turns one early failure into a permanent one. Latch it only after a pass that painted. Returning rates_total when nothing was drawn tells every iCustom caller that an empty buffer is a finished answer.

References

• Hallac, Vare, Boyd, Leskovec (2017). Toeplitz Inverse Covariance-Based Clustering of Multivariate Time Series Data. KDD 2017.
• Boyd, Parikh, Chu, Peleato, Eckstein (2011). Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Foundations and Trends in Machine Learning 3(1).
• Friedman, Hastie, Tibshirani (2008). Sparse Inverse Covariance Estimation with the Graphical Lasso. Biostatistics 9(3).
• Dempster (1972). Covariance Selection. Biometrics 28(1).

Disclaimer

Educational code. Past behaviour of any model or dataset says nothing about future results. Test on your own data and broker conditions before drawing conclusions.