L1 Trend Filtering
The primary goal of L1 trend filtering is to extract the underlying trend from a time series in such a way that:
- long-term dynamics of the data are preserved;
- short-term fluctuations and noise are suppressed;
- structural breakpoints (changes in trend slope) are automatically detected.
Unlike classical smoothing methods, this approach does not enforce smoothness of the trend and instead produces a piecewise-linear approximation, which is particularly important for financial time series analysis.
L1 Trend Filtering Method
The main idea of L1 trend filtering is to find a trend that is close to the original data but contains as few changes in slope as possible. Unlike classical smoothing methods that minimize the squared curvature, the L1 approach minimizes the sum of absolute values of second differences.
This leads to a fundamentally different result:
- most second differences become equal to zero,
- the trend is automatically split into linear segments.
Thus, the L1 filter does not attempt to make the trend smooth but instead finds the minimal number of structural changes that explain the observed data. This makes the method particularly suitable for financial time series, where dynamics often consist of sequences of quasi-linear growth and decline phases.
In L1 trend filtering, the quadratic penalty on second differences is replaced by the L1 norm, and the trend is defined as the solution to a convex optimization problem:
In matrix form:
where:
- y — input time series;
- x — the estimated trend;
- D — second-difference matrix;
- λ>=0 — regularization parameter.
The use of the L1 norm leads to a fundamentally different result: many second differences become zero, meaning that the trend is piecewise linear.
The second difference is defined as:
If
, then the points
lie on a straight line.
Therefore, a zero second difference corresponds to a linear segment of the trend, while a nonzero second difference corresponds to a breakpoint. The L1 norm promotes sparsity in the vector Dx, meaning that most second differences become zero. This implies that over the corresponding intervals, the trend is linear. Points where second differences are nonzero are interpreted as trend breakpoints.
Thus, the L1 Trend Filtering method automatically constructs the trend as a set of linear segments connected at points of structural change.
Main properties of L1 trend filtering:
- The trend consists of linear segments;
- Breakpoints are interpreted as structural changes in the time series;
- At λ = 0, the trend coincides with the original data;
- For sufficiently large λ, the trend becomes exactly the best linear approximation;
- Computational complexity remains linear in the number of observations.
