Multivariate Trend Filtering for Lattice Data

We study a multivariate version of trend filtering, called Kronecker trend filtering or KTF, for the case in which the design points form a lattice in d dimensions. KTF is a natural extension of univariate trend filtering (Steidl et al., 2006; Kim et al., 2009; Tibshirani, 2014), and is defined by minimizing a penalized least squares problem whose penalty term sums the absolute (higher-order) differences of the parameter to be estimated along each of the coordinate directions. The corresponding penalty operator can be written in terms of Kronecker products of univariate trend filtering penalty operators, hence the name Kronecker trend filtering. Equivalently, one can view KTF in terms of an ℓ_1-penalized basis regression problem where the basis functions are tensor products of falling factorial functions, a piecewise polynomial (discrete spline) basis that underlies univariate trend filtering. This paper is a unification and extension of the results in Sadhanala et al. (2016, 2017). We develop a complete set of theoretical results that describe the behavior of k^th order Kronecker trend filtering in d dimensions, for every k ≥ 0 and d ≥ 1. This reveals a number of interesting phenomena, including the dominance of KTF over linear smoothers in estimating heterogeneously smooth functions, and a phase transition at d=2(k+1), a boundary past which (on the high dimension-to-smoothness side) linear smoothers fail to be consistent entirely. We also leverage recent results on discrete splines from Tibshirani (2020), in particular, discrete spline interpolation results that enable us to extend the KTF estimate to any off-lattice location in constant-time (independent of the size of the lattice n).


page 5

page 24


Trend Filtering on Graphs

We introduce a family of adaptive estimators on graphs, based on penaliz...

Divided Differences, Falling Factorials, and Discrete Splines: Another Look at Trend Filtering and Related Problems

This paper serves as a postscript of sorts to Tibshirani (2014); Wang et...

The Falling Factorial Basis and Its Statistical Applications

We study a novel spline-like basis, which we name the "falling factorial...

L_1 Trend Filtering: A Modern Statistical Tool for Time-Domain Astronomy and Astronomical Spectroscopy

The problem of estimating a one-dimensional signal possessing mixed degr...

Tensor denoising with trend filtering

We extend the notion of trend filtering to tensors by considering the k^...

Exponential Family Trend Filtering on Lattices

Trend filtering is a modern approach to nonparametric regression that is...

Additive Models with Trend Filtering

We consider additive models built with trend filtering, i.e., additive m...

Please sign up or login with your details

Forgot password? Click here to reset