research
∙
02/13/2023
Simple Hardware-Efficient Long Convolutions for Sequence Modeling
State space models (SSMs) have high performance on long sequence modelin...
share
research
∙
12/28/2022
Hungry Hungry Hippos: Towards Language Modeling with State Space Models
State space models (SSMs) have demonstrated state-of-the-art sequence mo...
research
∙
06/24/2022
Arithmetic Circuits, Structured Matrices and (not so) Deep Learning
This survey presents a necessarily incomplete (and biased) overview of r...
research
∙
06/24/2022
How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections
Linear time-invariant state space models (SSM) are a classical model fro...
research
∙
05/27/2022
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Transformers are slow and memory-hungry on long sequences, since the tim...
research
∙
04/06/2022
Computing expected multiplicities for bag-TIDBs with bounded multiplicities
In this work, we study the problem of computing a tuple's expected multi...
research
∙
12/02/2021
Worst-case Optimal Binary Join Algorithms under General â„“_p Constraints
Worst-case optimal join algorithms have so far been studied in two broad...
research
∙
11/30/2021
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models
Overparameterized neural networks generalize well but are expensive to t...
research
∙
10/28/2021
Scatterbrain: Unifying Sparse and Low-rank Attention Approximation
Recent advances in efficient Transformers have exploited either the spar...
research
∙
10/26/2021
Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers
Recurrent neural networks (RNNs), temporal convolutions, and neural diff...
research
∙
12/29/2020
Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps
Modern neural network architectures use structured linear transformation...
research
∙
08/17/2020
HiPPO: Recurrent Memory with Optimal Polynomial Projections
A central problem in learning from sequential data is representing cumul...
research
∙
03/21/2020
Covering the Relational Join
In this paper, we initiate a theoretical study of what we call the join ...
research
∙
03/12/2020
Topology Dependent Bounds For FAQs
In this paper, we prove topology dependent bounds on the number of round...
research
∙
07/19/2019
Sparse Recovery for Orthogonal Polynomial Transforms
In this paper we consider the following sparse recovery problem. We have...
research
∙
03/14/2019
Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizations
Fast linear transforms are ubiquitous in machine learning, including the...
research
∙
10/04/2018
Learning Compressed Transforms with Low Displacement Rank
The low displacement rank (LDR) framework for structured matrices repres...
research
∙
04/05/2018
Hypertree Decompositions Revisited for PGMs
We revisit the classical problem of exact inference on probabilistic gra...
research
∙
02/08/2018
