research
∙
09/07/2023
Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck
This work investigates the nuanced algorithm design choices for deep lea...
share
research
∙
07/27/2023
Autocalibrating Gaze Tracking: A Demonstration through Gaze Typing
Miscalibration of gaze tracking devices and the resulting need for repea...
research
∙
06/01/2023
Exposing Attention Glitches with Flip-Flop Language Modeling
Why do large language models sometimes output factual inaccuracies and e...
research
∙
02/28/2023
Learning Hidden Markov Models Using Conditional Samples
This paper is concerned with the computational complexity of learning th...
research
∙
11/02/2022
Neural Active Learning on Heteroskedastic Distributions
Models that can actively seek out the best quality training data hold th...
research
∙
10/19/2022
Transformers Learn Shortcuts to Automata
Algorithmic reasoning requires capabilities which are most naturally und...
research
∙
09/01/2022
Recurrent Convolutional Neural Networks Learn Succinct Learning Algorithms
Neural Networks (NNs) struggle to efficiently learn certain problems, su...
research
∙
07/18/2022
Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit
There is mounting empirical evidence of emergent phenomena in the capabi...
research
∙
02/28/2022
Understanding Contrastive Learning Requires Incorporating Inductive Biases
Contrastive learning is a popular form of self-supervised learning that ...
research
∙
11/19/2021
Machine Learning for Mechanical Ventilation Control (Extended Abstract)
Mechanical ventilation is one of the most widely used therapies in the I...
research
∙
10/21/2021
Anti-Concentrated Confidence Bonuses for Scalable Exploration
Intrinsic rewards play a central role in handling the exploration-exploi...
research
∙
10/19/2021
Inductive Biases and Variable Creation in Self-Attention Mechanisms
Self-attention, an architectural motif designed to model long-range inte...
research
∙
10/12/2021
Sparsity in Partially Controllable Linear Systems
A fundamental concept in control theory is that of controllability, wher...
research
∙
03/01/2021
Acceleration via Fractal Learning Rate Schedules
When balancing the practical tradeoffs of iterative methods for large-sc...
research
∙
02/19/2021
Deluca – A Differentiable Control Library: Environments, Methods, and Benchmarking
We present an open-source library of natively differentiable physics and...
research
∙
02/12/2021
Machine Learning for Mechanical Ventilation Control
We consider the problem of controlling an invasive mechanical ventilator...
research
∙
10/26/2020
Stochastic Optimization with Laggard Data Pipelines
State-of-the-art optimization is steadily shifting towards massively par...
research
∙
02/26/2020
Disentangling Adaptive Gradient Methods from Learning Rates
We investigate several confounding factors in the evaluation of optimiza...
research
∙
02/06/2020
No-Regret Prediction in Marginally Stable Systems
We consider the problem of online prediction in a marginally stable line...
research
∙
06/11/2019
Calibration, Entropy Rates, and Memory in Language Models
Building accurate language models that capture meaningful long-term depe...
research
∙
05/23/2019
Robust guarantees for learning an autoregressive filter
The optimal predictor for a linear dynamical system (with hidden state a...
research
∙
02/12/2019
Extreme Tensoring for Low-Memory Preconditioning
State-of-the-art models are now trained with billions of parameters, rea...
research
∙
06/08/2018
The Case for Full-Matrix Adaptive Regularization
Adaptive regularization methods come in diagonal and full-matrix variant...
research
∙
02/12/2018
Spectral Filtering for General Linear Dynamical Systems
We give a polynomial-time algorithm for learning latent-state linear dyn...
research
∙
11/02/2017
Learning Linear Dynamical Systems via Spectral Filtering
We present an efficient and practical algorithm for the online predictio...
research
∙
10/27/2017
Not-So-Random Features
We propose a principled method for kernel learning, which relies on a Fo...
research
∙
07/31/2017