research
∙
08/31/2023
Language-Conditioned Path Planning
Contact is at the core of robotic manipulation. At times, it is desired ...
share
research
∙
08/23/2023
Language Reward Modulation for Pretraining Reinforcement Learning
Using learned reward functions (LRFs) as a means to solve sparse-reward ...
research
∙
08/11/2023
The Impact of Overall Optimization on Warehouse Automation
In this study, we propose a novel approach for investigating optimizatio...
research
∙
07/31/2023
Convolutional Occupancy Models for Dense Packing of Complex, Novel Objects
Dense packing in pick-and-place systems is an important feature in many ...
research
∙
07/31/2023
Learning to Model the World with Language
To interact with humans in the world, agents need to understand the dive...
research
∙
07/07/2023
SpawnNet: Learning Generalizable Visuomotor Skills from Pre-trained Networks
The existing internet-scale image and video datasets cover a wide range ...
research
∙
06/21/2023
Improving Long-Horizon Imitation Through Instruction Prediction
Complex, long-horizon planning and its combinatorial nature pose steep c...
research
∙
06/16/2023
ALP: Action-Aware Embodied Learning for Perception
Current methods in training and benchmarking vision models exhibit an ov...
research
∙
06/02/2023
Probabilistic Adaptation of Text-to-Video Models
Large text-to-video models trained on internet-scale data have demonstra...
research
∙
05/31/2023
Accelerating Reinforcement Learning with Value-Conditional State Entropy Exploration
A promising technique for exploration is to maximize the entropy of visi...
research
∙
05/30/2023
Blockwise Parallel Transformer for Long Context Large Models
Transformers have emerged as the cornerstone of state-of-the-art natural...
research
∙
05/26/2023
Emergent Agentic Transformer from Chain of Hindsight Experience
Large transformer models powered by diverse data and model scale have do...
research
∙
05/25/2023
DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models
Learning from human feedback has been shown to improve text-to-image mod...
research
∙
05/25/2023
The False Promise of Imitating Proprietary LLMs
An emerging method to cheaply improve a weaker language model is to fine...
research
∙
05/23/2023
Video Prediction Models as Rewards for Reinforcement Learning
Specifying reward signals that allow agents to learn complex behaviors i...
research
∙
05/10/2023
Self-Supervised Instance Segmentation by Grasping
Instance segmentation is a fundamental skill for many robotic applicatio...
research
∙
05/04/2023
Masked Trajectory Models for Prediction, Representation, and Control
We introduce Masked Trajectory Models (MTM) as a generic abstraction for...
research
∙
05/03/2023
Distributional Instance Segmentation: Modeling Uncertainty and High Confidence Predictions with Latent-MaskRCNN
Object recognition and instance segmentation are fundamental skills in a...
research
∙
03/31/2023
Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?
We present the largest and most comprehensive empirical study of pre-tra...
research
∙
03/07/2023
Foundation Models for Decision Making: Problems, Methods, and Opportunities
Foundation models pretrained on diverse data at scale have demonstrated ...
research
∙
03/02/2023
Preference Transformer: Modeling Human Preferences using Transformers for RL
Preference-based reinforcement learning (RL) provides a framework to tra...
research
∙
02/23/2023
Aligning Text-to-Image Models using Human Feedback
Deep generative models have shown impressive results in text-to-image sy...
research
∙
02/19/2023
Robust and Versatile Bipedal Jumping Control through Multi-Task Reinforcement Learning
This work aims to push the limits of agility for bipedal robots by enabl...
research
∙
02/13/2023
Guiding Pretraining in Reinforcement Learning with Large Language Models
Reinforcement learning algorithms typically struggle in the absence of a...
research
∙
02/10/2023
The Wisdom of Hindsight Makes Language Models Better Instruction Followers
Reinforcement learning has seen wide success in finetuning large languag...
research
∙
02/10/2023
Controllability-Aware Unsupervised Skill Discovery
One of the key capabilities of intelligent agents is the ability to disc...
research
∙
02/06/2023
Languages are Rewards: Chain of Hindsight Finetuning using Human Feedback
Learning from human preferences is important for language models to be h...
research
∙
02/05/2023
Multi-View Masked World Models for Visual Robotic Manipulation
Visual robotic manipulation research and applications often use multiple...
research
∙
02/02/2023
Language Quantized AutoEncoders: Towards Unsupervised Text-Image Alignment
Recent progress in scaling up large language models has shown impressive...
research
∙
01/31/2023
Learning Universal Policies via Text-Guided Video Generation
A goal of artificial intelligence is to construct an agent that can solv...
research
∙
11/23/2022
Multi-Environment Pretraining Enables Transfer to Action Limited Datasets
Using massive datasets to train large-scale models has emerged as a domi...
research
∙
11/23/2022
Masked Autoencoding for Scalable and Generalizable Decision Making
We are interested in learning scalable agents for reinforcement learning...
research
∙
11/21/2022
VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models
Diffusion models have shown impressive results in text-to-image synthesi...
research
∙
11/03/2022
StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCS
Most existing methods for category-level pose estimation rely on object ...
research
∙
10/25/2022
Sim-to-Real via Sim-to-Seg: End-to-end Off-road Autonomous Driving Without Real Data
Autonomous driving is complex, requiring sophisticated 3D scene understa...
research
∙
10/24/2022
Dichotomy of Control: Separating What You Can Control from What You Cannot
Future- or return-conditioned supervised learning is an emerging paradig...
research
∙
10/24/2022
FCM: Forgetful Causal Masking Makes Causal Language Models Better Zero-Shot Learners
Large language models (LLM) trained using the next-token-prediction obje...
research
∙
10/24/2022
Instruction-Following Agents with Jointly Pre-Trained Vision-Language Models
Humans are excellent at understanding language and vision to accomplish ...
research
∙
10/19/2022
CLUTR: Curriculum Learning via Unsupervised Task Representation Learning
Reinforcement Learning (RL) algorithms are often known for sample ineffi...
research
∙
10/14/2022
Skill-Based Reinforcement Learning with Intrinsic Reward Matching
While unsupervised skill discovery has shown promise in autonomously acq...
research
∙
10/13/2022
Autoregressive Uncertainty Modeling for 3D Bounding Box Prediction
3D bounding boxes are a widespread intermediate representation in many c...
research
∙
10/06/2022
Real-World Robot Learning with Masked Visual Pre-training
In this work, we explore self-supervised visual pre-training on images f...
research
∙
10/05/2022
Temporally Consistent Video Transformer for Long-Term Video Prediction
Generating long, temporally consistent video remains an open challenge i...
research
∙
09/16/2022
Reducing Variance in Temporal-Difference Value Estimation via Ensemble of Deep Networks
In temporal-difference reinforcement learning algorithms, variance in va...
research
∙
09/15/2022
HARP: Autoregressive Latent Video Prediction with High-Fidelity Image Generator
Video prediction is an important yet challenging problem; burdened with ...
research
∙
09/15/2022
Multi-Objective Policy Gradients with Topological Constraints
Multi-objective optimization models that encode ordered sequential const...
research
∙
08/03/2022
AdaCat: Adaptive Categorical Discretization for Autoregressive Models
Autoregressive generative models can estimate complex continuous data di...
research
∙
06/29/2022
Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervision
Commercial and industrial deployments of robot fleets often fall back on...
research
∙
06/28/2022
Masked World Models for Visual Control
Visual model-based reinforcement learning (RL) has the potential to enab...
research
∙
06/28/2022