b'Yilun Du'

research

∙ 09/15/2023

Compositional Foundation Models for Hierarchical Planning

To make effective decisions in novel environments with long-horizon goal...

0 Anurag Ajay, et al. ∙

research

∙ 07/24/2023

3D-LLM: Injecting the 3D World into Large Language Models

Large language models (LLMs) and Vision-Language Models (VLMs) have been...

0 Yining Hong, et al. ∙

research

∙ 07/05/2023

Building Cooperative Embodied Agents Modularly with Large Language Models

Large Language Models (LLMs) have demonstrated impressive planning abili...

0 Hongxin Zhang, et al. ∙

research

∙ 06/08/2023

Unsupervised Compositional Concepts Discovery with Text-to-Image Generative Models

Text-to-image generative models have enabled high-resolution image synth...

19 Nan Liu, et al. ∙

research

∙ 06/02/2023

Probabilistic Adaptation of Text-to-Video Models

Large text-to-video models trained on internet-scale data have demonstra...

5 Mengjiao Yang, et al. ∙

research

∙ 05/31/2023

FlowCam: Training Generalizable 3D Radiance Fields without Camera Poses via Pixel-Aligned Scene Flow

Reconstruction of 3D neural fields from posed images has emerged as a pr...

0 Cameron Smith, et al. ∙

research

∙ 05/23/2023

Improving Factuality and Reasoning in Language Models through Multiagent Debate

Large language models (LLMs) have demonstrated remarkable capabilities i...

4 Yilun Du, et al. ∙

research

∙ 05/22/2023

Training Diffusion Models with Reinforcement Learning

Diffusion models are a class of flexible generative models trained with ...

0 Kevin Black, et al. ∙

research

∙ 04/17/2023

Learning to Render Novel Views from Wide-Baseline Stereo Pairs

We introduce a method for novel view synthesis given only a single wide-...

0 Yilun Du, et al. ∙

research

∙ 03/28/2023

Planning with Sequence Models through Iterative Energy Minimization

Recent works have shown that sequence modeling can be effectively used t...

0 Hongyi Chen, et al. ∙

research

∙ 03/20/2023

3D Concept Learning and Reasoning from Multi-View Images

Humans are able to accurately reason in 3D by gathering multi-view obser...

0 Yining Hong, et al. ∙

research

∙ 03/13/2023

NeuSE: Neural SE(3)-Equivariant Embedding for Consistent Spatial Understanding with Objects

We present NeuSE, a novel Neural SE(3)-Equivariant Embedding for objects...

0 Jiahui Fu, et al. ∙

research

∙ 03/07/2023

Foundation Models for Decision Making: Problems, Methods, and Opportunities

Foundation models pretrained on diverse data at scale have demonstrated ...

0 Sherry Yang, et al. ∙

research

∙ 02/22/2023

Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC

Since their introduction, diffusion models have quickly become the preva...

0 Yilun Du, et al. ∙

research

∙ 02/07/2023

Local Neural Descriptor Fields: Locally Conditioned Object Representations for Manipulation

A robot operating in a household environment will see a wide range of un...

0 Ethan Chun, et al. ∙

research

∙ 01/31/2023

Learning Universal Policies via Text-Guided Video Generation

A goal of artificial intelligence is to construct an agent that can solv...

7 Yilun Du, et al. ∙

research

∙ 12/12/2022

MIRA: Mental Imagery for Robotic Affordances

Humans form mental images of 3D scenes to support counterfactual imagina...

0 Lin Yen-Chen, et al. ∙

research

∙ 12/06/2022

Visibility-Aware Navigation Among Movable Obstacles

In this paper, we examine the problem of visibility-aware robot navigati...

0 Jose Muguira-Iturralde, et al. ∙

research

∙ 11/28/2022

Is Conditional Generative Modeling all you need for Decision-Making?

Recent improvements in conditional generative modeling have made it poss...

0 Anurag Ajay, et al. ∙

research

∙ 11/17/2022

SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fields

We present a method for performing tasks involving spatial relations bet...

0 Anthony Simeonov, et al. ∙

research

∙ 11/08/2022

Self-conditioned Embedding Diffusion for Text Generation

Can continuous diffusion models bring the same performance breakthrough ...

1 Robin Strudel, et al. ∙

research

∙ 10/20/2022

Composing Ensembles of Pre-trained Models via Iterative Consensus

Large pre-trained models exhibit distinct and complementary capabilities...

1 Shuang Li, et al. ∙

research

∙ 08/01/2022

Robust Change Detection Based on Neural Descriptor Fields

The ability to reason about changes in the environment is crucial for ro...

1 Jiahui Fu, et al. ∙

research

∙ 07/22/2022

Seeing 3D Objects in a Single Image via Self-Supervised Static-Dynamic Disentanglement

Human perception reliably identifies movable and immovable parts of 3D s...

0 Prafull Sharma, et al. ∙

research

∙ 07/13/2022

3D Concept Grounding on Neural Fields

In this paper, we address the challenging problem of 3D concept groundin...

5 Yining Hong, et al. ∙

research

∙ 06/30/2022

Learning Iterative Reasoning through Energy Minimization

Deep learning has excelled on complex pattern recognition tasks such as ...

9 Yilun Du, et al. ∙

research

∙ 06/03/2022

Compositional Visual Generation with Composable Diffusion Models

Large text-guided diffusion models, such as DALLE-2, are able to generat...

8 Nan Liu, et al. ∙

research

∙ 05/20/2022

Planning with Diffusion for Flexible Behavior Synthesis

Model-based reinforcement learning methods often use learning only for t...

2 Michael Janner, et al. ∙

research

∙ 05/02/2022

Streaming Inference for Infinite Non-Stationary Clustering

Learning from a continuous stream of non-stationary data in an unsupervi...

0 Rylan Schaeffer, et al. ∙

research

∙ 04/04/2022

Learning Neural Acoustic Fields

Our environment is filled with rich and dynamic acoustic information. Wh...

3 Andrew Luo, et al. ∙

research

∙ 02/03/2022

Pre-Trained Language Models for Interactive Decision-Making

Language model (LM) pre-training has proven useful for a wide variety of...

68 Shuang Li, et al. ∙

research

∙ 12/09/2021

Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation

We present Neural Descriptor Fields (NDFs), an object representation tha...

7 Anthony Simeonov, et al. ∙

research

∙ 11/17/2021

Learning to Compose Visual Relations

The visual world around us can be described as a structured set of objec...

22 Nan Liu, et al. ∙

research

∙ 11/11/2021

Learning Signal-Agnostic Manifolds of Neural Fields

Deep neural networks have been used widely to learn the latent structure...

6 Yilun Du, et al. ∙

research

∙ 11/04/2021

Unsupervised Learning of Compositional Energy Concepts

Humans are able to rapidly understand scenes by utilizing concepts extra...

5 Yilun Du, et al. ∙

research

∙ 10/14/2021

The Neural MMO Platform for Massively Multiagent Research

Neural MMO is a computationally accessible research platform that combin...

0 Joseph Suarez, et al. ∙

research

∙ 10/07/2021

Weakly Supervised Human-Object Interaction Detection in Video via Contrastive Spatiotemporal Regions

We introduce the task of weakly supervised learning for detecting human ...

11 Shuang Li, et al. ∙

research

∙ 05/03/2021

Curious Representation Learning for Embodied Intelligence

Self-supervised representation learning has achieved remarkable success ...

0 Yilun Du, et al. ∙

research

∙ 04/08/2021

3D Shape Generation and Completion through Point-Voxel Diffusion

We propose a novel approach for probabilistic generative modeling of 3D ...

2 Linqi Zhou, et al. ∙

research

∙ 12/17/2020

Neural Radiance Flow for 4D View Synthesis and Video Processing

We present a method, Neural Radiance Flow (NeRFlow),to learn a 4D spatia...

2 Yilun Du, et al. ∙

research

∙ 12/02/2020

Improved Contrastive Divergence Training of Energy Based Models

We propose several different techniques to improve contrastive divergenc...

3 Yilun Du, et al. ∙

research

∙ 11/24/2020

Energy-Based Models for Continual Learning

We motivate Energy-Based Models (EBMs) as a promising model class for co...

1 Shuang Li, et al. ∙

research

∙ 11/16/2020

A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objects

We present a framework for solving long-horizon planning problems involv...

6 Anthony Simeonov, et al. ∙

research

∙ 11/06/2020

Learning Online Data Association

When an agent interacts with a complex environment, it receives a stream...

10 Yilun Du, et al. ∙

research

∙ 07/24/2020

Unsupervised Discovery of 3D Physical Objects from Video

We study the problem of unsupervised physical object discovery. Unlike e...

7 Yilun Du, et al. ∙

research

∙ 04/27/2020

Energy-based models for atomic-resolution protein conformations

We propose an energy-based model (EBM) of protein conformations that ope...

9 Yilun Du, et al. ∙

research

∙ 04/13/2020

Compositional Visual Generation and Inference with Energy Based Models

A vital aspect of human intelligence is the ability to compose increasin...

1 Yilun Du, et al. ∙

research

∙ 01/31/2020

Neural MMO v1.3: A Massively Multiagent Game Environment for Training and Evaluating Neural Networks

Progress in multiagent intelligence research is fundamentally limited by...

7 Joseph Suarez, et al. ∙

research

∙ 12/06/2019

Observational Overfitting in Reinforcement Learning

A major component of overfitting in model-free reinforcement learning (R...

21 Xingyou Song, et al. ∙

research

∙ 09/15/2019

Model Based Planning with Energy Based Models

Model-based planning holds great promise for improving both sample effic...

5 Yilun Du, et al. ∙

Yilun Du

Featured Co-authors