b'Lijuan Wang'

research

∙ 09/18/2023

Multimodal Foundation Models: From Specialists to General-Purpose Assistants

This paper presents a comprehensive survey of the taxonomy and evolution...

0 Chunyuan Li, et al. ∙

research

∙ 08/26/2023

ORES: Open-vocabulary Responsible Visual Synthesis

Avoiding synthesizing specific visual concepts is an essential challenge...

0 Minheng Ni, et al. ∙

research

∙ 07/27/2023

Spatial-Frequency U-Net for Denoising Diffusion Probabilistic Models

In this paper, we study the denoising diffusion probabilistic model (DDP...

0 Xin Yuan, et al. ∙

research

∙ 06/30/2023

DisCo: Disentangled Control for Referring Human Dance Generation in Real World

Generative AI has made significant strides in computer vision, particula...

0 Tan Wang, et al. ∙

research

∙ 06/26/2023

Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning

Despite the promising progress in multi-modal tasks, current large multi...

0 Fuxiao Liu, et al. ∙

research

∙ 06/07/2023

MultiSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos

Multimodal summarization with multimodal output (MSMO) has emerged as a ...

0 Jielin Qiu, et al. ∙

research

∙ 05/07/2023

Neural Voting Field for Camera-Space 3D Hand Pose Estimation

We present a unified framework for camera-space 3D hand pose estimation ...

0 Lin Huang, et al. ∙

research

∙ 04/28/2023

An Empirical Study of Multimodal Model Merging

Model merging (e.g., via interpolation or task arithmetic) fuses multipl...

3 Yi-Lin Sung, et al. ∙

research

∙ 04/16/2023

The Contour integral method for Feynman-Kac equation with two internal states

We develop the contour integral method for numerically solving the Feynm...

0 Fugui Ma, et al. ∙

research

∙ 04/13/2023

Diagnostic Benchmark and Iterative Inpainting for Layout-Guided Image Generation

Spatial control is a core capability in controllable image generation. A...

4 Jaemin Cho, et al. ∙

research

∙ 04/12/2023

Adaptive Human Matting for Dynamic Videos

The most recent efforts in video matting have focused on eliminating tri...

0 Chung-Ching Lin, et al. ∙

research

∙ 03/25/2023

Equivariant Similarity for Vision-Language Foundation Models

This study explores the concept of equivariance in vision-language found...

0 Tan Wang, et al. ∙

research

∙ 03/20/2023

MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action

We propose MM-REACT, a system paradigm that integrates ChatGPT with a po...

0 Zhengyuan Yang, et al. ∙

research

∙ 02/21/2023

Learning 3D Photography Videos via Self-supervised Diffusion on Single Images

3D photography renders a static image into a video with appealing 3D vis...

0 Xiaodong Wang, et al. ∙

research

∙ 12/21/2022

Generalized Decoding for Pixel, Image, and Language

We present X-Decoder, a generalized decoding model that can predict pixe...

10 Xueyan Zou, et al. ∙

research

∙ 12/01/2022

GRiT: A Generative Region-to-text Transformer for Object Understanding

This paper presents a Generative RegIon-to-Text transformer, GRiT, for o...

0 Jialian Wu, et al. ∙

research

∙ 11/24/2022

MPT: Mesh Pre-Training with Transformers for Human Pose and Mesh Reconstruction

We present Mesh Pre-Training (MPT), a new pre-training framework that le...

0 Kevin Lin, et al. ∙

research

∙ 11/21/2022

Exploring Discrete Diffusion Models for Image Captioning

The image captioning task is typically realized by an auto-regressive me...

0 Zixin Zhu, et al. ∙

research

∙ 10/17/2022

Non-Contrastive Learning Meets Language-Image Pre-Training

Contrastive language-image pre-training (CLIP) serves as a de-facto stan...

1 Jinghao Zhou, et al. ∙

research

∙ 10/17/2022

Vision-Language Pre-training: Basics, Recent Advances, and Future Trends

This paper surveys vision-language pre-training (VLP) methods for multim...

0 Zhe Gan, et al. ∙

research

∙ 10/17/2022

Prompting GPT-3 To Be Reliable

Large language models (LLMs) show impressive abilities via few-shot prom...

0 Chenglei Si, et al. ∙

research

∙ 09/04/2022

An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling

Masked visual modeling (MVM) has been recently proven effective for visu...

8 Tsu-Jui Fu, et al. ∙

research

∙ 07/20/2022

NUWA-Infinity: Autoregressive over Autoregressive Generation for Infinite Visual Synthesis

In this paper, we present NUWA-Infinity, a generative model for infinite...

4 Chenfei Wu, et al. ∙

research

∙ 06/15/2022

Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone

Vision-language (VL) pre-training has recently received considerable att...

13 Zi-Yi Dou, et al. ∙

research

∙ 06/14/2022

LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling

Unified vision-language frameworks have greatly advanced in recent years...

9 Linjie Li, et al. ∙

research

∙ 06/12/2022

GLIPv2: Unifying Localization and Vision-Language Understanding

We present GLIPv2, a grounded VL understanding model, that serves both l...

10 Haotian Zhang, et al. ∙

research

∙ 05/27/2022

GIT: A Generative Image-to-text Transformer for Vision and Language

In this paper, we design and train a Generative Image-to-text Transforme...

14 Jianfeng Wang, et al. ∙

research

∙ 05/03/2022

Cross-modal Representation Learning for Zero-shot Action Recognition

We present a cross-modal Transformer-based framework, which jointly enco...

4 Chung-Ching Lin, et al. ∙

research

∙ 04/20/2022

K-LITE: Learning Transferable Visual Models with External Knowledge

Recent state-of-the-art computer vision systems are trained from natural...

3 Sheng Shen, et al. ∙

research

∙ 03/10/2022

The Overlooked Classifier in Human-Object Interaction Recognition

Human-Object Interaction (HOI) recognition is challenging due to two fac...

0 Ying Jin, et al. ∙

research

∙ 12/13/2021

Decoupling Object Detection from Human-Object Interaction Recognition

We propose DEFR, a DEtection-FRee method to recognize Human-Object Inter...

0 Ying Jin, et al. ∙

research

∙ 12/09/2021

Injecting Semantic Concepts into End-to-End Image Captioning

Tremendous progress has been made in recent years in developing better i...

0 Zhiyuan Fang, et al. ∙

research

∙ 12/08/2021

MLP Architectures for Vision-and-Language Modeling: An Empirical Study

We initiate the first empirical study on the use of MLP architectures fo...

2 Yixin Nie, et al. ∙

research

∙ 12/07/2021

Grounded Language-Image Pre-training

This paper presents a grounded language-image pre-training (GLIP) model ...

4 Liunian Harold Li, et al. ∙

research

∙ 11/25/2021

SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning

The canonical approach to video captioning dictates a caption generation...

29 Kevin Lin, et al. ∙

research

∙ 11/24/2021

VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling

A great challenge in video-language (VidL) modeling lies in the disconne...

19 Tsu-Jui Fu, et al. ∙

research

∙ 11/24/2021

Scaling Up Vision-Language Pre-training for Image Captioning

In recent years, we have witnessed significant performance boost in the ...

0 Xiaowei Hu, et al. ∙

research

∙ 11/23/2021

Crossing the Format Boundary of Text and Boxes: Towards Unified Vision-Language Modeling

In this paper, we propose UNICORN, a vision-language (VL) model that uni...

7 Zhengyuan Yang, et al. ∙

research

∙ 11/22/2021

Florence: A New Foundation Model for Computer Vision

Automated visual understanding of our diverse and open world demands com...

4 Lu Yuan, et al. ∙

research

∙ 11/19/2021

UFO: A UniFied TransfOrmer for Vision-Language Representation Learning

In this paper, we propose a single UniFied transfOrmer (UFO), which is c...

0 Jianfeng Wang, et al. ∙

research

∙ 11/03/2021

An Empirical Study of Training End-to-End Vision-and-Language Transformers

Vision-and-language (VL) pre-training has proven to be highly effective ...

0 Zi-Yi Dou, et al. ∙

research

∙ 09/10/2021

An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA

Knowledge-based visual question answering (VQA) involves answering quest...

0 Zhengyuan Yang, et al. ∙

research

∙ 08/08/2021

OVIS: Open-Vocabulary Visual Instance Search via Visual-Semantic Aligned Representation Learning

We introduce the task of open-vocabulary visual instance search (OVIS). ...

0 Sheng Liu, et al. ∙

research

∙ 07/27/2021

Is Object Detection Necessary for Human-Object Interaction Recognition?

This paper revisits human-object interaction (HOI) recognition at image ...

0 Ying Jin, et al. ∙

research

∙ 06/16/2021

End-to-End Semi-Supervised Object Detection with Soft Teacher

This paper presents an end-to-end semi-supervised object detection appro...

0 Mengde Xu, et al. ∙

research

∙ 06/08/2021

VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation

Most existing video-and-language (VidL) research focuses on a single dat...

3 Linjie Li, et al. ∙

research

∙ 04/05/2021

Compressing Visual-linguistic Model via Knowledge Distillation

Despite exciting progress in pre-training for visual-linguistic (VL) rep...

0 Zhiyuan Fang, et al. ∙

research

∙ 04/01/2021

Mesh Graphormer

We present a graph-convolution-reinforced transformer, named Mesh Grapho...

0 Kevin Lin, et al. ∙

research

∙ 03/30/2021

DAP: Detection-Aware Pre-training with Weak Supervision

This paper presents a detection-aware pre-training (DAP) approach, which...

0 Yuanyi Zhong, et al. ∙

research

∙ 03/22/2021

Adversarial Feature Augmentation and Normalization for Visual Recognition

Recent advances in computer vision take advantage of adversarial data au...

14 Tianlong Chen, et al. ∙

Lijuan Wang

Featured Co-authors