research
∙
07/31/2023
DAVIS: High-Quality Audio-Visual Separation with Generative Diffusion Models
We propose DAVIS, a Diffusion model-based Audio-VIusal Separation framew...
share
research
∙
07/27/2023
PEANUT: A Human-AI Collaborative Tool for Annotating Audio-Visual Data
Audio-visual learning seeks to enhance the computer's multi-modal percep...
research
∙
05/15/2023
Training Neural Networks without Backpropagation: A Deeper Dive into the Likelihood Ratio Method
Backpropagation (BP) is the most important gradient estimation method fo...
research
∙
03/23/2023
Egocentric Audio-Visual Object Localization
Humans naturally perceive surrounding scenes by unifying sound and sight...
research
∙
02/04/2023
AV-NeRF: Learning Neural Fields for Real-World Audio-Visual Scene Synthesis
Human perception of the complex world relies on a comprehensive analysis...
research
∙
01/30/2023
Improving Adversarial Transferability with Scheduled Step Size and Dual Example
Deep neural networks are widely known to be vulnerable to adversarial ex...
research
∙
12/09/2022
A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Others
Machine learning models have been found to learn shortcuts – unintended ...
research
∙
07/20/2022
Discover and Mitigate Unknown Biases with Debiasing Alternate Networks
Deep image classifiers have been found to learn biases from datasets. To...
research
∙
03/29/2022
StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesis
Although progress has been made for text-to-image synthesis, previous me...
research
∙
03/26/2022
Learning to Answer Questions in Dynamic Audio-Visual Scenarios
In this paper, we focus on the Audio-Visual Question Answering (AVQA) ta...
research
∙
03/26/2022
Transformer-empowered Multi-scale Contextual Matching and Aggregation for Multi-contrast MRI Super-resolution
Magnetic resonance imaging (MRI) can present multi-contrast images of th...
research
∙
01/18/2022
Cross-modal Contrastive Distillation for Instructional Activity Anticipation
In this study, we aim to predict the plausible future action steps given...
research
∙
12/12/2021
Anomaly Crossing: A New Method for Video Anomaly Detection as Cross-domain Few-shot Learning
Video anomaly detection aims to identify abnormal events that occurred i...
research
∙
11/30/2021
SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Editing
Recently, large pretrained models (e.g., BERT, StyleGAN, CLIP) have show...
research
∙
11/10/2021
Space-Time Memory Network for Sounding Object Localization in Videos
Leveraging temporal synchronization and association within sight and sou...
research
∙
10/05/2021
Procedure Planning in Instructional Videosvia Contextual Modeling and Model-based Policy Learning
Learning new skills by observing humans' behaviors is an essential capab...
research
∙
09/06/2021
Learning to Generate Scene Graph from Natural Language Supervision
Learning from image-text data has demonstrated recent success for many r...
research
∙
06/24/2021
Learning by Planning: Language-Guided Global Image Editing
Recently, language-guided global image editing draws increasing attentio...
research
∙
04/29/2021
Discover the Unknown Biased Attribute of an Image Classifier
Recent works find that AI algorithms learn biases from data. Therefore, ...
research
∙
04/15/2021
Zooming SlowMo: An Efficient One-Stage Framework for Space-Time Video Super-Resolution
In this paper, we address the space-time video super-resolution, which a...
research
∙
04/05/2021
Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separation
There are rich synchronized audio and visual events in our daily life. I...
research
∙
04/05/2021
Can audio-visual integration strengthen robustness under multimodal attacks?
In this paper, we propose to make a systematic study on machines multise...
research
∙
03/29/2021
High-fidelity Face Tracking for AR/VR via Deep Lighting Adaptation
3D video avatars can empower virtual communications by providing compres...
research
∙
10/05/2020
A Benchmark and Baseline for Language-Driven Image Editing
Language-driven image editing can significantly save the laborious image...
research
∙
10/03/2020
Cubic Spline Smoothing Compensation for Irregularly Sampled Sequences
The marriage of recurrent neural networks and neural ordinary differenti...
research
∙
07/21/2020
Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing
In this paper, we introduce a new problem, named audio-visual video pars...
research
∙
07/16/2020
Talking-head Generation with Rhythmic Head Motion
When people deliver a speech, they naturally move heads, and this rhythm...
research
∙
06/12/2020
Explaining Local, Global, And Higher-Order Interactions In Deep Learning
We present a simple yet highly generalizable method for explaining inter...
research
∙
05/07/2020
What comprises a good talking-head video generation?: A Survey and Benchmark
Over the years, performance evaluation has become essential in computer ...
research
∙
03/29/2020
Learning a Weakly-Supervised Video Actor-Action Segmentation Model with a Wise Selection
We address weakly-supervised video actor-action segmentation (VAAS), whi...
research
∙
03/25/2020
Deep Grouping Model for Unified Perceptual Parsing
The perceptual-based grouping process produces a hierarchical and compos...
research
∙
02/26/2020
Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution
In this paper, we explore the space-time video super-resolution task, wh...
research
∙
02/23/2020
Assembling Semantically-Disentangled Representations for Predictive-Generative Models via Adaptation from Synthetic Domain
Deep neural networks can form high-level hierarchical representations of...
research
∙
12/21/2019
Deep Audio Prior
Deep convolutional neural networks are known to specialize in distilling...
research
∙
12/04/2019
Learning from Interventions using Hierarchical Policies for Safe Learning
Learning from Demonstrations (LfD) via Behavior Cloning (BC) works well ...
research
∙
11/17/2019
Weakly Supervised Object Localization with Inter-Intra Regulated CAMs
Weakly supervised object localization (WSOL) aims to locate objects in i...
research
∙
09/30/2019
Unsupervised Pose Flow Learning for Pose Guided Synthesis
Pose guided synthesis aims to generate a new image in an arbitrary targe...
research
∙
05/09/2019
Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Loss
We devise a cascade GAN approach to generate talking face video, which i...
research
∙
12/13/2018
Dynamic Graph Modules for Modeling Higher-Order Interactions in Activity Recognition
Video action recognition, as a critical problem towards video understand...
research
∙
12/07/2018
TDAN: Temporally Deformable Alignment Network for Video Super-Resolution
Video super-resolution (VSR) aims to restore a photo-realistic high-reso...
research
∙
12/07/2018
An Attempt towards Interpretable Audio-Visual Video Captioning
Automatically generating a natural language sentence to describe the con...
research
∙
12/02/2018
How to Make a BLT Sandwich? Learning to Reason towards Understanding Web Instructional Videos
Understanding web instructional videos is an essential branch of video u...
research
∙
12/02/2018
GAN-EM: GAN based EM learning framework
Expectation maximization (EM) algorithm is to find maximum likelihood so...
research
∙
11/01/2018
Navigation by Imitation in a Pedestrian-Rich Environment
Deep neural networks trained on demonstrations of human actions give rob...
research
∙
03/28/2018
Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment
In this work, we address the task of weakly-supervised human action segm...
research
∙
03/28/2018
Lip Movements Generation at a Glance
Cross-modality generation is an emerging topic that aims to synthesize d...
research
∙
03/26/2018
Generating Talking Face Landmarks from Speech
The presence of a corresponding talking face has been shown to significa...
research
∙
03/23/2018
Audio-Visual Event Localization in Unconstrained Videos
In this paper, we introduce a novel problem of audio-visual event locali...
research
∙
05/22/2017
TricorNet: A Hybrid Temporal Convolutional and Recurrent Network for Video Action Segmentation
Action segmentation as a milestone towards building automatic systems to...
research
∙
04/26/2017