research
∙
09/14/2023
Speech-to-Speech Translation with Discrete-Unit-Based Style Transfer
Direct speech-to-speech translation (S2ST) with discrete self-supervised...
share
research
∙
08/28/2023
TextrolSpeech: A Text Style Control Speech Corpus With Codec Language Text-to-Speech Models
Recently, there has been a growing interest in the field of controllable...
research
∙
08/17/2023
Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenes
3D scene understanding has gained significant attention due to its wide ...
research
∙
07/25/2023
3DRP-Net: 3D Relative Position-aware Network for 3D Visual Grounding
3D visual grounding aims to localize the target object in a 3D point clo...
research
∙
07/19/2023
DisCover: Disentangled Music Representation Learning for Cover Song Identification
In the field of music information retrieval (MIR), cover song identifica...
research
∙
07/18/2023
Distilling Coarse-to-Fine Semantic Matching Knowledge for Weakly Supervised 3D Visual Grounding
3D visual grounding involves finding a target object in a 3D scene that ...
research
∙
07/14/2023
Gloss Attention for Gloss-free Sign Language Translation
Most sign language translation (SLT) methods to date require the use of ...
research
∙
07/14/2023
Mega-TTS 2: Zero-Shot Text-to-Speech with Arbitrary Length Speech Prompts
Zero-shot text-to-speech aims at synthesizing voices with unseen speech ...
research
∙
06/12/2023
MSSRNet: Manipulating Sequential Style Representation for Unsupervised Text Style Transfer
Unsupervised text style transfer task aims to rewrite a text into target...
research
∙
06/10/2023
OpenSR: Open-Modality Speech Recognition via Maintaining Multi-Modality Alignment
Speech Recognition builds a bridge between the multimedia streaming (aud...
research
∙
06/06/2023
Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias
Scaling text-to-speech to a large and wild dataset has been proven to be...
research
∙
06/06/2023
Ada-TTA: Towards Adaptive High-Quality Text-to-Talking Avatar Synthesis
We are interested in a novel task, namely low-resource text-to-talking a...
research
∙
06/04/2023
Detector Guidance for Multi-Object Text-to-Image Generation
Diffusion models have demonstrated impressive performance in text-to-ima...
research
∙
05/30/2023
Make-A-Voice: Unified Voice Synthesis With Discrete Representation
Various applications of voice synthesis have been developed independentl...
research
∙
05/29/2023
Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation
Large diffusion models have been successful in text-to-audio (T2A) synth...
research
∙
05/24/2023
AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation
Direct speech-to-speech translation (S2ST) aims to convert speech from o...
research
∙
05/23/2023
FluentSpeech: Stutter-Oriented Automatic Speech Editing with Context-Aware Diffusion Models
Stutter removal is an essential scenario in the field of speech editing....
research
∙
05/22/2023
Connecting Multi-modal Contrastive Representations
Multi-modal Contrastive Representation (MCR) learning aims to encode dif...
research
∙
05/22/2023
ViT-TTS: Visual Text-to-Speech with Scalable Diffusion Transformer
Text-to-speech(TTS) has undergone remarkable improvements in performance...
research
∙
05/21/2023
Wav2SQL: Direct Generalizable Speech-To-SQL Parsing
Speech-to-SQL (S2SQL) aims to convert spoken questions into SQL queries ...
research
∙
05/18/2023
CLAPSpeech: Learning Prosody from Text Context with Contrastive Language-Audio Pre-training
Improving text representation has attracted much attention to achieve ex...
research
∙
05/18/2023
RMSSinger: Realistic-Music-Score based Singing Voice Synthesis
We are interested in a challenging task, Realistic-Music-Score based Sin...
research
∙
05/08/2023
AlignSTS: Speech-to-Singing Conversion via Cross-Modal Alignment
The speech-to-singing (STS) voice conversion task aims to generate singi...
research
∙
05/04/2023
ANetQA: A Large-scale Benchmark for Fine-grained Compositional Reasoning over Untrimmed Videos
Building benchmarks to systemically analyze different capabilities of vi...
research
∙
05/01/2023
GeneFace++: Generalized and Stable Real-Time Audio-Driven 3D Talking Face Generation
Generating talking person portraits with arbitrary speech audio is a cru...
research
∙
04/25/2023
AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head
Large language models (LLMs) have exhibited remarkable capabilities acro...
research
∙
04/13/2023
Set-Based Face Recognition Beyond Disentanglement: Burstiness Suppression With Variance Vocabulary
Set-based face recognition (SFR) aims to recognize the face sets in the ...
research
∙
04/07/2023
DATE: Domain Adaptive Product Seeker for E-commerce
Product Retrieval (PR) and Grounding (PG), aiming to seek image and obje...
research
∙
03/24/2023
MUG: A General Meeting Understanding and Generation Benchmark
Listening to long video/audio recordings from video conferencing and onl...
research
∙
03/24/2023
Overview of the ICASSP 2023 General Meeting Understanding and Generation Challenge (MUG)
ICASSP2023 General Meeting Understanding and Generation Challenge (MUG) ...
research
∙
03/09/2023
MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition
Multi-media communications facilitate global interaction among people. H...
research
∙
02/05/2023
ShiftDDPMs: Exploring Conditional Diffusion Models by Shifting Diffusion Trajectories
Diffusion models have recently exhibited remarkable abilities to synthes...
research
∙
01/31/2023
GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis
Generating photo-realistic video portrait with arbitrary speech audio is...
research
∙
01/30/2023
Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models
Large-scale multimodal generative modeling has created milestones in tex...
research
∙
12/26/2022
Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Models
Diffusion Probabilistic Models (DPMs) have shown a powerful capacity of ...
research
∙
11/21/2022
Diffusion Denoising Process for Perceptron Bias in Out-of-distribution Detection
Out-of-distribution (OOD) detection is an important task to ensure the r...
research
∙
11/19/2022
VarietySound: Timbre-Controllable Video to Sound Generation via Unsupervised Information Disentanglement
Video to sound generation aims to generate realistic and natural sound g...
research
∙
09/08/2022
Frame-Subtitle Self-Supervision for Multi-Modal Video Question Answering
Multi-modal video question answering aims to predict correct answer and ...
research
∙
09/01/2022
Video-Guided Curriculum Learning for Spoken Video Grounding
In this paper, we introduce a new task, spoken video grounding (SVG), wh...
research
∙
08/20/2022
AntCritic: Argument Mining for Free-Form and Visually-Rich Financial Comments
The task of argument mining aims to detect all possible argumentative co...
research
∙
08/17/2022
CCL4Rec: Contrast over Contrastive Learning for Micro-video Recommendation
Micro-video recommender systems suffer from the ubiquitous noises in use...
research
∙
08/17/2022
Re4: Learning to Re-contrast, Re-attend, Re-construct for Multi-interest Recommendation
Effectively representing users lie at the core of modern recommender sys...
research
∙
08/11/2022
HERO: HiErarchical spatio-tempoRal reasOning with Contrastive Action Correspondence for End-to-End Video Object Grounding
Video Object Grounding (VOG) is the problem of associating spatial objec...
research
∙
07/13/2022
ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech
Denoising diffusion probabilistic models (DDPMs) have recently achieved ...
research
∙
07/08/2022
FastLTS: Non-Autoregressive End-to-End Unconstrained Lip-to-Speech Synthesis
Unconstrained lip-to-speech synthesis aims to generate corresponding spe...
research
∙
06/10/2022
AntPivot: Livestream Highlight Detection via Hierarchical Attention Mechanism
In recent days, streaming technology has greatly promoted the developmen...
research
∙
06/05/2022
Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech
Polyphone disambiguation aims to capture accurate pronunciation knowledg...
research
∙
05/25/2022
TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation
Direct speech-to-speech translation (S2ST) systems leverage recent progr...
research
∙
05/15/2022
GenerSpeech: Towards Style Transfer for Generalizable Out-Of-Domain Text-to-Speech Synthesis
Style transfer for out-of-domain (OOD) speech synthesis aims to generate...
research
∙
04/25/2022