A Survey on Audio Synthesis and Audio-Visual Multimodal Processing

by   Zhaofeng Shi, et al.
University of Electronic Science and Technology of China

With the development of deep learning and artificial intelligence, audio synthesis has a pivotal role in the area of machine learning and shows strong applicability in the industry. Meanwhile, significant efforts have been dedicated by researchers to handle multimodal tasks at present such as audio-visual multimodal processing. In this paper, we conduct a survey on audio synthesis and audio-visual multimodal processing, which helps understand current research and future trends. This review focuses on text to speech(TTS), music generation and some tasks that combine visual and acoustic information. The corresponding technical methods are comprehensively classified and introduced, and their future development trends are prospected. This survey can provide some guidance for researchers who are interested in the areas like audio synthesis and audio-visual multimodal processing.


page 1

page 2

page 3

page 4


A Survey on Neural Speech Synthesis

Text to speech (TTS), or speech synthesis, which aims to synthesize inte...

Deep Audio-Visual Learning: A Survey

Audio-visual learning, aimed at exploiting the relationship between audi...

An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation

Speech enhancement and speech separation are two related tasks, whose pu...

Learning in Audio-visual Context: A Review, Analysis, and New Perspective

Sight and hearing are two senses that play a vital role in human communi...

Retrieving Multimodal Information for Augmented Generation: A Survey

In this survey, we review methods that retrieve multimodal knowledge to ...

A Demand-Driven Perspective on Generative Audio AI

To achieve successful deployment of AI research, it is crucial to unders...

Deploying Enhanced Speech Feature Decreased Audio Complaints at SVT Play VOD Service

At Public Service Broadcaster SVT in Sweden, background music and sounds...

Please sign up or login with your details

Forgot password? Click here to reset