FAIR4Cov: Fused Audio Instance and Representation for COVID-19 Detection

by   Tuan Truong, et al.

Audio-based classification techniques on body sounds have long been studied to support diagnostic decisions, particularly in pulmonary diseases. In response to the urgency of the COVID-19 pandemic, a growing number of models are developed to identify COVID-19 patients based on acoustic input. Most models focus on cough because the dry cough is the best-known symptom of COVID-19. However, other body sounds, such as breath and speech, have also been revealed to correlate with COVID-19 as well. In this work, rather than relying on a specific body sound, we propose Fused Audio Instance and Representation for COVID-19 Detection (FAIR4Cov). It relies on constructing a joint feature vector obtained from a plurality of body sounds in waveform and spectrogram representation. The core component of FAIR4Cov is a self-attention fusion unit that is trained to establish the relation of multiple body sounds and audio representations and integrate it into a compact feature vector. We set up our experiments on different combinations of body sounds using only waveform, spectrogram, and a joint representation of waveform and spectrogram. Our findings show that the use of self-attention to combine extracted features from cough, breath, and speech sounds leads to the best performance with an Area Under the Receiver Operating Characteristic Curve (AUC) score of 0.8658, a sensitivity of 0.8057, and a specificity of 0.7958. This AUC is 0.0227 higher than the one of the models trained on spectrograms only and 0.0847 higher than the one of the models trained on waveforms only. The results demonstrate that the combination of spectrogram with waveform representation helps to enrich the extracted features and outperforms the models with single representation.


page 1

page 2

page 3

page 4


The EIHW-GLAM Deep Attentive Multi-model Fusion System for Cough-based COVID-19 Recognition in the DiCOVA 2021 Challenge

Aiming to automatically detect COVID-19 from cough sounds, we propose a ...

Using Self-Supervised Feature Extractors with Attention for Automatic COVID-19 Detection from Speech

The ComParE 2021 COVID-19 Speech Sub-challenge provides a test-bed for t...

COVID-19 Detection from Respiratory Sounds with Hierarchical Spectrogram Transformers

Monitoring of prevalent airborne diseases such as COVID-19 characteristi...

Interpretable Acoustic Representation Learning on Breathing and Speech Signals for COVID-19 Detection

In this paper, we describe an approach for representation learning of au...

Speech Denoising in the Waveform Domain with Self-Attention

In this work, we present CleanUNet, a causal speech denoising model on t...

EIHW-MTG DiCOVA 2021 Challenge System Report

This paper aims to automatically detect COVID-19 patients by analysing t...

Multi-modal Point-of-Care Diagnostics for COVID-19 Based On Acoustics and Symptoms

The research direction of identifying acoustic bio-markers of respirator...

Please sign up or login with your details

Forgot password? Click here to reset