Brouhaha: multi-task training for voice activity detection, speech-to-noise ratio, and C50 room acoustics estimation - Signaux et Images Access content directly
Conference Papers Year : 2023

Brouhaha: multi-task training for voice activity detection, speech-to-noise ratio, and C50 room acoustics estimation

Abstract

Most automatic speech processing systems register degraded performance when applied to noisy or reverberant speech. But how can one tell whether speech is noisy or reverberant? We propose Brouhaha, a neural network jointly trained to extract speech/non-speech segments, speech-to-noise ratios, and C50room acoustics from single-channel recordings. Brouhaha is trained using a data-driven approach in which noisy and reverberant audio segments are synthesized. We first evaluate its performance and demonstrate that the proposed multi-task regime is beneficial. We then present two scenarios illustrating how Brouhaha can be used on naturally noisy and reverberant data: 1) to investigate the errors made by a speaker diarization model (pyannote.audio); and 2) to assess the reliability of an automatic speech recognition model (Whisper from OpenAI). Both our pipeline and a pretrained model are open source and shared with the speech community.
Fichier principal
Vignette du fichier
_ASRU__Brouhaha-1.pdf (565.76 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04247647 , version 1 (20-10-2023)

Identifiers

Cite

Marvin Lavechin, Marianne Métais, Hadrien Titeux, Alodie Boissonnet, Jade Copet, et al.. Brouhaha: multi-task training for voice activity detection, speech-to-noise ratio, and C50 room acoustics estimation. IEEE Automatic Speech Recognition and Understanding (ASRU 2023 ), IEEE, Dec 2023, Taipei, Taiwan. pp.1--7. ⟨hal-04247647⟩
112 View
32 Download

Altmetric

Share

Gmail Facebook X LinkedIn More