Current browse context:
eess.AS
Change to browse by:
References & Citations
Electrical Engineering and Systems Science > Audio and Speech Processing
Title: EMA2S: An End-to-End Multimodal Articulatory-to-Speech System
(Submitted on 7 Feb 2021 (v1), last revised 9 Jun 2021 (this version, v2))
Abstract: Synthesized speech from articulatory movements can have real-world use for patients with vocal cord disorders, situations requiring silent speech, or in high-noise environments. In this work, we present EMA2S, an end-to-end multimodal articulatory-to-speech system that directly converts articulatory movements to speech signals. We use a neural-network-based vocoder combined with multimodal joint-training, incorporating spectrogram, mel-spectrogram, and deep features. The experimental results confirm that the multimodal approach of EMA2S outperforms the baseline system in terms of both objective evaluation and subjective evaluation metrics. Moreover, results demonstrate that joint mel-spectrogram and deep feature loss training can effectively improve system performance.
Submission history
From: Yu-Wen Chen [view email][v1] Sun, 7 Feb 2021 12:14:14 GMT (1397kb,D)
[v2] Wed, 9 Jun 2021 05:40:18 GMT (1395kb,D)
Link back to: arXiv, form interface, contact.