journal article·2024-07-31Open access

Assessment of Self-Supervised Denoising Methods for Esophageal Speech Enhancement

Madiha Amarjouf, El Hassan Ibn Elhaj, Mouhcine Chami, Kadria Ezzine, Joseph Di Martino

Applied Sciences· Volume 14· Issue 15

Abstract

Esophageal speech (ES) is a pathological voice that is often difficult to understand. Moreover, acquiring recordings of a patient’s voice before a laryngectomy proves challenging, thereby complicating enhancing this kind of voice. That is why most supervised methods used to enhance ES are based on voice conversion, which uses healthy speaker targets, things that may not preserve the speaker’s identity. Otherwise, unsupervised methods for ES are mostly based on traditional filters, which cannot alone beat this kind of noise, making the denoising process difficult. Also, these methods are known for producing musical artifacts. To address these issues, a self-supervised method based on the Only-Noisy-Training (ONT) model was applied, consisting of denoising a signal without needing a clean target. Four experiments were conducted using Deep Complex UNET (DCUNET) and Deep Complex UNET with Complex Two-Stage Transformer Module (DCUNET-cTSTM) for assessment. Both of these models are based on the ONT approach. Also, for comparison purposes and to calculate the evaluation metrics, the pre-trained VoiceFixer model was used to restore the clean wave files of esophageal speech. Even with the fact that ONT-based methods work better with noisy wave files, the results have proven that ES can be denoised without the need for clean targets, and hence, the speaker’s identity is retained.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Speech and Audio Processing · Speech Recognition and Synthesis · Voice and Speech Disorders

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Research Laboratory in Telecommunications Systems: Networks and Services (STRS), Research Team: Multimedia, Signal and Communications Systems (MUSICS), National Institute of Posts and Telecommunications (INPT), Av. Allal Al Fassi, Rabat 10112, Morocco

Research Laboratory in Telecommunications Systems: Networks and Services (STRS), Research Team: Secure and Mixed Architecture for Reliable Technologies and Systems (SMARTS), National Institute of Posts and Telecommunications (INPT), Av. Allal Al Fassi, Rabat 10112, Morocco

LORIA-Laboratoire Lorrain de Recherche en Informatique et ses Applications, B.P. 239, 54506 Vandœuvre-lès-Nancy, France

References (30)

Show source-provided references
1. The Effect of Clear Speech on Cantonese Alaryngeal Speakers’ Intelligibility10.1159/000517676 ↗
2. Raman, S., Sarasola, X., Navas, E., and Hernaez, I. (2021). Enrichment of Oesophageal Speech: Voice Conversion with Duration–Matched Synthetic Speech as Target. Appl. Sci., 11.10.3390/app11135940 ↗
3. Dinh, T., Kain, A., Samlan, R., Cao, B., and Wang, J. (2020, January 25–29). Increasing the Intelligibility and Naturalness of Alaryngeal Speech Using Voice Conversion and Synthetic Fundamental Frequency. Proceedings of the 21st Annual Conference of the International Speech Communication Association, INTERSPEECH, Shanghai, China.10.21437/Interspeech.2020-1196 ↗
4. Predicted Phase Using Deep Neural Networks to Enhance Esophageal Speech
5. A Voice-Producing System with Naturalness and Variable Multi-Frequency Vocalization for Patients Who Have Undergone Laryngectomy10.1109/ACCESS.2023.3262122 ↗
6. Esophageal Speech Enhancement Based on Statistical Voice Conversion with Gaussian Mixture Models10.1587/transinf.E93.D.2472 ↗
7. Yamamoto, K., Toda, T., Doi, H., Saruwatari, H., and Shikano, K. (2012, January 25–30). Statistical Approach to Voice Quality Control in Esophageal Speech Enhancement. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Kyoto, Japan.10.1109/ICASSP.2012.6287949 ↗
8. Caeiros, A.V.M., and Meana, H.M.P. (2012). Esophageal Speech Enhancement Using a Feature Extraction Method Based on Wavelet Transform. Modern Speech Recognition Approaches with Case Studies, IntechOpen.10.5772/49943 ↗
9. A Spanish Multispeaker Database of Esophageal Speech10.1016/j.csl.2020.101168 ↗
10. Acoustic Assessment of Erygmophonic Speech of Moroccan Laryngectomized Patients10.11604/pamj.2015.21.270.4301 ↗
11. Enhancement of Esophageal Speech Using Statistical and Neuromimetic Voice Conversion Techniques
12. Ezzine, K., Di Martino, J., and Frikha, M. (2022). Intelligibility Improvement of Esophageal Speech Using Sequence-To-Sequence Voice Conversion with Auditory Attention. Appl. Sci., 12.10.3390/app12147062 ↗
13. Denoising Esophageal Speech Using Combination of Complex and Discrete Wavelet Transform with Wiener Filter and Time Dilated Fourier Cepstra
14. Enhancement of Esophageal Speech Obtained by a Voice Conversion Technique Using Time Dilated Fourier Cepstra
15. Zhang, M., Wang, X., Fang, F., Li, H., and Yamagishi, J. (2019). Joint Training Framework for Text-To-Speech and Voice Conversion Using Multi-Source Tacotron and WaveNet. arXiv.10.21437/Interspeech.2019-1357 ↗
16. Huang, Z., Watanabe, S., Yang, S., Garcia, P., and Khudanpur, S. (2022, January 23–27). Investigating Self-Supervised Learning for Speech Enhancement and Separation. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9746303 ↗
17. Walczyna, T., and Piotrowski, Z. (2023). Overview of Voice Conversion Methods Based on Deep Learning. Appl. Sci., 13.10.3390/app13053100 ↗
18. Ruiz, I., Garcia, B., Mendez, A., and Villanueva, V. (2007, January 15–18). Oesophageal speech enhancement using Kalman filters. Proceedings of the IEEE International Symposium on Signal Processing and Information Technology, Giza, Egypt.10.1109/ISSPIT.2007.4458117 ↗
19. Research on Speech Enhancement Algorithm of Multiresolution Cochleagram Based on Skip Connection Deep Neural Network10.1155/2022/5208372 ↗
20. Self-Supervised Speech Denoising Using Only Noisy Audio Signals10.1016/j.specom.2023.03.009 ↗
21. Liu, H., Kong, Q., Tian, Q., Zhao, Y., Wang, D., Huang, C., and Wang, Y. (2021). VoiceFixer: Toward General Speech Restoration with Neural Vocoder. arXiv.
22. Trabelsi, C., Bilaniuk, O., Zhang, Y., Serdyuk, D., Subramanian, S., Felipe Santos, J., Santos, F., Mehri, S., Rostamzadeh, N., and Bengio, Y. (May, January 30). Deep Complex Networks. Proceedings of the ICLR 2018, Vancouver, BC, Canada.
23. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, Proceedings of the 18th International Conference, Munich, Germany, 5–9 October 2015, Springer International Publishing.10.1007/978-3-319-24574-4_28 ↗
24. Kashyap, M., Tambwekar, A., Manohara, K., and Natarajan, S. (2021). Speech Denoising Without Clean Training Data: A Noise2Noise Approach. arXiv.10.21437/Interspeech.2021-1130 ↗
25. Glorot, X., and Bengio, Y. (2010, January 13–15). Understanding the Difficulty of Training Deep Feedforward Neural Networks. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Sardinia, Italy. JMLR Workshop and Conference Proceedings.
26. Valentini-Botinhao, C. (2017). Noisy Speech Database for Training Speech Enhancement Algorithms and TTS Models, Centre for Speech Technology Research (CSTR), School of Informatics, University of Edinburgh.
27. Salamon, J., Jacoby, C., and Bello, J.P. (2014, January 3–7). A Dataset and Taxonomy for Urban Sound Research. Proceedings of the 22nd ACM International Conference on Multimedia, Orlando, FL, USA. Available online: https://urbansounddataset.weebly.com/urbansound8k.html.10.1145/2647868.2655045 ↗
28. Rix, A.W., Beerends, J.G., Hollier, M.P., and Hekstra, A.P. (2001, January 7–11). Perceptual Evaluation of Speech Quality (PESQ)—A New Method for Speech Quality Assessment of Telephone Networks and Codecs. Proceedings of the 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221), Salt Lake City, UT, USA.
29. Objective Measures for Predicting Speech Intelligibility in Noisy Conditions Based on New Band-Importance Functions10.1121/1.3097493 ↗
30. An algorithm for intelligibility prediction of time-frequency weighted noisy speech10.1109/TASL.2011.2114881 ↗