Researchers train machines to recognize vocal fatigue


Credit: University of Missouri

Even before COVID-19 made them speak in online classes or project their voices behind masks, teachers were at high risk of vocal fatigue. This condition can cause persistent hoarseness, sore throat, and permanent damage to the vocal cords. Currently, the diagnosis of vocal fatigue requires an in-person consultation. But someday, a wearable device or smart app could detect vocal fatigue early and help sufferers prevent further problems.

Before that happens, however, a machine must learn to recognize the difference between a healthy voice and a tired voice. This is where Gui DeSouza comes in. DeSouza, an associate professor of electrical engineering and computer science, and a German collaborator have spent years training a computer to detect voice problems by providing the system with hundreds of samples of student teachers and control groups.

“Student teachers are much more affected by vocal fatigue than other professionals,” DeSouza said. “We are approaching the diagnostic side because early detection can alert a person to change their habits or take corrective action.”

With funding from the National Institutes of Health, the research team collected 160 voice samples from 90 participants. The team uses surface electromyography (EMG) sensors placed on the neck to detect vibrations. A participant is asked to pronounce certain vowels and consonants which tend to indicate problems in the vocal cords.

Researchers then use this data to train the system to detect changes that indicate vocal fatigue.

Find a reliable system

Originally, the team tested the model using simulated samples and the results were promising. However, in a more recent study, the team intentionally left out samples of a human participant’s voice and saw a drop in accuracy.

“If you look at the literature, no one has done this before,” DeSouza said, referring to the “leave one out” method. “This indicates that the machine is good at learning people’s voices but not necessarily at learning to recognize fatigue.”

Another complication is that there is no consistent standard for classifying fatigue. Currently, doctors use patient surveys to collect this information. However, a person can have a high tolerance for discomfort and report a low ranking. Someone else who is more sensitive might rank higher for essentially the same level of pain.

“A big problem for us is how to make sense of the data when it’s very subjective,” DeSouza said. “The dataset is not reliably labeled. Ultimately, we want to have a system that reliably indicates that it is fatigue or that it is not fatigue independently. subjective measurement or self-assessment. “

The research team presented their findings in the journal of Applied Science at the start of this year. Most recently, they presented their test results at the 14th International Conference on Advances in Quantitative Research in Laryngology, Voice and Speech.

They also have additional funding from the National Institutes of Health to see if stress induces vocal problems.

“We are now finalizing our original study and also looking at the MRI data to see if brain activities correlate with phenomena that occur in the voice,” DeSouza said. “The idea is to put the patient through some sort of stressor to see if it shows up in the voice. A lot of voice fatigue could be related to emotional stress.”

Hoarsely? There are many reasons to grate, say experts

More information:
Yixiang Gao et al, Classification of Vocal Fatigue Using sEMG: Data Disbalance, Normalization, and the Role of Vocal Fatigue Index Scores, Applied Science (2021). DOI: 10.3390 / app11104335

Provided by the University of Missouri

Quote: Researchers training machines to recognize vocal fatigue (2021, October 22) retrieved October 22, 2021 from

This document is subject to copyright. Other than fair use for private study or research purposes, no part may be reproduced without written permission. The content is provided for information only.

Leave A Reply

Your email address will not be published.