University of Wisconsin–Madison: Videoconferencing apps may listen even when mic is off

April 18, 2022

Kassem Fawaz’s brother was on a videoconference with the microphone muted when he noticed that the microphone light was still on — indicating, inexplicably, that his microphone was being accessed.

Alarmed, he asked Fawaz, an expert in online privacy and an assistant professor of electrical and computer engineering at the University of Wisconsin–Madison, to look into the issue.

Fawaz and graduate student Yucheng Yang investigated whether this “mic-off-light-on” phenomenon was more widespread. They tried out many different videoconferencing applications on major operating systems, including iOS, Android, Windows and Mac, checking to see if the apps still accessed the microphone when it was muted.

“It turns out, in the vast majority of cases, when you mute yourself, these apps do not give up access to the microphone,” says Fawaz. “And that’s a problem. When you’re muted, people don’t expect these apps to collect data.”

After their initial testing, Fawaz and Yang, along with colleagues from Loyola University Chicago, conducted a more formal investigation of just what happens when videoconferencing software microphones are muted. They will present their results at the Privacy Enhancing Technologies Symposium in July.

First, the team conducted a user study, asking 223 videoconferencing app users how they understand the function of the mute button and how they think it should handle audio data. While the participants were split about whether they thought the chat applications were accessing their microphones when muted, most believed that the apps should not be able to collect data while set to mute.

For the second part of the study, the team investigated the actual behavior of the mute button on many popular apps, determining what type of data is collected and whether it could reveal personal information.

They used runtime binary analysis tools to trace raw audio in popular videoconferencing applications as the audio traveled from the app to the computer audio driver and then to the network while the app was muted.

They found that all of the apps they tested occasionally gather raw audio data while mute is activated, with one popular app gathering information and delivering data to its server at the same rate regardless of whether the microphone is muted or not.

The researchers then decided to see if they could use data collected on mute from that app to infer the types of activities taking place in the background. Using machine learning algorithms, they trained an activity classifier using audio from YouTube videos representing six common background activities, including cooking and eating, playing music, typing and cleaning. Applying the classifier to the type of telemetry packets the app was sending, the team could identify the background activity with an average of 82% accuracy.

“When you’re cooking, the acoustic signature is different from someone who is driving or watching a video,” says Fawaz. “So these types of activities can be distinguished just based on this acoustic fingerprint that was actually sent out to the cloud.”

Whether or not the data is being accessed or used, the findings raise privacy concerns.

“With a camera, you can turn it off or even put your hand over it, and no matter what you do, no one can see you,” says Fawaz. “I don’t think that exists for microphones.”

Turning off a microphone is possible in most device operating systems, but it usually means navigating through several menus. Instead, the team suggests the solution might lie in developing easily accessible software “switches” or even hardware switches that allow users to manually enable and disable their microphones.

Other authors include George K. Thiruvathukal and Neil Klingensmith of Loyola University Chicago as well as Loyola graduate student Jack West, who will join Fawaz’s lab the fall.