ClearBuds, the first headphones that cancel noise through deep learning

0
40
clearbuds.jpg
clearbuds.jpg

Inspired by the surprising noises of everyday life that can creep into a video call, three researchers who lived together during the pandemic lockdown set out to create better headphones for these situations.

With the ability to enhance the speaker’s voice and reduce background noise, the first “ClearBuds” were introduced, an example of headphones that uses a novel microphone system, incorporating machine learning in real time, with the support of a telephone. intelligent.

Noise cancellation in headphones, thanks to artificial intelligence

Co-lead author of the research behind the development of ClearBuds, Maruchi Kim, described these earbuds highlighting two key things about them. “First, the ClearBuds use a dual microphone array. Microphones in each earcup create two synchronized audio streams that deliver information and allow us to spatially separate sounds coming from different directions at higher resolution. Second, the lightweight neural network further enhances the speaker’s voice.”detailed the doctoral student at the Allen School of Computer Science and Engineering at the University of Washington, United States.

Although most commercially available headsets now also have microphones built into each earbud, only one earbud actively sends audio to a phone at a time. With ClearBuds, instead, each earbud sends an audio stream to the phone. The researchers designed Bluetooth network protocols to allow these transmissions to be synchronized within 70 microseconds of each other.

The heavy lifting falls to a neural network algorithm, which runs on a mobile to process the audio streams. When processing incoming audio, the system first suppresses any non-voice sounds, then isolates and enhances any frequencies coming from both earphones at the same time – the speaker’s voice, to be precise.

SEE ALSO  Raspberry Pi launches operating system with new features and small corrections

“Because the speaker’s voice is close to and roughly equidistant from the two headphones, the neural network can be trained to focus only on their speech and remove background sounds, including other voices.”said co-senior author Ishan Chatterjee, a doctoral student at the Allen School. “This method is quite similar to how your own ears work. They use the time difference between sounds reaching your left and right ears to determine which direction a sound is coming from.”.

“It’s remarkable when you consider the fact that our neural network has to run in less than 20 milliseconds on an iPhone that has a fraction of the computing power compared to a large commercial graphics card, which is typically used to run neural networks.”said co-author Vivek Jayaram, also a doctoral student at the Allen School. “That’s part of the challenge we had to address in this paper: How do we take a traditional neural network and reduce its size while preserving the quality of the output?”.

The headphones developed by this team were tested, in addition to office or teleworking environments, in public and noisy spaces. After these tests, ClearBuds were the specimens that presented the best performance, compared to other commercial headphones, including the Apple AirPods Pro. However, despite its technical qualities, among which the achievement of a higher signal-to-distortion ratio stands out. In all tests, a fundamental requirement for its efficient operation is the use of both headphones at the same time.

The team behind this project continues to explore new possibilities for exploiting this technology, while at the same time seeking to enhance ClearBuds, updating the algorithm for greater efficiency.