This paper introduces a novel capability for hearable devices, termed ‘semantic hearing’. Semantic hearing allows the user to focus on or ignore specific sounds in real-world environments, while preserving spatial cues. The authors present a neural network that achieves binaural target sound extraction in the presence of interfering sounds and background noise. They also design a training methodology for the system to generalize to real-world use. The system has a runtime of 6.56 ms on a smartphone and can operate with 20 sound classes.
Publication date: 3 Nov 2023
Project Page: https://semantichearing.cs.washington.edu
Paper: https://arxiv.org/pdf/2311.00320
