Binaural Telephony

Hybrid Audio-Visual Conferencing

Current audio-visual conferencing systems are mostly designed for online-only meetings. In this use case, all participants are located in different places and each participant has their own device. However, the future will see an increasing demand for hybrid meetings, in which some participants meet physically in a room and further participants join remotely.

The experience for the remote participants is still unnatrual and tiring compared to being physically present. Binaural audio conferencing aims to improve their experience by providing them with more natural audio signals.

Binaural Audio Communication

By recording the physical meeting with microphones placed on or near the ears of a human or artificial head, the natural cues used by the human auditory system are captured and transmitted to the remote participants. This allows remote participants to differentiate individual speakers from each other and from the background sounds more easily.

The binaural audio conferecing system no longer feels like a telephone. It feels like actually being there.

Technical Background

The vision of binaural audio conferencing relies on the use of binaural recording and reproduction devices. It also requires adjustments of the audio processing and transmission chain, in order to ensure that the binaural cues contained in the signals are not distorted or destroyed, e.g., by speech enhancementecho compensation, listening enhancement, gain control, jitter buffers or audio codecs.

The user experience for the remote participants can be further improved with advanced technologies such as interactive binaural reproduction or adaptive loudspeaker-based binaural reproduction.

Demonstration

In this video, the impact of binaural audio for conferencing will be demonstrated by switching between narrowband telephone quality and binaural audio.

Note: Use headphones when watching the video! The sound track starts after the first 30 seconds of the video.

TODO: REPLACE THE VIDEO WITH VIDEO FROM CENTRE CHARLEMAGNE EXPOSITION?

References

[Kabzinski19a]
Tobias Kabzinski, and Peter Jax
An Adaptive Crosstalk Cancellation System Using Microphones at the Ears
Audio Engineering Society Convention, Oktober 2019

[Nagel18a]
Sebastian Nagel, and Peter Jax
Dynamic Binaural Cue Adaptation
Proceedings of International Workshop on Acoustic Signal Enhancement (IWAENC), September 2018

[Ruengeler13c]
Matthias Rüngeler, Hauke Krüger, Gottfried Behler, and Peter Vary
HD-Voice-3D: Herausforderungen und Lösungen bei der Audiosignalverarbeitung
Workshop Audiosignal- und Sprachverarbeitung (WASP), September 2013

[Ruengeler12]
Matthias Rüngeler, Hauke Krüger, Thomas Schlien, and Peter Vary
Spatial Audio Conferencing using Binaural HD Voice
International Workshop on Acoustic Signal Enhancement (IWAENC), September 2012

[Geiser11]
Bernd Geiser, Magnus Schäfer, and Peter Vary
Binaural Wideband Telephony Using Steganography
Konferenz Elektronische Sprachsignalverarbeitung (ESSV), September 2011