Hybrid Audio-Visual Conferencing
Current audio-visual conferencing systems are mostly designed for online-only meetings. In this use case, all participants are located in different places and each participant has their own device. However, the future will see an increasing demand for hybrid meetings, in which some participants meet physically in a room and further participants join remotely.
The experience for the remote participants is still unnatrual and tiring compared to being physically present. Binaural audio conferencing aims to improve their experience by providing them with more natural audio signals.
Binaural Audio Communication
By recording the physical meeting with microphones placed on or near the ears of a human or artificial head, the natural cues used by the human auditory system are captured and transmitted to the remote participants. This allows remote participants to differentiate individual speakers from each other and from the background sounds more easily.
The binaural audio conferecing system no longer feels like a telephone. It feels like actually being there.
The vision of binaural audio conferencing relies on the use of binaural recording and reproduction devices. It also requires adjustments of the audio processing and transmission chain, in order to ensure that the binaural cues contained in the signals are not distorted or destroyed, e.g., by speech enhancement, echo compensation, listening enhancement, gain control, jitter buffers or audio codecs.
In this video, the impact of binaural audio for conferencing will be demonstrated by switching between narrowband telephone quality and binaural audio.
Note: Use headphones when watching the video! The sound track starts after the first 30 seconds of the video.
TODO: REPLACE THE VIDEO WITH VIDEO FROM CENTRE CHARLEMAGNE EXPOSITION?
Tobias Kabzinski, and Peter Jax
An Adaptive Crosstalk Cancellation System Using Microphones at the Ears
Audio Engineering Society Convention, Oktober 2019
Sebastian Nagel, and Peter Jax
Dynamic Binaural Cue Adaptation
Proceedings of International Workshop on Acoustic Signal Enhancement (IWAENC), September 2018
Matthias Rüngeler, Hauke Krüger, Gottfried Behler, and Peter Vary
HD-Voice-3D: Herausforderungen und Lösungen bei der Audiosignalverarbeitung
Workshop Audiosignal- und Sprachverarbeitung (WASP), September 2013
Matthias Rüngeler, Hauke Krüger, Thomas Schlien, and Peter Vary
Spatial Audio Conferencing using Binaural HD Voice
International Workshop on Acoustic Signal Enhancement (IWAENC), September 2012
Bernd Geiser, Magnus Schäfer, and Peter Vary
Binaural Wideband Telephony Using Steganography
Konferenz Elektronische Sprachsignalverarbeitung (ESSV), September 2011