On July 25, United States Pacific time, Google announced the technical details of its Web VR audio system Omnitone project on its official blog. The Omnitone project is a cross-browser supported open source spatial audio renderer.
The problem solved by Google this time is actually solved by other VR devices. It is to create a "panoramic audio" environment.
As shown in the above figure, the position of the sound should be different as the head worn by the user moves. When the user hears the clapping from behind, he turns back and the sound is supposed to come from the front. The VR experience needs to simulate a full range of sound environments around and allow the sound to be automatically positioned in the 3D environment as your head moves. Google uses the "Head-related Transfer Function" (HRTF) here, which is also a technique commonly used by products such as Oculus Rift.
However , the main challenge for the Google Omnitone o team is to introduce Ambisonics in existing browsers without interfering with the browser's original operation. The solution they came up with was smart and straightforward.
The following is an illustration of Omnitone audio processing.
Photo from: Google Open Source Blog
Surround sound contains four channels that can be decoded in any speaker setup. Instead of using physical speakers, Omnitone uses 8 virtual speakers to render the final binaural audio stream based on the HRTF settings. Binaural-rendered audio can create a sense of space with only headphones.
The key point of this system lies in the sound field conversion in the middle (as shown in the Rotator module in the figure), which seamlessly connects the data of the VR head or direction sensor in the earphone with Omnitone's decoder to smoothly perform the entire sound field. Conversion.
Omnitone has been open sourced on Github and has rendered samples. Interested friends can click on Github code and audio samples to reach the presentation page.
When the VR application's pendulum swings back from the dedicated app to the web system, Google is ready.
Picture from: TC
Via TC&Google Open Source Blog
