VIDEO. To recreate Notre-Dame de Paris in virtual reality, sound also has its importance
“It has come the time of cathedrals, the world has entered a new millennium”. We know the song : but who could have predicted, in the time of Victor Hugo, the contemporary irruption of virtual reality or binaural sound ? Because immersion passes as much through the eyes as through the ears, both artists and researchers have understood it well. Thus was born Ghost Orchestra, a joint project of the CNRS and the Conservatoire de Paris (CNSMDP). The challenge : to reconstruct the sound characteristics of the cathedral, beyond the geometry of the place alone. The result can be heard in the video above. A feat made possible by an ultra-precise sound capture made on the occasion of a concert given by the Conservatoire de Paris.
In virtual reality, immersion also comes through sound
In order to make recordings of the concert capable of correctly reproducing the sound in space, ultra-precise microphones were used to record 45 simultaneous channels at 360°. Preliminary measurements were also made in order to build a detailed acoustic model for the place (sound clarity, reverb …). What to transform the sound in an interactive (and realistic) way according to a virtual reality path in a digital cathedral !
SOUNDS & AMP; IMAGES.“3D sound is now a major topic of interest for virtual reality, both for researchers and for industrialists,” says Brian Katz, research director at the CNRS and main instigator of the project. In this regard, “multimodal interactions, i.e. the balance between visual and auditory perceptions are paramount in order to impart the feeling of immersion” and to properly represent one’s body in a virtual space.
A new field at the crossroads of scientific disciplines
It is this same principle that is exploited for binaural sound, where the personalization of sound is pushed even further : the shape of the skull (and its acoustic effects in auditory perception) or the shape of the ears are for example taken into account. The objective is to render a sound as close as possible to direct listening. “The spatialization of sound is for me a field at the border of theoretical acoustics, psycho-acoustics, the sciences of perception and cognition and finally signal processing,” enthuses Brian Katz.
Even without binaural headphones, you can in any case get an idea of the initial recording of the concert… and its reconstruction through the acoustic model. This approach has other concrete applications, from 3D to sound… but also reciprocally : for example, it made it possible to reconstruct a concert hall destroyed by fire in Montreux during a Frank Zappa concert, thanks to a recording of a concert that was given there before its destruction, as we explain in our issue 845 of July 2017.