At the beginning of 4DR Studios (October 2019), our HOLOSYS-equipment only recorded the video and no sound, which meant we had to record the sound separately. But we had an update!
The power of audio synchronisation
The current HOLOSYS-system records raw audio (including noise and background noise), we have no control over that, that's how the HOLOSYS-system works. To keep the control, we record the audio also in our own audio system. So, when we are recording in the studio we edit the audio ourselves right away so that the background noises and other noises are being filtered out, and the voices of the recorded people sound good and clear. Through experimentation we found out that our own audio system offers more possibilities and a higher quality of the audio, that's why we use the audio from our own audio system to polish it!
During recording, the volumetric sound engineer determines exactly where the audio should come from, because the audio in the volumetric video studio does not have a single focus point, but 32 focus (!) points. The position of the viewer in the final product determines where the audio comes from. So, it is important that the audio from every angle in the studio has the same quality. In addition, for each recorded scene we record multiple takes, and so more audio files are generated. When it is made known which scene comes in the final product, the corresponding audio of that specific scene is polished in a sound studio and then synchronized with the volumetric video.
The heaviness of volumetric video
A volumetric video can be quite heavy on mobile devices, a 3-minute music video can easily consume multiple gigabytes if you are not careful. Adding too many assets can also cause the volumetric video to act as a buffer, creating a-synchronization between the audio and video. Your internet speed is of course a crucial factor in the playback of the video, but we cannot determine that speed. However, the volumetric capture operator can make the content as small as possible: determine the length and size of the video, making it less heavy for the mobile device and making it easier to play the video. In contrast to the volumetric recording, you can reduce the volume of the audio because you can compress it more easily.
In order to make the volumetric video recording and the audio synchronous we prepare a clicktrack. This clicktrack can be heard by the people in the green studio. The clicktrack has 2 important functions:
1: When the clicktrack is running, the people in the green studio for example hear 1 or 2 clicks, which means that they can perform timed actions at these specific moments.
2: With this clicktrack we can synchronize the audio and video together.
With practical tools, creative and logical thinking you will of course go a long way!