Lulela is an existing musical theatre performance created by Gabriele di Franco and produced by LOD in Ghent.
Because the playful character and tree structure of the piece begs to be gamified, Gabriele approached us to work together on a VR version of the performance.
For a relatively small project, the technical requirements were quite ambitious: show a (recorded) live performance to an audience in VR. On top of that, we decided to go for a Meta Quest 2, not exactly a performance powerhouse.
The first challenge was getting the ‘real’ performance in VR. We used Depthkit for this. Using Azure Kinects, artists in front of a green screen are recorded. With the depth info, we placed the 3D versions of the individual artists in the virtual space and synchronized them with the recorded music.
The second challenge was getting lots of people at the same time in the same performance. As an audience, they should not only interact with the environment, but also with each other. At the end of every chapter, all audience members could vote for the next one. That way, we created a truly interactive and connected experience.
Cool stuff? Wait to see the multi-user stuff we did for Koeken Troef!