#1494: Innovative VR Opera “From Dust” Combines GenAI Prompts, Digital Serenading, & Personality Branching

From Dust is a 24-minute VR Opera by composer Michel van der Aa that is premiering this week at the Rotterdam Immersive Tech Week, and I had a chance to get a sneak peak at the end of my trip covering IDFA DocLab in Amsterdam. The emotional core of From Dust is the top-notch composition from van der Aa and brilliant musical performance by Sjaella. It was a really powerful and moving immersive journey that seamlessly integrated my GenAI prompts creating a personalized experience. I can highly recommend checking it out to see where immersive music experiences are headed here in the future. The throughput of this piece is only 3-4 people per hour, and so it may be hard to be able to see it once they start touring it around Europe to different musical festivals. van der Aa told me that this is the lease commercial project that he's had a chance to ever work on, and acknowledges that this is a piece of subsidized art. There have been a lot of broader discussions happening within the XR industry about the sustainability of these types of location-based experiences, and the need for creators to understand some of the more pragmatic financial constraints for exhibition taking into consideration earlier in the production process. I'm in the process of finishing up the public report from the Think Tank at Venice Immersive this year on this very issue, and the MIT Open Documentary Lab is also actively starting to study these types of distribution questions as well at the R&D Summit at IDFA DocLab this year. But at the same time, it's also great to see artists who are able to get this type of work funded who are willing to push the virtual reality medium to the limits of creative expression despite some of the financial impracticalities of exhibition. Especially as it may drive the adaptation of this type of work into a format like the Apple Vision Pro or PCVR where it can be within a form factor that is a lot more scalable, even if there are compromises on fidelity or on the GenAI elements using the open source LLM of Flux that can not be distributed Steam due to Valve's restrictions on AI integrations within games. https://www.youtube.com/watch?v=ig9VqdLW9X0 I had a chance to catch up with van der Aa last week in order to get more context on his journey and process on creating the piece, as well as get some elaboration on the degree on some of the branching mechanics. There are some questions that they ask during the onboarding process that categorize the user's Big Five Personality characteristics, and then they are creating some invisible branching of my experience that was completely imperceptible to me. van der Aa told me that about 75% of the musical experience is the same for everyone, but that there are some explicit and implicit branching that is happening musically, but also a bit more visually as they are integrating my GenAI prompts in 4-5 different places throughout the experience. It's certainly an innovative and ambitious piece that has some GenAI parallels to Tulpamancer from Venice 2023, but one that shows the potential of combining musical composition with these types of immersive adventures that are able to tell a much richer story when combined. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality