Multi-platinum recording artist Sebastián Yatra performed a show-stopping rendition of his hit song Tacones Rojos during a virtual concert powered by Evercoast volumetric capture technology, Verizon’s 5G Ultra Wideband network, and mobile edge computing. Produced by Verizon, Pandora, and the industry-leading virtual production and XR technologists at Magnopus, the concert featured a bilingual duet where Yatra performed alongside an AR hologram of himself against an immersive virtual environment with vibrant, eye-catching visuals.
Evercoast CTO Sebastian Marino shared: “Evercoast’s volumetric capture technology reduces the cost of production and simplifies the creation of high-end 3D digital humans, enabling teams to integrate AR holograms in virtual production workflows. Partnering with Magnopus, Verizon, and Pandora to bring Sebastian Yatra’s memorable performance to life is just one of many examples of how volumetric capture is poised to enhance the future of virtual production technology. Evercoast lets creators directly lens photorealistic 3D performances in-camera, without complex asset builds and motion capture clean-up.”
To create Yatra’s AR hologram, Magnopus collaborated with Evercoast during a one-day shoot at Evercoast’s volumetric capture facility at the Verizon 5G Lab in Playa Vista, CA. Prior to the shoot, Yatra’s performance, choice of song, choreographed movements, wardrobe, and hairstyle were methodically planned out in months of pre-production to ensure all elements would be optimized for volumetric capture. On the day of the shoot, Yatra performed the English portion of his iconic duet against a green screen while surrounded by 360-degree cameras. Yatra’s performance was volumetrically captured as an AR hologram via Evercoast technology and broadcast to monitors in the space, allowing Yatra to view himself singing in real-time as a 3D avatar. The full song was captured as one continuous shot, though the crew filmed multiple takes with different variations of the performance.
AJ Sciutto, Director of Virtual Production at Magnopus, shared, “This was my first time working with Evercoast technology, and the volumetric video processing time and cost-per-minute was significantly less than competitors. It was nice working with a creative partner who understood the technical requirements of getting this done and stepped up to the plate to make sure those needs were executed well.”
The concert was live operated and captured in a virtual production shoot on an LED volume, where Yatra performed the Spanish portion of his duet. Magnopus developed a real-time workflow that enabled the crew to view the AR hologram meshed with Yatra’s live performance. Using Verizon’s 5G network, Magnopus sent the camera tracking data from the LED volume to an instance of Unreal Engine running on Verizon’s 5G Edge with AWS Wavelength. This allowed the team to render Yatra’s hologram with the virtual camera data applied, then stream the video to on-set monitors, where the camera operators could view all virtual and live elements combined. Sciutto added, “This all happened in real-time, with millisecond latency, allowing the team to frame shots in final composition, with the real Sebastián, hologram, and virtual background all on their monitors.”
In addition to Tacones Rojos, Magnopus filmed four additional songs and a Q&A with Yatra, which were edited into the final virtual concert and streamed to audiences in June 2022.
“The streaming of high fidelity 3D objects into game engines is part of our exploratory work of distributed rendering using Verizon 5G Edge with AWS Wavelength,” shared Andrew Zarick, Head of Game Engine Tech Incubation, Metaverse & Web 3.0 team, Verizon. “Working with Evercoast, Magnopus, and Sebastian Yatra was a fantastic way to make this exploration visible to an engaged audience through a timely musical moment.”
Sciutto added: “The visuals were amazing, working with such a talented artist was incredible, and my favorite part is knowing that this is the first step towards a revolution in filmmaking. By offloading part of the rendering pipeline to the cloud via edge compute, we took the first steps towards proving that multi-million dollar machine rooms aren’t required to be on site to run an LED wall stage. Evercoast, Verizon, and Pandora were all fantastic partners to work with, and everyone was committed to making a standout virtual experience.