This time last year, the idea of an artist preforming on an extended reality (XR) stage would certainly be out of the norm, but with the usual method of live performance currently out of the question, artists are looking at novel ways to create visually stimulating moments for their fans across the globe.
While many studios now offer this state-of-the-art technology, most are quick to explain that the solution is still in the early stages of development and a great deal of trial and error is required before it is perfected. This curiosity to play with technology is exactly what has been happening since May 2020 at a studio tucked away in a corner of North London. The brainchild of Pod Bluman of Bluman Associates, milkit studio has been established to act as, in the words of Bluman, “an XR Lab”.
While Bluman Associates had already been experimenting with disguise’s xR technology long before the lockdown – most notably with Dave’s critically acclaimed performance at the BRITS last year [featured in TPi March 2020] – it wasn’t until the then tenants of the Bluman Associates studio handed in their notice to vacate in May that the idea for a dedicated XR studio began to take shape.
“We suddenly had an empty studio and we wondered what we were going to do with it,” reflected Bluman. With demand for XR technology increasing rapidly, the team opted to create its own studio space. “We were very methodical while pulling together the pieces of this space,” stated Bluman. “Filming a project using disguise xR is a completely different process to filming in a regular studio in terms of workflow and process.”
So, rather than putting the studio on the commercial market, Bluman decided to bring in creatives to collaborate in the space and, over a period of four months, put on three different shoots.
“The goal of these collaborations was to really understand not only how the technology works but more importantly to understand the production process in a studio like this,” he explained. Bluman was pleased to report that after three successful projects, the team is now re-configuring the studio space, considering everything they have learned during this time, with the goal of opening up for demos in the next few months.
“The best way to explain XR as a concept is to demonstrate it,” he stated. “I try to explain it and people’s eyes start to glaze over, but when they see it in the flesh, the penny drops. My goal with the space is to make this into an XR lab, so show designers can come in here to experiment with ideas before taking it to a client on a large stage.”
Among the three successful projects so far was a performance from London-based Venezuelan artist, Baldo Verdú. The key creative on the production was Henrique Ghersi, who took numerous titles for this show including, Director, Designer, Notch Lead Designer and disguise Programmer. “I took on all these roles out of necessity,” he began, walking TPi through the history of the collaboration. “It allows me to deeply understand the creative and technical possibilities, challenges and requirements of the XR process in which every creative decision – from the design and cinematography, to the action of the performer – has to be informed by the capabilities and limitations of the technology. Despite this, virtual productions do open up a whole new world of real-time possibilities and it’s a very fertile area of live productions.”
In the first lockdown in March, Ghersi was involved in several virtual event projects while working from home. “One thing that came from those early experiments was the importance of having audience interaction with the DJs and performers I was collaborating with at the time.” During this time, Ghersi experimented a great deal with Notch in the creation of virtual events. “I began looking at virtual productions and spoke to Pod about his new studio,” said Ghersi.
This resulted in several months of collaboration, leading to a livestream performance from Verdú, broadcast on YouTube in September 2020. “What was fantastic about this project was the amount of time we had to work on this show, which you simply wouldn’t have on a bigger production, as well as the amazing support from Pod and the milkit team,” stated Ghersi.
This was paramount as one of the main things the creative was hoping to achieve was to look at new ways of incorporating an audience during the livestream and see how they could interact with the performer.
Interactivity Integrator, Kira May O’Brien and disguise Technician, Kyle Reseigh worked with Ghersi to develop this workflow. “During Baldo’s performance, we had an invited audience tune in via Zoom then we took their feeds and ran them through Notch to add effects to match the virtual environment,” Ghersi said. “This was then fed into our disguise media server and inserted in the XR world, allowing the audience to become part of the show.”
Baldo was able to see the audience throughout the performance not only on the rear LED but on the monitors, allowing him to visualise the audience and interact with them in real time. “After we were done, he commented on how powerful the show was, stating that he hadn’t felt that type of connection with an audience in months,” remarked Ghersi. As well as incorporating a virtual audience, Ghersi created a narrative arc for the four-song set. “I find the ability to take an audience on a journey from start to finish so compelling. It was something I really wanted to bring over to an XR performance.”
XR allowed the singer to inhabit and move dynamically through various virtual landscapes – the first of which was a stage floating on a rising ocean as the sun set, which Ghersi called “the last show on earth as we know it” – in which audience members were streamed onto sinking virtual IMAG screens. From there, the landscape morphed into a brutalist black and white number, drawing to a close in an ethereal energy driven world.
“The idea was to begin in a setting that would seem familiar to the audience but that was running out of time to then deconstruct the environment in different ways as the show progressed,” Ghersi revealed. “The narrative aimed to conceptually mirror the changes taking place in the world today as well as reflect on the new possibilities for the concert experience with XR.”
“As this happened, the aim was to integrate the audiences’ video feed onto the virtual environment in more and more prominent ways, such as a giant wall of moving cubes, so that they felt empowered and became protagonists alongside Baldo.”
GETTING TECHNICAL
Aiding Ghersi in the creation of this event was Lighting Designer, Douglas Green. No stranger to the world of virtual events, Green was involved in Javis Cocker’s JARV IS… Live from the Centre of the Earth performance, broadcast from a cave in the Derbyshire Peaks [see TPi August 2020]. “I paid Pod a visit to see the studio and he explained he was collaborating with Henrique for this project,” he reminisced.
Having worked with Ghersi during the We Will Rock You UK Tour, Green jumped at the chance to collaborate again. “We were keen to explore the role of lighting in this project,” stated Green. “My main role was to design both the physical lighting rig as well as the one that would exist in the virtual world and ensure they matched.”
To achieve this, Green designed the entire virtual rig, which was exposed to DMX and controllable from an MA Lighting grandMA2 PC Wing. The LD explained what it was like to transition from working with a physical rig, to one that existed within the disguise software. “Their software is just superb, and it gave us so many more options than you would have with a regular rig,” he recalled.
“We were able to control the ambient lighting via the HDR map within the virtual world as well as the digital lighting fixtures,” he added. “On some scenes, I even had the sun connected to DMX so I could alter its height and brightness.”
The LED screens used in the performance comprised two products. The rear wall was made up of Unilumen UpadIII 2.6mm, provided by AT Communication; meanwhile, the floor was made using Absen X2v 2.6mm provided by 4Wall UK (formally Smart AV). The backbone powering the system was made up of disguise gx 2 media servers. The camera capturing all the action was a Sony PXW-FS7 in combination with a Mo-Sys Engineering Tracking system. “We used the Mo-Sys system for the Dave performance,” interjected Bluman. “They were incredibly helpful during that show and it only made sense to bring them in for this project.”
The physical lighting for the show was provided by GLP. “When Pod approached me last year about this project, I was excited about the advancements of XR technology and how GLP can enhance the image and experience, bringing lighting and video closer together,” GLP UK Sales Manager, David Stewart commented.
The GLP team supported Bluman Associates by providing a flexible rig using some of the latest GLP products including the S350 LED profiles as key light, FR1 LED washes for back light, KNV for effects and experimenting in disguise, along with the versatile impression X4 bars to enhance many aspects of the performance.
“It has been a great collaboration and a learning curve to support the milkit studio with this exciting technology,” Stewart reflected. “As always, GLP is keen to support creative designers as they discover the possibilities of using both video and lighting pixels.” Alex McManus assisted Green, supplying the lighting control, while Alex Murphy handled timecode integration and Aaron Veness provided technical support.
“I’ve really enjoyed venturing into the world of livestream events,” concluded Green, giving his thoughts on the current movement to online solutions to live music. “As an LD, I’m not sure where these projects will lead, but it’s certainly been exciting to find ways of capturing the live feel into filmed events.”
It was a statement that was echoed by Ghersi, who is excited to see where these developments could lead in the future. “You can reach so many people with this style of performance, with geographical boarders disappearing,” he concluded. “I think the virtual gig was always going to happen, but due to the current situation, the timeline has been sped up by a number of years. There are still a lot of questions, but it’s amazing seeing the evolution and all the productions there have been since last year. I’m excited to see what comes next.”
This article originally appeared in issue #258 of TPi, which you can read here.
Photos: Lucio Martus