A disguise gx 1 media server drove the visuals, for the live video installation that accompanied Esa-Pekka Salonen’s final performance as the New York Philharmonic’s composer in residence. The video was designed by artist Tal Rosner and responded to prismatic changes in the music with imaginative graphics and bold colour schemes and used a live feed of the orchestra as the basis for more patterns.
‘Foreign Bodies’ Salonen’s own composition opened the program: “we set the tone for a groundbreaking evening. Even if people didn’t understand how it worked, they could see the relationship of the orchestra, the music, the visuals and the audience. It was an art piece in response to an existing art piece, like a live ‘Fantasia 2018’” commented Rosner.
Rosner has been creating visuals for classical music performed by various orchestras for 15 years. “Rock and pop music always have visuals, but classical has not had it,” he noted. “It’s very exciting for me to create for classical music; it’s very close to my heart. I’ve worked on everything from Stravinsky to Benjamin Britten.”
Tal has partnered with Salonen before animating a previous piece for the composer-conductor. “ ‘Foreign Bodies’ was already in existence but he wanted to do a new spin on it, to challenge himself with a multidisciplinary approach. To make the music come alive I wanted to not only respond live to the orchestra but also generate dynamic images from the orchestra.”
Tal’s hyper-synched video interpretation of the orchestral piece drew inspiration from action paintings and Expressionist abstractions. By animating textural shifts and instrumental gestures and by manipulating live-feed images of both the players and the conductor on stage, the performance became a sort of closed circuit where the flow of sound and projected images were intertwined.
“We only had two actual rehearsals with the orchestra; we were very tight for time,” Technical Director/Video Software David Shepherd recalled. “Everything else was done with recordings. Before we left for the show we needed to build as much as possible.” He praises disguise’s “extremely tight integration with Notch,” which they used extensively. “When we develop a Notch look we know it will work with the media server, that it will run smoother and faster than it ran on the laptop.”
The orchestra was initially seen with no visuals displayed on the giant projection screen behind them, then visuals slowly began being introduced as the orchestra continued to play. “Layered on top of that were treatments of that particle data: the particles took the colours from the stage then created their own look. Although we had the freedom to abstract the movement viewers still felt that it followed the actual movement. Trumpet moves didn’t look like a trumpet, but the audience could still sense its specific movement.”
Shepherd explained that in pre-pro: “we used pre-existing clips of live orchestras playing and we developed some very simple looks and rendered them out and into After Effects to do something quite a bit more interesting. Then we took them back to Notch for the dynamic effects.”
During the program, Rosner and Shepherd used 2 4K cameras facing the orchestra to create 8 HD shots for the live feed. “We took those 8 outputs into a 12 by 12 matrix, which disguise was switching live,” Shepherd said. “We keyframed on the disguise timeline to switch between the 8 camera shots and focus on part of the raster. That night we had almost the equivalent of 8 cameras. We had 140 cues: 8 camera views, a lot of looks created in Notch plus some pre-rendered stuff.”
According to Rosner: “the power of the gx 1 definitely shone through on ‘Foreign Bodies.’ This was a very Notch-heavy piece. At times we were running a very intensive Notch layer with two pre-rendered video layers and two 4K camera layers coming in. We were processing a lot of data, and disguise didn’t even blink.”
“Foreign Bodies” was produced by Treatment Ltd. The Executive Producer was Sam Pattinson with Lizzie Pocock Producer and Nick Joyce Technical Director/Video Hardware.
