As the pandemic catalysed the growth of virtual production, disguise’s pioneering extended reality platform found a natural new home in the film and TV realms. 

The world of virtual production has brought together many different technologies and techniques that give broadcasters and filmmakers the creative freedom to produce exciting new visuals for their programmes and films. Key among these is extended reality (xR), which brings together a wide range of different technology for immersing people in a story or event. Among the first developers to embrace both the term and the technique was disguise, who has developed a solution for bringing together virtual and physical worlds for live production. 

The disguise xR system blends both the concepts of augmented reality (AR) and mixed reality (MR) with leading experience production software packages and powerful media server hardware to create a flexible set of tools that enable artists, producers and brands to tell inspiring stories. The field of extended reality has developed over many years but, as disguise’s general manager for broadcast, Grigory Mindlin, observes, modern technologies have allowed for a clearer definition of the term. 

“When we now talk about extended reality, it is more around LED volumes, whether it’s for broadcast or film,” he comments. “It’s about the use of LED to create a virtual environment or space and disguise has different components for building the ecosystem that enables this. These include the rx rendering nodes, where you can render Unreal, Notch or Unity graphics, and the vx media servers that take in the tracking and lighting information along with camera and video feeds. The vx servers then composite all that together with the content generated by the rendering units, with the output going to the video processor.”  

LEADING THE FIELD 

Mindlin joined disguise earlier this year, upon disguise’s acquisition of broadcast data and content visualisation solutions platform, Polygon Labs where Mindlin was CEO. Leading disguise’s innovation in the broadcast industry, Mindlin’s mission is to help broadcasters achieve the next level of quality broadcast graphics using Unreal Engine and disguise tools.

disguise is the leading developer of virtual and extended reality technology for the broadcast, cinema, and live event markets, powering over 600 immersive productions in over 50 countries. As well as its flagship xR platform, its product portfolio includes the Designer software for visualising, designing and sequencing projects; the rx range of rendering devices; the pro range of media servers; fabric, a preconfigured network switch; and RenderStream for connecting physical stages with virtual sets. 

“RenderStream is our proprietary protocol based around SMPTE ST 2110,” comments Mindlin. “It has our own adaptions on it and is for IP communication between the rendering nodes and the media server, with fabric connecting everything together. This allows us to ensure the lowest latency along the pipeline, with all the components being the base platform for xR. The advantages of xR include the ability to configure the virtual space and make sure that everything is calibrated and aligned. Usually, that is the hardest part, because when you are working in an LED volume you need to know where the cameras are and what the exact lens distortion is to match the graphics so it is correct on the wall, and you need to do all that as fast as possible.” 

At a basic level, extended reality is a replacement for green screen, which is long established as a technology to put presenters and actors in constructed, virtual environments. “It is the same concept as a green screen studio,” acknowledges Mindlin. “You have a volume that is in the space where the talent has to operate. And with xR we are just replacing the green with LED walls. The concept of set extension is identical. But extended reality as an overall technology has some very clear advantages over green screen. 

“Firstly, it’s much easier for the talent on the stage. In green screen, they have no clue what is going on around them. They have to imagine how the set looks or constantly look sideways or up or down to look at the confidence monitor. With xR, you actually see the content around you, and you have line of sight, which is much easier to work with. And the talent or guests can wear whatever they want, because they don’t have to worry about colours or green spill anymore, and it doesn’t matter if they are wearing metal glasses or holding a tablet. Reflections of the virtual world come from the LED surrounding them and are easily captured.” 

Because of this, Mindlin says, “using an extended reality system such as xR is much more user-friendly than green screen and allows productions to focus more on storytelling.” Leading broadcasters are already appreciating this greater operational and creative freedom, with many major recent productions made possible by disguise’s extended reality offering. 

ITV Sport opted for a virtual set as part of its studio presentation during the UEFA Euro championship, which took place in the UK last year. The broadcaster brought in disguise certified solution provider White Light to design and install a hybrid xR/physical studio space for real-time coverage. White Light employed six disguise rx II render nodes and three vx 2 media playback servers to generate and run the entire studio. With this set-up it was possible to easily scale rendering power for the Unreal Engine real-time scenes through the simple addition of more render nodes on a 25Gb IP network. 

The disguise xR workflow was also part of the staging for the 2020 MTV Video Music Awards. The production featured the disguise gx 2c media server, which was used as the central technology hub for a show that had to ensure social distancing was observed between the participants. The result was the first major virtual awards show in the US, which was filmed across several locations in New York and with two xR stages in California.  

Since the relaxing of social distancing restrictions post-pandemic, audiences are now back in their rightful place as a major part of live TV shows and other events. 

Mindlin comments that the continuing development of xR enables an audience to watch everything that is going on as if it were a traditional physical set. “They see the content on the wall itself, there is no need to put monitors all around. It’s also much easier for the director and the crew to see what they are working on,” he says. “Another thing that is really handy is it’s much easier to blend the talent in the virtual environment. Because the LED wall emits light, the talent is affected by the content you are putting on screen. That is true for broadcast or virtual production or cinematography or episodics. If you have a sunny scene with blue skies, there would be more light from the LED. And it’s the same thing if it’s a night scene and everything is dark blue-ish. This really helps in blending the virtual environment with the talent and physical props.” 

The capability and flexibility of disguise’s xR offering was further expanded this year through its acquisition of broadcast data and content visualisation specialist Polygon Labs, which already works with leading broadcasters including CNN, The Weather Channel and TV Globo. Through the combination of Polygon Labs’ powerful cloud-native production platform, turnkey services, and high-end graphics with the xR platform, disguise is aiming to offer filmmakers and programme producers tools for the next generation of graphics. 

“The focus of the acquisition was more on helping broadcasters introduce these new technologies like real-time graphics and xR into the traditional broadcast pipeline,” comments Mindlin. “This vision drove the development of Polygon Porta, which allows for easy control, editing, previewing, and managing of Unreal Engine graphics playlists.” 

In September this year, disguise launched Polygon Porta 2.0 and px, the integrated hardware and software solution for cloud-native broadcast graphics production. Together, these solutions enable broadcasters to seamlessly create, control, and collaborate on real-time graphics directly from a web browser. Porta and px will automatically match studio lighting conditions to graphics, manage tracking, control LEDs and more, all in one place.

“With these solutions, anyone can work on an extended reality production collaboratively, making everyday Unreal Engine graphics a simple operation. Team members can edit dynamic content on the fly, create playlists, and modify Unreal parameters, such as light values, as well as trigger disguise timeline cues to synchronise physical and virtual lights,” says Mindlin.

With Porta 2.0 and px, broadcasters can now reinvent the graphics in broadcast productions quickly and more easily than ever before. 

–

Learn more about Porta 2.0 and px

This article was sponsored by disguise

Words by Kevin Hilton