Brompton Technology Powers Virtual Film Environments

For anyone who missed the opportunity to visit Rebellion Studios and get a look at the virtual production technology demonstration provided by MBSE, Chris Deighton, CTO of Brompton Technology, a specialist designer/manufacturer of cutting-edge LED video processing products for the live events, film and broadcast industries, gives an overview using LED screens.

Getting started with LED screens on camera

They are not just for big-budget sci-fi: LED screens are becoming an increasingly common sight on all kinds of film projects. From simple video backdrops and greenscreen replacements that emulate the world outside a window, through to full virtual productions with camera tracking and real-time perspective rendering to capture a fully-immersive new world live in-camera, LED screens offer many benefits to film sets – but only when set up correctly and matched with the right processing system.

As with all on-set technologies, the choice of equipment has a huge bearing on usability and the quality of the end result. For film industry veterans who are new to LED screens it may seem a bit daunting – but the creative flexibility they offer, as well as benefits such as realistic lighting and reflections, can make them well worth careful investment.

Benefits of LED screens on-set

LED screens offer a dynamic alternative to traditional greenscreens, allowing for a seamless blend of physical set with pre-shot or VFX environments. Rather than a static green backdrop, directors and DPs can now see the full picture through the lens, live on set, and can work collaboratively to make changes there and then. This makes it much easier to get the right shot, without needing to fix things later with expensive post-production or pick-ups.

Expensive location shoots with full cast can now be streamlined into one trip with minimal crew – capture the scenic elements once in ideal light, then bring them back to the studio and recreate them as many times as needed in a controlled environment. VFX teams can create intricate, fantastical worlds that, for the first time, actors can actually see and react to in real time.

One of the most notable benefits of using LED screens is also a key creative limitation of greenscreens: realistic ambient lighting. LED screens are emissive light sources, so by wrapping a screen around the set, more dynamic and nuanced light can be delivered from every direction. This increases realism and allows for reflections off physical surfaces. To ensure the screen retains sufficient contrast and brightness, traditional set lighting should be controlled to balance with the screen and minimise light spill onto any screen surfaces that will appear in shot.

When it comes to LED screens, whether buying or renting, there are two key aspects to consider: the LED panels that bolt together to make up a complete screen; and the processing system that takes in a video signal, distributes it to all the panels and drives the LEDs to display the desired image.

Choosing the right panels

The choice of LED panels is typically considered far in advance of a shoot, and there are several important decisions to be made. First, a suitable pixel pitch (typically around a few millimetres) must be selected. It needs to be fine enough to minimise the chances of seeing moiré patterns, based on the planned shots and lenses, but not so fine it drives up the price unnecessarily (as finer pitch screens are more expensive).

Also, the smaller LEDs used for finer pitch screens are typically less capable when it comes to the brightness and colour gamut the screen can achieve – so it is important to determine the right trade-off for your project. If you need a huge LED wall to serve as a distant landscape backdrop, ensure that LEDs with sufficient brightness and an appropriately wide colour gamut are specified (especially if HDR performance is required – more on this later), and look for a suitably low ‘scan-mux ratio’, which translates to better performance on-camera.

These are all hardware choices, so it is important to get them right up front, as incorrect choices cannot easily be corrected later.

Choosing the right processor

The next important decision is picking the processing system. Panels are tightly integrated with a particular processing system, with electronics built into each panel paired with a central LED processor that controls the whole screen and feeds it a video signal. Again, this means it is important to select the correct processing system from the start, as it cannot be changed later if issues arise. As well as being responsible for the entire video pipeline within the screen, the processing system includes the user interface for configuring the overall appearance of the LED screen, ensuring it appears completely seamless, scales the content properly, and shows precisely correct colours with no colour casts.

Typically, filmmaking environments run LED panels at lower brightness levels than might be used for a live event or outdoor situation. Running a panel designed to output thousands of nits of brightness at a tiny fraction of that level can result in banding and colour-casts, so it is important to specify an LED processor that can compensate for it. Very few of the processing systems available are actually capable of achieving the performance required to deliver artefact-free images on camera, especially at low brightness, as the majority of these systems were originally designed for much less demanding applications like low cost digital signage.

A system will always be let down by the weakest link in the chain, so as when pairing cameras and lenses, it is important to pair together good LED panels with a premium LED processing system.

A camera test at this early stage is highly recommended to build confidence for both the creative and technical teams that the combination of LED panels and LED processing are able to deliver the desired performance. Testing should check for issues like rolling black bars, flickering, banding or poor colour quality. The moiré effect is also a common problem when LED screens are too close to the camera or in too sharp focus, requiring careful lens and depth of field choices. Camera tests must be done using exactly the same camera type as will later be used on set, and trying out the same types of shots that will be needed. Similarly, the correct processing system must be used during these tests. All cameras and LED screens have their own characteristics, and it is important to confirm that the proposed pairing of equipment works well together.

Set up for success

Selecting the correct processing system can give you a great deal of flexibility later on-set. This is crucial to enable delivery of the necessary image quality in what is often a rapidly changing environment.

Premium LED processing systems offer a full range of film-centric features such as the ability to freely drive the screen at any desired framerate, from 24fps up to high frame rates such as 144fps. Matching this to the camera framerate helps avoid motion artefacts, especially when shooting at high framerate or over-cranking the camera for slow motion shots. It is often also necessary to lock the screen refresh to the precise camera timing, and this can be achieved via genlock, with phase offset controls essential to avoid black bars appearing on modern rolling-shutter cameras.

Cameras see colour differently to the human eye and can pick up on colour casts or other distracting artefacts that destroy the intended illusion, so it is vital to have a full range of nuanced colour correction controls within the processing system. This ensures any unwelcome surprises, that only become apparent once on-set, can be immediately addressed so that colours are accurately portrayed on-camera. Making these adjustments directly within the display system ensures the best possible image quality can be maintained, as the display system internally operates at higher bit depth than the incoming video signal.

The value of HDR

To date, almost all LED screens have only been SDR-capable, but now some are also able to display HDR (High Dynamic Range) content. HDR is particularly interesting for on-set use. Compared to SDR it offers much larger contrast ratios and colour gamuts that more closely mimic what we see in the real world, and what cameras are capable of capturing. This ultimately provides a much more lifelike final image when the LED screen is being shot on camera, translating to an improvement in image quality and realism, and offering more flexibility in grading than if using a ‘flatter-looking’ SDR screen.

Coupled with a processing system that is capable of achieving accurate colours even for wide colour gamuts such as Rec.2020, HDR video content (whether pre-rendered or generated in real-time) can specify precise absolute brightness and colour values for every pixel, giving much better control over the display, and much more predictable results once the screen is integrated with other on-set elements, additional lighting, and even LED panels of different types or from different vendors.

How to get HDR

Achieving genuine HDR performance on LED screens has proven surprisingly elusive until recently because traditional LED panel calibration methods limit the brightness and colour gamut of the LEDs to match the capabilities of that particular batch of LEDs. Recently, new ‘dynamic’ calibration techniques have been developed to overcome this limitation and deliver the necessary performance gains to achieve genuine HDR output.

As well as being the only way to deliver genuine HDR performance, dynamic calibration technology enables the LED screen to be instantly reconfigured to achieve different colour or brightness targets. This maintains colour accuracy and uniformity, while unlocking the level of flexibility needed in a professional environment to ensure a perfect image can be captured first time, every time.

The film industry is perhaps the most demanding environment for LED screens to date, and therefore it also requires the most capable and flexible processing. By carefully coupling the right LED screen with the right LED processing there are significant benefits to the resulting workflow and creative process. This is why more and more projects, from car ads through to blockbuster films, are choosing to utilise this set-up. Maybe one day all films will be made like this?

Article by Chris Deighton, CTO, Brompton Technology