A virtual production stage is filled with equipment both familiar and unfamiliar to a DP, but even the familiar must be considered in new ways.

The prime requirement when selecting a camera for virtual production work is that it must either have a genlock input or be able to sync to a timecode input. This keeps the shutter in step with the graphics cards running the LEDs and prevents ghosting on the screens.

Large format cameras are favoured for their shallower depth of field, helping to blur the screens and prevent moiré, but they are not essential. Some DPs have reported that global or mechanical shutters get the most flicker-free results from the screens.

A potential benefit of volume shooting over conventional green screen is that vintage or characterful lenses, diffusion filters and haze can be used; many DPs have found that these help to blend the real foreground and virtual background together. It should be noted that a proportion of shots will likely be fixed in post- production, however, so not capturing clean images can still complicate the VFX work. Lens grids are often shot by the virtual production team and will certainly be required for unusual glass.

Lens data must be transmitted to the game engine by a motor controller. To render the correct frustum – the area of the LED screen being filmed – a crown or “sputnik” is mounted on the camera. This spiky attachment is tracked by a number of mocap cameras, so DPs and gaffers should forewarn the VP team of any “occluders” they intend to use – large objects like flags that might block the view of mocap cameras and impede tracking.

Shooting with two cameras on a volume is becoming commonplace. If the two frustums overlap, filmmakers must decide which camera the right perspective will be displayed for, while the other will need to be corrected in post.

The key specifications of the LED screens are pixel pitch and brightness. Pixel pitch denotes the distance between the centre of one pixel and the next. A screen with a higher pixel pitch will need to be more defocused to look convincing on camera. It will also have a poorer viewing angle, meaning that there will be greater colour shifts when seen off axis.

The walls of a typical volume today will have a pixel pitch of 2.8mm and a brightness of up to 1,500 NITs. Ceiling screens or fly-in walls that are intended more for interactive light than in-camera backgrounds will have a wider pitch of around 5mm but greater brightness, up to 6,000 NITs. It is worth remembering that LED screens achieve dimming by flashing at a lower frequency, which impacts colour rendition, so the best results will be produced when they are running brighter.

As the screens are RGB, skin tones can be improved by adding lighting fixtures which have white emitters. DMX data can be output from the game engine so that supplementary fixtures emit interactive light to match the backgrounds. Care must be taken to avoid “flashing” the screens – spilling light onto them – and the minimum black level of the LEDs is another specification to look out for.

Last but by no means least, the latency of the system should be considered. The game engine can be run by as many as 12 computers, each with a couple of GPUs – and the virtual production industry uses the fastest graphic cards in the world. Nonetheless, there will always be a lag between the camera moving and the frustum updating, potentially manifesting as a visible disconnect between foreground and background during fast camera moves or handheld work. Seven or eight frames of latency is considered good.

–

Words by Neil Oseman