Get to grips with the technology that’s helping filmmakers bring crowds to life without thousands of extras.
Given that Queen (Bohemian Rhapsody), Elton John (Rocketman) and Elvis (er, Elvis) have all been given the biopic treatment, it was about time that “The Voice” was given a starring role. Enter I Wanna Dance With Somebody – the all-singing, all-dancing tribute to Whitney Houston, a decade on from the songstress’ death, lensed by Barry Ackroyd BSC for director Kasi Lemmons.
Viewers follow Houston’s soaring star, from wowing the audience at the 25th Super Bowl to singing at a concert for Nelson Mandela’s release from prison. It’s these electrifying scenes that present one of the film’s greatest technical challenges: bringing digitally generated crowds to life with as much authenticity as possible.
Crowd scenes of yesteryear required throngs of extras. Around 25,000 performers were involved in Metropolis (1927), 14,000 in The Ten Commandments (1956) and a whopping 69,000 in Around the World in 80 Days (also 1956). The advent of digital makes such scenes more efficient and cost-effective to create, but a recent development takes realism to another level.
London-based creative production studio Dimension Studio came on board I Wanna Dance With Somebody to help create digitally generated crowds in the largest volumetric capture production to date. The film’s VFX supervisor, Paul Norris, and VFX producer, Tim Field, had worked on Bohemian Rhapsody, which used similar crowd generation techniques to recreate Mercury and co.’s iconic performances, although the technology was quite new at that point.
Dimension Studio has a long-standing interest in volumetric capture, partnering with Microsoft in 2017 to launch the tech giant’s first mixed reality capture stage. Early on, they had been interested in exploring how volumetric capture could increasingly integrate with the filmmaking process and provide new approaches to digital human creation.
“With I Wanna Dance With Somebody, the timing was right, where we felt technology had got to a point and some of the toolsets we’d developed around it were mature enough to prove out that pipeline,” explains Simon Windsor, Dimension Studio’s co-founder and co-CEO.
Ahead of production, the Dimension team spent a year testing in 2021 to make sure that the assets and the pipeline needed to create the content (as well as the number of people needed to be filmed in a short timeframe) could be accommodated and work within the pipelines of the film’s VFX houses Zero and ReDefine.
“The key things in testing were to ensure we could provide assets with a level of realism but also flexibility for the post-production team to work with,” Windsor says. “We captured the performers in the volumetric stage with global illumination, which helped the post team with relighting when the performers are in different crowd scenes and there’s different times of day or different lighting conditions.
“The other aspect is scalability. So, capturing 300 people (filmed around 1,000 times), processing that volume of data, moving that data, then providing that to the VFX team in a way that worked within the schedule needed a significant amount of prep time to get right.”
Once workflows were firmed up, Dimension’s team headed to Boston, Massachusetts, where around 300 actors were captured in 2k and 4k on set over a two-week period in partnership with Avatar Dimension (a separate company). The process involved an 80-camera set-up with two types of cameras, RGB and infrared, to ensure that both texture detail and detail for areas where it typically might be difficult to see, such as folds in clothing, are adequately captured.
After the selects were made from around 700 terabytes of data, Dimension’s tech team processed some 510 minutes of volumetric content using the Microsoft Mixed Reality Capture solver. The assets allowed for recolouring of clothing and hair replacement for added variety.
The company has developed two proprietary techniques to increase the authenticity of volumetrically captured crowds, including prop replacement: “When we were filming the volumetric characters, they might have been waving a little stick. That was tracked with the motion capture technology then prop replaced in post with a flag.” There was also the development of motion vectors to ensure that the action on screen looks totally believable to the human eye.
Windsor sees volumetric as the future of filmmaking – not just for crowds. As well as the realism benefits, he also thinks it’s a valuable tool for a director during the post process. He notes: “They have complete creative control over the actions or the positioning of the crowd, as well as the way in which the camera can then move through that audience or the type of shot they can create. Traditionally, if you film plates then go to post and you change your mind, you’d need to necessarily reshoot the plates. In this instance, you’ve captured everything in three dimensions, so you don’t necessarily have to reshoot the plates. It provides creative efficiencies and more flexible decision-making.”
It’s not only the film and TV world where Dimension is implementing its volumetric expertise. The studios have worked closely with Sky Sports over the last five years, pioneering volumetric video for live sports broadcasts for the likes of the Open Golf Championship.
“We could recreate the entirety of Wembley Stadium by capturing no more than a few hundred people,” says Windsor,” and scaling that with the pipelines and toolsets we’ve developed to create some differentiation between the characters to enable it to not only look authentic, but to give it a next level of realism.”
With more productions embracing the technology than ever this year, it’s clear that volumetrically captured crowds are more than a fad. Who will be the next filmmaker singing its praises?
