With virtual production still in its youth, workflows and jargon can vary from project to project. Four specialists reflect on the importance of clear on-set communication for successful collaboration in the volume.
If there’s a single imperative that characterises more or less everything that’s said about virtual production in 2023, it’s the critical need for communication. Virtual production (VP) is no longer entirely a new idea, but it’s recent enough that there’s a lot of iteration, the tools are far from mature, and the division of labour is nowhere near as universally understood as it is in other areas of film and TV production.
Pete Blakemore is a camera assistant whose experience in virtual production encompasses most of the young field’s history, with credits on the BBC’s Dinosaurs, a music video for Coldplay and commercials for Rolls Royce and Ford. In a field which draws from both virtual and real-world technique, Blakemore’s background is as member of an entirely conventional camera department. “I was a trainee, then a loader, then a focus puller. Six or seven years ago I specialised in commercials and then about three years ago VP came along and I loved it. Something really attracted me about the way we can shoot four locations in a day!”
Taking advantage of that means navigating a fast-moving situation in which techniques and technologies can change quickly. “I started on one of the first commercial shoots back in November 2020,” Blakemore says. “Since then, I’ve done 17 commercials or small-scale shoots. Bar two or three, they’ve all been really different and there’s been a learning process with each one. On the first one, I was like, OK, this is how it works. On the second one, it was a different tracking system, the VP team had different people doing different things.”
The solution is in sheer communication. “When one of my DPs says there might be a VP job, the first thing I ask is which volume is it in, and if it’s one I haven’t worked in before I’ll go and recce it, come down for a chat and a coffee and learn about the process as they’ve done it.” The way that process interfaces with an established camera department, Blakemore says, “depends on the DP. Sometimes they’ll want me to handle more of it, sometimes they’re very clued-up on VP. Sometimes I’ll be talking to the VP people quite heavily; sometimes a creative decision may get passed through me.”
Fluid as that situation has been, Blakemore suggests that friendly conversation between crew members might be a route to settling things down, particularly in the lighting department. “Gaffers are learning what they need to ask for. The DP may ask for a 20×20 with an 18K behind it. If they explain that light to an Unreal operator, the operator will then remember what the equivalent is in Unreal Engine. There are already some great Unreal operators and VP teams out there who know how to work very closely with the camera department. They come from a different background but we’re all on the same team and that eagerness is very much appreciated.”
That process of regularisation is something Blakemore has already started to notice in practice. “When I started in November 2020, it was the DP’s first job on virtual production, and it was mine. Since then I’ve developed a shorthand with my team. They know a VP job is different to a regular job. I did an Unreal course with Final Pixel to help me understand more of what I was asking of them in some cases. There are a few jobs when I was like – hey, can we do this?”
Shifting roles
As film crews have explored the techniques of the virtual world, specialists with a background in realtime interactive simulation – video games – have moved in the opposite direction. VP supervisor Adrian Weber’s roots, as he puts it, are firmly in the computer world. “I wanted to get into the gaming industry first and foremost, programming video games. At university I did it, at a commercial company in Switzerland, then got hooked on film and commercial production. It was sweet timing that when I came back from that, the first season of Mandalorian dropped and everyone was talking about it.”
The way people are talking about it, though, often suffers from something we might call a collision of terminology, as Weber goes on. “It’s quite difficult to combine all the departments. We have people speaking film language, people from the gaming industry, and people from the VFX industry. There are now three universities in Germany rolling into this and we’ve talked about the curriculum. They all want to do the fancy in-camera VFX stuff, there’s a lot of top-level training going on, but you can’t discuss on a professional level if you only know the top level.”
“In the early days I explained my role as being a translator,” Weber says. “What does a director of photography mean by two stops darker? For a game artist that’s “what the hell, is it half as bright, a bit brighter?” And it’s still kind of a thing for people not coming from the film industry because we love to throw around abbreviations and nicknames. It’s definitely getting better because people are assembling crews around studios who learn how to work in both worlds. But still… the first time we had a gaming artist on set for a test shoot, an internal one, he started walking up to the director and DP, suggesting how he’d frame the shot. Maybe don’t do that in future! But if nobody told me, I’d have probably done the same thing myself.”
“Educating people across all three worlds is kind of tough,” Weber accepts, “because you get very good Unreal artists who’ve never been on set, LED techs who’ve been on set but have no clue how a gaming engine works. It’s kind of the supervisor’s role and to be able to talk to DPs in a language they’re comfortable with and understand what they’re trying to achieve. They’re all a challenge to find and in my experience it’s super helpful to have people who can talk all three languages. To understand where to troubleshoot when things go wrong, and to know their way around Unreal Engine, to suggest the solutions.”
Beyond the immediate on-set environment, things can be more variable. Where productions use a fully three-dimensional world, designing and building that world may involve a large effort from the virtual art department. That department, Weber says, is one of the most likely to change job to job. “Maybe a huge VFX vendor brings in their own team, then on another show you might get a different vendor… this is the part that’s still very unclear – who talks to whom. In theory it should be the virtual production supervisor [who] mostly will say the head of the virtual art department reports to them, but sometimes people say they should report to DP and production. It’ll be very fluid until we’re at the stage we have some more established workflows.”
Weber spoke during a lull in production created by a combination of factors: the writers’ and actors’ strike, the waning post-pandemic production bulge and the first signs of change in the business of streaming distribution. With skilled crew having recently been in demand, though, and likely to be again, discussion has turned to training for virtual production specialists. Given the film industry’s dubious record on the treatment of new entrants for much of its history, training is always an interesting question. “I’m curious to see how it turns out over the next couple of years,” Weber muses. “A ton of universities in the UK, Germany and Switzerland have been investing. Budapest has set up a stage. I’ll be curious to see how they match their education programme to the live sets.”
Combining disciplines
Cinematographer Maria Guerberof was among the first intake on the NFTS certificate in virtual production and found that the approach took care to reflect a combination of disciplines. “I really think the way they’ve done it is very clever. It’s about mixing people from different industries, making everyone is aware of the different ways of working. We had a common space to talk about what we thought we knew about it. For people who’ve never put a foot on set before, it’s something that’s completely alien for them. When we first talked about it together, it was very interesting to see the number of different roles.”
Guerberof’s principal interest, though, was practical, and something often raised: that a cinematographer is well served by at least a working knowledge of the fundamentals behind crafting and operating a virtual world. “As a DP the thing for me was to learn how to work with Unreal. I’m [teaching] at the moment, and I’m pushing them to learn to use Unreal and to have some introductory classes. I’m trying to help students understand how important it is to start looking at Unreal.”
Much as it shouldn’t be necessary for everyone to know everyone else’s job, some crossover knowledge, Guerberof confirms, can be “very important, because you can have a better communication with the VP supervisor, and understand how long it takes to change things. I hear a lot of producers saying we can’t find DPs who can work with this stuff – people who work with Unreal on set, specialists, are difficult to find. There’s a massive lack of people working on this and that’s what we were being told at the beginning, so I think that we need to make more money available to training to get people involved.”
Perhaps inevitably, that leads to ad-hoc working practices which must be explained before they can be put into practice, and that takes leadership. The average camera department, for instance, doesn’t usually need to explain to all of its members how the department operates, but that’s not (yet) true for a VP crew. Guerberof’s course involved not only time with the technology but explored how best to keep everyone involved up to speed, an exercise in management technique. “At the end of the course we took all the assets from Unreal, and from shooting, and we produced a document to explain everything. We set each step with graphics to explain who’s doing what. Once you’re on set and you know the DP is going to be talking only to the VP supervisor, there’s no time.”
The need to establish common practices could hardly be more urgent, and those involved seem keenly aware of the pressures of production. “At the level of the budgets that are using VP,” Guerberof says, “nobody should be guessing what they’re doing on set. Unreal specialists, computer people, people who’d been working with 3D and different software… I imagine the studios kind being more proactive about having the right crew. It’s important, because if you don’t have the resources to help the industry grow there’s not much you can do.”
Practical approach
Bright as the future may look, getting things done right now means dealing with a few practicalities. Leonidas Jaramillo, a cinematographer recently resident at Orbital Virtual Studios in Los Angeles, knows them well. “I’m a former teacher turned cinematographer, in 2005. I did the entire route as a DP. If you think about the history of our trade, about the classic master-apprentice role that we’ve had, the education system thought they could tap into that and to a certain point they did. You had these old masters, now retired out of the business, into academia. But there’s a disconnect between the real world and what happens in the confines of an educational facility.”
In March, Jaramillo found himself invited to Munich by ARRI to provide the keynote speech at a conference on the future of production. “They had invited all the young cinematographers and the gaffers to their secret, closed-door event. I was scared because these were my heroes in the audience – who am I to tell them what to do? – but it struck me that there’s so many methodologies, what works, what doesn’t work. There’s gaffers, DPs and production designers who have been in the wheelhouse of problem-solving. The best directors come from those positions, they understand what the pressures are, and they’ve been amenable and been able to solve problems.”
Even the core video wall technology, Jaramillo points out, is built from different panels, servers and processors, and capabilities necessarily vary. “Things you can do on [one] wall, you can’t do at [another], and they might have things we can’t do. Do you want high speed, do you want proximity for set building? If you can’t get within eighteen or sixteen feet of these walls which are 2.8mm [pitch] without moiré, how do you lay out the set? In the case of something like Mandalorian, a lot of those wide exteriors are easy because that’s scalable – you use a 50mm or an 85mm lens and we don’t see the ground so we don’t even care about scale, but that complicates your lens choices.”
“If we got to a place where that was happening in production, [problems] can be a time sink,” Jaramillo states, with admirable straightforwardness. “But that’s a very practical example of handling things in preproduction where a designer, production designer, art director, has an issue, and we can immediately diagnose that problem. A lot of the time colour is an issue – if you have, say, costume on [actors] on the background versus what we have in the foreground. The Unreal Engine designers are used to working in the tunnel of the computer system, but they can talk to the designer and see the texture of what’s going to be on set.”
That sort of fix-it-in-pre approach is universally recommended, but that only works when everyone is aware of the need to do it, which is rare in a world where post has long been established as the time for comparatively relaxed tinkering. Historical realities, such as the need to shoot model shots for traditional back-projection, are perhaps a better model. “We get the production designer, myself, and the Unreal Engine designer together in the same room,” Jaramillo says. “Sometimes with the director, sometimes without – if the director is off doing amazing things in Monte Carlo, we’re the people on the ground who can figure it out, we can connect the dots right there.”
If it seems that any discussion of crewing for virtual production inevitably boils down to a need for education, then that’s true to at least a degree. Some of that education might need to target technical know-how; much of it certainly needs to target process and procedure. Doing that, Jaramillo concludes, is key to the success of both individual productions and the technique itself as a business proposition. “If virtual production is to be adopted and become as ubiquitous as a greenscreen,” he concludes, “Education is paramount. And if you have a good experience with a doctor, you’re going to go back to that same doctor.”
–
Words: Phil Rhodes
