Streets ahead

Coronation Street holds the official Guinness World Record for being the longest-running television soap opera in history. The show premiered in 1960 and has since aired over 10,200 episodes. In October 2022, the series ran its ‘Super Soap Week’ storyline and turned to the specialist virtual production team at Recode XR to portray a nail-biting rooftop storyline. Recode’s virtual production supervisor Paul McHugh reveals the technical challenges of the work that went into in bringing ITV’s vision to life.

Coronation Street is known around the world for its stories depicting life in one of Manchester’s most famous streets. Every inch of its much-loved set was recently carefully reconstructed at its new home in MediaCityUK. Rumours were told that even the familiar sound of the overhead seagulls from its original home were replicated in its new home to keep a careful consistency for the audience. When the vision came for an ambitious storyline to air during the ‘Super Soap Week’, of a rooftop scene overlooking the Manchester skyline, the question was, how could this be achieved at the visual fidelity expected and still maintain the look and feel of the series? It wasn’t only the obvious concerns around safety and the unpredictable Manchester weather, it was also a question of authenticity for this heritage soap.

Paul McHugh, virtual production supervisor at Recode XR, describes that first call. “The call from ITV came one morning during some regular testing in the studio. It was immediately clear that the team at Corrie knew what they wanted and were dead-set on finding a way to bring that vision to life. They moved so quickly – in fact, they came in that very afternoon and we gave a full demo of how shooting in the volume worked. There was an immediate recognition of the expanded creative possibilities the virtual production process enables. Once we saw the brief, our minds were racing as to the potential pitfalls, after all this is still a very new technology and each project poses new challenges, especially for the level of quality required for a programme like this.”

From the outset, the two production teams seemed to gel. There was an understanding that the success of the work would draw upon a collaborative in-depth expertise across every step of the pipeline. The challenge for McHugh and the team at Recode was meeting the rigorous quality levels, using this new technology. The RecodeXR volume is situated just a stone’s-throw away from the ITV studio at MediaCityUK so it was a close working relationship from the start.

This really does feel like a brave step by the Coronation Street team, yet it was a risk driven by the desire to bring to life a storyline which would otherwise have proven very difficult using traditional techniques. Can we see this as a real shift for content production, enabling new storytelling possibilities? After all, crafters like this all work within the boundaries of budget, the weather and even the laws of physics, can we now start to look beyond those limits to see deeper, richer stories?

The challenge of this project was the requirement to replicate, in all its fine detail, a real rooftop mill based in Manchester. The level of scrutiny required
for this was a challenge in known tools used to create pre-rendered visuals, however this also needed to be rendered in the LED volume in real-time at the 8ms per frame required.

For this shoot, 50p was the required shooting format, so the scene required extreme optimisation when assembled within the Unreal Engine. Managing a heavy scene filled with photo scanned assets and millions of polygons to run at very high resolution at solid 50fps was definitely no easy task. This work is a real showcase of the power that virtual production puts into the hands of creative storytellers. The director for the Super Soap Week episodes, David Beauchamp, was able to craft an exact vision around an authentic version of the real mill, yet able to modify that to enable the required storyline to its best effect.

Beauchamp explains: “The whole team at RecodeXR were fantastic, guiding us through a steep learning curve so that we could create the world we needed to be in, tell the story dramatically and all whilst achieving the ambitious shooting schedule. We were able to modify the Mill design to suit the scene requirements, test every single shot and design the lighting well before the shoot began. With the support of Recode, the results were stunning and allowed us to create visuals that would simply not have been possible in the ‘real’ world’.

Photogrammetry was used to capture key characteristics of the physical space and also detailed features of the mill. A lot of time was spent on top of the real mill so that the talent didn’t need to, hundreds of photographs were taken and fine details of all the physical assets were recorded. Photogrammetry uses multiple different still photographs and or video clips to capture every possible angle of a subject, in this case a large mill and the Manchester skyline. The still images of the mill were put through photogrammetry software which creates a very high-density polygon mesh along with textures. This mesh is unusable in the virtual production pipeline, so it was then projected onto low resolution geometry and the resulting model and textures were used in the engine. The colour, texture, lighting conditions, angle, size, location could all be changed and even relocated in Unreal Engine. Details such as wall structures, coping stones and the shape of the blocks were recreated to a high degree of accuracy.

At one point, one particular section of the background needed to be dropped lower than the rest of the skyline due to a camera angle requirement. The virtual production team were able to do this instantly in Unreal.

The process guaranteed the authenticity of the environment, yet it enabled real creative flexibility to the director. Working alongside the VP pipeline, the Coronation Street art department went to immense lengths to create detailed rooftop props, which were then also brought into Unreal via the photogrammetry process so that digital doubles could be placed in different parts of a shot. There was a lot of measuring and matching between the physical and the digital.

Photogrammetry was also used to recreate the physical props created by the Coronation Street team. The scanned images of the props were imported into the engine. This helped in blending the virtual and the physical world and added a high degree of realism to the shot.

The next challenge was the not so insignificant task of creating the Manchester skyline which would form the immersive volume to the scene. The location of the actual mill was more than 20 miles away from Manchester, so another challenge was where and how to place the virtual mill in such a way that it was identifiable as being close to the city. The scene was achieved with a hybrid approach involving creating 3D buildings in the fore and middle ground and shooting plates for skyscrapers in the distance. The closer 3D buildings ensured there was a sense of parallax for any camera movement as well as reflecting real-time changes in light from the moon as it was moved into different positions. All importantly, the plates ensured that the skyline was unmistakably Manchester.

After one of the test shoot days, the director felt that the skyscrapers should be closer to the action, so the plates were cut up and placed on cards in Unreal so that, on set, their position could be adjusted in real-time on a shot-by-shot basis.

One thing which really impresses with this new pipeline, is that so much of the problem solving is achieved ahead of shoot day. It’s easy to see why this production methodology is proving a hit, not least because the talent and crew avoid cold winter evenings on the top of a Manchester mill. As we see more and more of these productions, one thing shines through from the director’s viewpoint, that of the control achieved within the volume. Everything from an all day long sunset, to playing God with the weather and altering entire physical environments at the relative push of a button, the director’s job seems to be that which it was originally intended, to capture the action in his/her original vision.

Secondly, from the point of view of the actors, what does being fully immersed in a photo-realistic environment mean to them? The talent are able to see final pixel as they go through each scene and they reported how they felt connected to the scene as they could view the rendered output in real time. “This is a soap first and we were all absolutely buzzing to be using technology usually reserved for Hollywood blockbusters,” says Mikey North, who plays Gary Windass. “It was an incredible experience and the finished result was even better than any of us could have expected. There was, of course, the added bonus of not having to be out in Manchester unpredictable weather for a change.”

Lee Rayner, head of production for Coronation Street, adds: “When a rooftop stunt sequence was mooted to open our autumn schedule, I immediately thought ‘volume wall’! Shooting in Manchester over several nights at the height of summer has ‘nightmare’ written all over it. Having total control of the shooting environment is a benefit to production that cannot be overstated. We did our homework in advance of the shoot, with many meetings between our senior engineers and RecodeXR, to ensure there were no unexpected surprises and the output would cut with our other scenes and still look like Coronation Street. VFX in-camera or in pre-production made the whole process stress free – especially when you have no flexibility with your transmission schedule.

“This sojourn into virtual production was a soap first – a great engagement piece with our crews, keeping them relevant and ahead of the curve. Editorially it opens up many new exciting story telling opportunities and I anticipate we will be back, playing with this technology again, in the very near future.”

Finally, what does all this mean for the VP supervisor? “We have worked with the Unreal engine for 5 years now and it’s amazing to see how productions are coming to realise its immense power within virtual production work,” says McHugh. “Many of those shots previously dreamt of which were too expensive or indeed impossible, are now within their grasp. For us as visual artists? It’s honestly the dream come true to be in a position to make this happen, it’s like opening a whole new box of magic tricks to bring to productions and of course we feel really lucky that these game-engine tools are ones we have championed for many years now. This particular collaboration was one of the best working experiences of my career – the team at Recode were made to feel a real part of the Coronation Street family working towards an incredibly ambitious creative goal. The two teams worked professionally and brilliantly together, with both aiming for the highest visual fidelity possible within every frame.”

–

This piece was provided by Recode XR Studio, a full-service virtual production studio based in Manchester, UK.