High frame rate Mo-Sys StarTracker and NearTime for LED boost ICVFX workflows

Mo-Sys Engineering, world leader in image robotics and virtual production solutions, reveals new solutions for LED virtual production with a high frame rate (HFR) StarTracker camera tracking system and the award-winning NearTime workflow extended to LED volumes.

NearTime for LED is a smart solution to solve multiple challenges when shooting in-camera visual effects (ICVFX) in an LED volume. At its most basic level it is an automated background re-rendering service for improving the quality and/or resolution of Unreal based virtual production scenes, and is used simultaneously alongside a real-time ICVFX shoot.

NearTime solves one of the key challenges of LED ICVFX shoots, which is balancing Unreal image quality whilst maintaining real-time frame rates. Currently every Unreal scene created for ICVFX has to be reduced in quality in order to guarantee real-time playback.

Using NearTime with an LED ‘green frustrum’, the same Unreal scene can be automatically re-rendered with higher quality or resolution, and this version can replace the original background Unreal scene. Whilst it takes longer to do this, no traditional post-production costs or time have been used, plus moiré issues can be avoided completely!

Mo-Sys CEO Michael Geissler said, “We patented NearTime almost 8 years ago knowing that real-time graphics quality versus playback frame rate was always going to be an issue with ICVFX shoots. At that time we were focussing on green/blue screen use, but today LED volumes have exactly the same challenge.”

Mo-Sys is also announcing a special StarTracker for LED that provides camera and lens tracking up to 240fps for slo-mo shots, as would be typically used for fight scenes in Asian action films. In this scenario, rather than drive the LED wall at high frame rates to match the camera, which requires significant hardware processing and comes with an Unreal image quality impact, NearTime for LED is used to post-render the Unreal scene at HFR following the HFR ‘green frustrum’ shoot of the talent performing. During the slo-mo shot review, the re-rendered Unreal background and the talent foreground are re-combined, delivering a significantly higher quality composited shot.

“Proxies were used as a smart solution to solving computer processing limits in the early days of digital compositing and colour grading,” Geissler commented. “Similarly, HFR StarTracker and NearTime for LED is a smart solution to shooting slo-mo shots in an LED volume.”

The HFR StarTracker for LED has been designed in anticipation of future GPU processing improvements enabling higher frame rate Unreal scene playback. In conjunction with Mo-Sys’ VP Pro XR content server, the HFR StarTracker’s programmable phase shift feature can utilise Brompton Technology’s Frame Remapping or Megapixel VR’s GhostFrame capability for simultaneous multi-camera shoots in an LED volume.

NearTime for LED will be available in Q4 to Mo-Sys customers with VP Pro and VP Pro XR. The HFR StarTracker for LED will be available early Q1 2022.