Development of LED Virtual Cinematography technology originated from the demand for real-time previews. In contrast to traditional blue or green screen-based real-time preview methods, LED Virtual Cinematography systems utilize a variety of technological means, including coordinated LED display systems, high-performance computer graphics rendering systems, camera tracking systems, and synchronization techniques. This achieves a true “what you see is what you get” shooting effect.
This technology not only avoids the color spill issues that may arise from large blue or green screens but also provides highly realistic dynamic environmental lighting for real scenes, ensuring accurate imaging of reflective or translucent objects. Simultaneously, through pre-production of virtual content and the rapid switching of digital scenes, it offers filmmakers an engaging and immersive shooting experience, greatly optimizing and shortening the film production process, enhancing production efficiency. Therefore, this technology has become a highly regarded direction in the field of virtual filmmaking.
However, as the virtual production industry gradually becomes a trend within the film sector, some significant technical challenges have begun to surface. Issues such as on-set reflections caused by high reflectance of screen surfaces, motion blur problems with high dynamic video sources, scanning line problems captured by cameras, and inaccuracies in color reproduction have emerged as pain points in the industry. These challenges have significantly compromised the overall experience of film production.
Advancing Virtual Filmmaking with AOTO Electronics LED Products
AOTO Electronics, in long-standing collaboration with China Film Group Corporation, a leading authority in Chinese film and television production, has played a pivotal role in the development of virtual filmmaking. Throughout this partnership, AOTO Electronics has continuously explored the dynamics of the virtual filmmaking market and user demands. Ultimately, its outstanding product performance has garnered praise from China Film Group and a vast user base.
Behind the numerous accolades from users, how exactly have AOTO Electronics products contributed to the advancement of virtual filmmaking?
Lower Screen Reflectance: The anti-reflective performance of the screen directly impacts the quality of the image during shooting. In the process of virtual filmmaking, the camera crew often uses additional lighting instruments to supplement. If the screen’s anti-reflective performance is poor, it can directly affect the integrity of the shot. AOTO Electronics places stringent requirements on screen reflectance, with diffuse reflectance ≤ 10% and specular reflectance ≤ 1.6%. This emphasis is evident in AOTO’s product development. The patented Mini 4-in-1 technology employs a cross-shaped light-absorbing groove design, effectively absorbing reflective light. According to experimental tests, it effortlessly meets the cinematic standards for screen reflectance.
Suppressing Moiré Patterns: In the process of virtual filmmaking, the occurrence of moiré patterns significantly reduces the overall lens adoption rate for the shot. The principle behind moiré pattern generation is the interference between two different light sources. AOTO Electronics believes that if a product can maximally increase the illuminated area of different light sources, it can fundamentally limit the conditions under which moiré patterns are generated. Here, we’ll compare AOTO’s patented Mini 4-in-1 technology with traditional SMD product light sources. It can be observed that the illuminated area of a single light source using AOTO’s patented Mini 4-in-1 technology is several times larger than that of traditional SMD products, resulting in an overall aperture ratio nearly twice that of traditional SMD products.
High Compatibility with Camera Image Sensors: In the shooting process, the most direct parameter indicating compatibility with the camera image sensor is the LED scan rate. The more driver ICs used in an LED display, the more LEDs lit at once, the smaller the denominator in the scan rate. This allows the LED display to complete a full image refresh more quickly, making it more friendly to the camera image sensor. This results in smoother images and reduces ghosting. Most mainstream products on the market are not below 1/16 scan. It is recommended to choose a display with no less than 1/16 scan, with a preference for products with 1/8 scan or lower. AOTO’s RM2.3 series can achieve a scan rate of up to 1/6, making it the highest among products with the same pixel pitch. This provides a more camera-friendly visual setting.
Higher Refresh Rate and Frame Frequency: When the camera captures rapidly moving backgrounds, choosing an LED display with a low frame frequency can result in flickering and image trailing due to the fast shutter speed. This decreases the overall lens adoption rate. AOTO’s products can achieve a 7680Hz high refresh rate and 250Hz high frame frequency, effectively addressing the mentioned issues.
AOTO Full-Angle Technology, Broader Shooting Perspective: In the process of virtual film and television production, to meet the shooting requirements of the tilted camera angle, the field of view should be as large as possible, typically not less than 140°. This ensures a flexible shooting angle for the on-site production. AOTO’s independently developed Full-Angle Technology achieves visual consistency at various shooting angles, guaranteeing color and image quality when shooting from an upward angle and maintaining brightness and color consistency when viewed from the side. This provides a more flexible and versatile shooting solution.
More Accurate Color Reproduction: Color gamut refers to the range of colors that an LED display system can reproduce and is a crucial indicator of the final image quality. In the virtual shooting process, it is recommended to choose a product with a DCI-P3 color gamut coverage of not less than 90%. AOTO Electronics’ virtual shooting products achieve a high color gamut level of 99.9% DCI-P3, ensuring the authentic reproduction of the shooting background. Through real-time rendering with game engines and camera tracking technology, virtual and real images seamlessly merge, creating a blend that is virtually indistinguishable.
AOTO Case: Tencent’s virtual shooting studio real shooting, the dotted line separates the real scene from the virtual scene.
AOTO RGBW Technology for Enhanced Lighting Effects: In the process of film shooting, the ceiling LED display is an indispensable component. It not only simulates realistic sky scenes to achieve more authentic object projection effects but also functions as a lighting source for supplementary lighting of actors/objects. Traditional LED displays consist of RGB chips, and using such technology for lighting could result in color deviation, leading to a lower overall CRI and an unnatural lighting effect. AOTO, keenly addressing this issue, has independently developed RGBW technology, ensuring a total CRI exceeding 90 and delivering lighting effects more closely resembling natural light.
Support for System Frame Reconstruction Technology: Frame reconstruction can achieve two effects: frame doubling and frame interpolation. Frame Doubling: In cases where the frame rate of the video source is low, frame doubling can increase the refresh rate of the LED display. Under normal circumstances, a video source image is refreshed and displayed on the LED screen only once in a given time frame. When the frame rate of the video source is low, the refresh rate is also low, making it easier for the camera to capture scan lines. In such cases, frame doubling can be employed to refresh the display of a single video source image multiple times within a given time frame, thereby increasing the LED display’s refresh rate. Frame Interpolation: This facilitates shooting foreground content with different backgrounds for multiple camera positions. During the time it takes to display one frame, multiple frames of video content are inserted. After synchronizing multiple cameras and LED displays using Genlock, adjusting the phase of the cameras with respect to each other introduces a phase difference between cameras at different positions, resulting in different images. This technology not only enhances overall shooting effects but also significantly increases fault tolerance in shooting. In case of a filming error, there is no need for actors to return for reshooting, allowing for easy post-production editing of the scene.
AOTO Virtual Production System
The components of the AOTO virtual production system can be categorized into the following areas based on their functions:
LED Screen System: Serving as the canvas for virtual imagery, this system, through an LED controller, accurately presents processed virtual images to the filming crew. The typical design involves a curved large screen combined with a skylight screen for immersive shooting experiences.
Rendering System, aka Virtual Production Server: This is the core component of virtual production, handling tasks such as rendering virtual images, regulating segmented outputs, tracking camera positions, and integrating virtual reality. The video server seamlessly integrates and debugs various software components within the system. In extensive projects, multiple servers can incorporate video frame synchronization cards to ensure consistency in video output.
Target Tracking System: This system enables real-time tracking of shooting targets, adjusting the screen to move with the tracked target. Target tracking includes camera tracking, character position tracking, and prop position tracking. This precise tracking allows for the generation of corresponding iterative relationships between the foreground and background within the large screen.
Video Production System: Specifically referring to 3D virtual digital asset production systems like UE, Unity, Notch, this system is responsible for constructing 3D virtual scenes. In virtual film and television production, all virtual digital assets are typically created in advance before shooting begins, a process known as front-end virtual content production.
All these systems synchronize through real-time data transmission, creating an integrated virtual production solution.
Attractive technology, AOTO’s journey in virtual film and television production continues to forge ahead! Currently, AOTO Electronics has successfully executed over 60 large-scale studio projects both domestically and internationally, playing a pivotal role in establishing virtual studios for tech giants such as Tencent, Microsoft, META, and more. In the future, AOTO Electronics remains committed to deeply cultivating the virtual production market, placing technological innovation at its core and aligning with user demands, thereby propelling the virtual production market to new heights.
