Ray Tracing is the Quality Step for Mobile Graphics

Ray tracing is a rendering technique used in computer graphics to generate realistic images by simulating the behavior of light in a scene. It works by tracing the path of light rays as they interact with objects in a scene and calculating the way that the light reflects, refracts, and/or absorbs off of those objects. The process starts with a virtual camera placed in the scene, which generates rays of light that pass through the pixel grid of the camera. These rays then interact with the objects in the scene, producing shadows, reflections, and other effects that make the final image look more realistic.

Ray tracing has become increasingly popular in recent years, particularly in the fields of video game development, animation, and special effects in films. But its computation intensiveness set high requirements for mobile devices' performance which was overcome by the powerful hardware of Galaxy S23. It brings the mobile gaming experience to the same level as PC and consoles.

While mobile hardware computational power grows, software rendering solutions must also be prepared to enable new technologies. SR Ukraine Graphics Team in collaboration with HQ System Platform Development Group did a great job bringing the ray tracing technology of Unreal Engine 5 to the mobile platform and prepared a realistic showcase to demonstrate a new level of Galaxy S23 graphics quality to users during Galaxy Unpacked 2023 event.

The Difference between Old Rasterization and New Ray Tracing Techniques

Ray tracing and rasterization are both methods used in computer graphics to create realistic images. However, they differ in how they approach the task of rendering 3D scenes into 2D images.

Rasterization is a technique used by most real-time 3D graphics engines. It involves breaking down the 3D scene into individual polygons, or "primitives", and rendering them one by one onto a 2D screen. This is done by determining which pixels on the screen are covered by each primitive, and then calculating the color and shading of each pixel based on the attributes of the primitive, such as its texture, lighting, and material properties. The result is a fast and efficient rendering process that can produce high frame rates for real-time applications, but it can sometimes lack the accuracy and realism of ray tracing.

On the other hand, ray tracing is a technique that simulates the physical behavior of light as it interacts with objects in a scene. It works by tracing the path of individual rays of light from the camera through each pixel on the screen and into the scene. As each ray hits an object, it bounces off in a different direction, with the color and intensity of the ray changing based on the material properties of the object it hits. It results in a highly accurate and realistic image that accurately captures the lighting and shadows of the scene.

A Practical Example of Image Quality Difference between Two Approaches Based on the Shadow Effect

In the following example, the castle wall has small gaps between the bricks. The shadow of this wall on the ground should have similar gaps that are lit because the light was able to pass through. The raytracing approach has no problem distinguishing those gaps because the occlusion test is being run for every pixel to detect whether it is shadowed or not.

However, the rasterization approach fails to distinguish the gaps due to insufficient shadow texture resolution (Figure 5 - left). This can be remedied by using higher shadow texture resolution (Figure 5 - right), but the trade-offs include longer render times and higher memory usage which is not always acceptable.

If we move the camera closer to the ground, with the raytracing approach shadow details scale accordingly (Figure 6 - left). With the rasterization approach (Figure 6 - right), however, the shadow details remain the same as before, because they are bound to the shadow texture resolution, and jagged edges quickly become apparent.

New Technology has to be used Optimally to Show Even More Performance

One of the most significant developments in the field of ray tracing is the introduction of the VK_KHR_ray_query extension. This extension, also known as "Ray Query," is a powerful tool that allows performing ray tracing queries in a much more efficient and effective way than ever before. These queries can be used to determine whether a given ray intersects with a particular object or set of objects, and to retrieve information about the intersection point, such as the surface normal, texture coordinates, and other attributes.

One of the most significant benefits of Ray Query, which was used during development, is to perform these queries without the need for additional buffers or data structures. Instead, Ray Query works by using the existing Vulkan pipeline, which can significantly improve performance and reduce memory usage. It makes Ray Query an ideal solution for the mobile platform due to the limitation of resources.

In addition to its performance benefits, Ray Query is also highly flexible. It performs a wide range of ray tracing queries, from basic intersection tests to more complex queries involving multiple rays and objects. This flexibility allows to creation high fidelity ray tracing applications.

One of the major challenges in raytracing is the computational resources required to trace a large number of rays. Each ray represents a calculation that must be performed to determine the color and intensity of the light at a particular point in the scene, and as the number of rays increases, so does the computational complexity of the task.

The number of rays used in a ray tracing simulation usually has a direct effect on "frames per second" or FPS. FPS refers to the number of times per second that the computer can calculate a complete image of the scene. Higher FPS values mean the computer can calculate more images per second, resulting in smoother and more realistic animation.

However, as the number of rays increases, the FPS value may decrease due to the amount of time it takes for the computer to calculate each ray. This can result in slower rendering times and a decrease in overall performance. To overcome this problem, various techniques are used to optimize raytracing algorithms, such as hierarchical data structures, caching, parallel processing, and decreased amount of rays to reduce the computational load and improve FPS.

Below are some of the practical optimizations applied on top of UE5's ray tracing rendering engine by the SR Ukraine Graphics Team to increase rendering performance.

Direct approach or rendering ray tracing shadows in the fragment shader is not optimal due to a lack of parallel processing. Instead, Ray Query functionality is used in compute shaders which provide advantages of rays parallel processing on hardware. In computing shader, a group of threads is executed in a unit called a "wave". Each wave can consist of either 32 or 64 threads, depending on the hardware and the configuration of the compute shader. In wave32 mode, each wave consists of 32 threads, which will provide a much better GPU load during rendering ray tracing shadows.

Another good point to increase ray tracing performance is to decrease the number of rays, but it also will decrease the quality of content. FidelityFX Super Resolution (FSR) and Unreal Engine's Temporal AA Upsampling based on Temporal Super Resolution (TAAU) were used to solve this problem.

It is a spatial upscaling technology that uses advanced algorithms to upscale lower-resolution images and video content to higher resolutions with minimal loss of image quality. It is designed to provide a performance boost for gaming systems by allowing games to be run at lower resolutions and then upscaled to higher resolutions without the typical performance penalty associated with running games at higher resolutions.

Ray Tracing is opening a new era of high-fidelity graphics on mobile. Described points & techniques are only beginning and many new fruitful outcomes are on the way.

GPU & Game Dev engineering family in Samsung will continue collaborating closely with our partners to deliver the best gaming experience for Samsung Products.

Stay Tuned!