If you’re a frequent gamer, by now you’ll have heard of ray tracing and NVIDIA’s DLSS (Deep Learning Super Sampling), two powerful tools that combine to boost performance in games and provide stunning visual fidelity by replicating realistic lighting and reflections. NVIDIA pioneered these technologies in gaming with the introduction of its RTX 20-Series graphics cards, and in the two hardware generations since then, the techniques and AI hardware behind them have improved considerably. With DLSS 3.5 and a new Ray Reconstruction system working hand-in-hand, ray-traced games can look all the more lifelike and run significantly smoother at the same time. So how do these technologies work?
DLSS has been an evolving technology. At its start, its focus was on rendering games at lower, easier-to-process resolution and then increasing the output resolution by filling in the gaps between pixels , giving gamers the benefit of sharper visuals with increased frame rates in lower resolution. NVIDIA managed this by training its AI model on high-quality game visuals so that it could understand what they should look like and know how to fill in the gaps when stretching a game’s lower resolution frames to higher resolutions. This process could also flipped somewhat with DLAA (Deep Learning Anti-aliasing), which can render at a display’s native resolution but use the same AI logic to figure out what an even higher-resolution frame would look like and then down-sample that to output an effectively anti-aliased image.
With DLSS 3, it introduced Frame Generation, which fills in pixel data between frames. While DLSS has always relied on the special Tensor Cores inside of RTX GPUs, which handle AI operations, DLSS 3 Frame Generation leverages even more hardware in RTX 40-Series GPUs, like Optical Flow Accelerators, to understand how objects in games are moving and intelligently blend between frames for extra smooth visuals. Combining the super sampling and Frame Generation tech in DLSS, can dramatically increase frame rates in games without a hit to visual quality.
Alongside these developments in DLSS, NVIDIA has continued to push forward its RTX ray tracing technology. Ray tracing remains a computationally intensive process, especially to get the most realistic results. It requires simulating a staggering amount of light rays and all reflections they make between a light source and the viewer, making it unreasonable to perform this for every pixel in every frame of a fast-running game. But taking a reasonable number of light ray samples produced a noisy image and entailed specially trained denoisers to get a usable image. Even then, the results could lack detail or have unusual ghosting artifacts, and the more denoisers necessary to handle numerous ray-traced effects could further bog down performance.
NVIDIA’s DLSS 3.5 introduced Ray Reconstruction to solve this. With Ray Reconstruction, all of the denoisers (and their corresponding computational needs) get replaced. Much like DLSS and Frame Generation use AI to figure out how to intelligently fill in pixels, Ray Reconstruction uses AI to fill in the gaps between simulated light rays. Ray Reconstruction has a deeper understanding of the games it runs on and their ray-traced effects, allowing it to know when to use different techniques to fill in missing data, helping it provide sharp visuals without artifacts.
For gamers, higher resolutions and higher graphical settings used to mean making big sacrifices to framerate. But the technologies wrapped up in DLSS 3.5, like Frame Generation and Ray Reconstruction, flip the script.
Now, gamers on any RTX GPU can peer into lifelike worlds with ray-traced lighting, shadows, reflections while enjoying higher resolutions. And those gaming on RTX 40-series graphics processors can tap into Frame Generation for yet another boost to frame rate. Taking advantage of these technologies is as simple as running an NVIDIA RTX graphics processor in your desktop or laptop (or even tapping into one through NVIDIA’s GeForce NOW Ultimate or Priority service), and making sure to toggle these features on inside of your games. You can find all the games and apps that support NVIDIA’s DLSS technologies here.
The way NVIDIA has leveraged AI to upgrade the gaming experience is just one way the company is putting AI to work. With NVIDIA’s RTX hardware, you can do a lot more than gaming, and NVIDIA’s AI Decoded blog series highlights many of the different ways you can take advantage of your RTX hardware running all sorts of AI-powered tools.