How Does AI Temporal Upscaling Like PSSR and DLSS Work?

AI temporal upscaling lets a graphics card or console render a game at a lower internal resolution, often half to one-quarter of the final pixel count, and then use a trained neural network to rebuild a sharp high-resolution image. Systems like Sony’s PSSR and NVIDIA’s DLSS do not guess missing details from a single still frame. Instead, they combine the current low-resolution frame with data carried over from previous frames, using game-engine motion vectors and tiny camera shifts to reconstruct fine edges while freeing the GPU to push higher frame rates.

What inputs power AI temporal upscaling?

Older spatial upscalers, including simple bilinear scaling and NVIDIA’s original 2019 DLSS 1, looked at one low-resolution frame at a time and tried to invent missing detail from neighboring pixels. That single-frame approach frequently left textures soft or introduced visual hallucinations around complex patterns. Temporal upscaling fixes the core problem by gathering pixel samples across multiple consecutive frames over time.

To keep each new frame from repeating the exact same grid of pixels, the game engine applies a sub-pixel jitter offset to the camera projection matrix every frame. According to a 2020 temporal antialiasing survey in Computer Graphics Forum, those microscopic shifts let the renderer sample slightly different points inside every pixel over several frames. As the frames stack up, the upscaler accumulates enough real geometric information to reconstruct edges finer than the internal render resolution could capture on its own.

According to NVIDIA’s technical documentation and Unity’s 2026 rendering manual, a modern neural upscaler pulls together four main streams of data on every frame:

  • The current jittered low-resolution frame rendered directly by the game engine, such as a 1080p image intended for a 4K display.
  • Per-pixel motion vectors generated by the engine, which record exactly where the camera and every moving object traveled between the previous frame and the current one.
  • The previous high-resolution output frame, stored in a history buffer and reprojected forward using those motion vectors in a loop NVIDIA calls temporal feedback.
  • Depth buffers and pre-trained neural weights, where depth data separates foreground objects from distant backgrounds while the network weights guide pixel blending based on ultra-high-resolution ground-truth training images, such as 16K reference renders in DLSS.

What does the neural network actually do each frame?

Non-AI temporal upscalers like Unreal Engine’s TSR or earlier versions of AMD FSR rely on hand-written mathematical rules to decide when old history pixels are safe to reuse and when they are stale. When those fixed rules misjudge fast movement, you see blurry textures or faint trails left behind moving characters. AI temporal upscalers hand that decision to a neural network that evaluates every pixel in context.

Early generalized AI upscalers, starting with DLSS 2.0 in 2020, used convolutional neural networks (CNNs) trained on non-game-specific content rather than per-game datasets. In 2025, NVIDIA’s DLSS 4 release replaced CNNs with the graphics industry’s first real-time transformer model for Super Resolution, Ray Reconstruction, and DLAA, followed by a second-generation transformer in DLSS 4.5 in 2026. NVIDIA Research documented in 2025 that transformer architectures evaluate broader spatial and temporal relationships across frames than CNNs, keeping thin lines, lighting, and fast-moving objects cleaner.

Running that math dozens of times per second requires specialized matrix hardware sitting alongside traditional graphics shaders. On PC, NVIDIA uses Tensor Cores and Intel uses XMX matrix engines on its Arc graphics cards. On console, Sony started designing custom machine-learning hardware for the PS5 Pro in 2020, targeting 300 TOPS (trillion operations per second) by enhancing AMD RDNA shader cores and tapping 15 MB of vector-register memory running at 200 TB/s, compared with 576 GB/s for the console’s main GDDR6 memory, as architect Mark Cerny detailed in Sony’s December 2024 technical seminar.

The performance payoff can be huge. In NVIDIA’s 2020 benchmarks for DLSS 2.0, Performance mode rendered internally at 1080p—one-quarter the pixels of 4K—and reconstructed a 3840×2160 output at two to three times the native frame rate. Quality mode rendered half the pixel count and boosted 1080p and 1440p frame rates by up to 76 percent.

How do DLSS, PSSR, FSR 4, and XeSS compare across platforms?

Every major GPU maker and console platform now ships its own neural temporal upscaler, though hardware support and software versions differ widely as of late 2026. On Windows PCs, Microsoft’s DirectSR API lets developers wire up a single interface that picks DLSS, FSR, or XeSS at runtime depending on your graphics card.

UpscalerHardware support (as of late 2026)Key architecture details
NVIDIA DLSS Super ResolutionAll GeForce RTX GPUs (RTX 20 through 50 Series)Real-time transformer model in DLSS 4 (2025) and 2nd-gen transformer in DLSS 4.5 (2026) running on Tensor Cores.
Sony PSSR and QSSRPS5 Pro (PSSR) and standard PS5 (QSSR)PSSR runs on PS5 Pro’s 300 TOPS custom RDNA ML hardware with a 2026 Project Amethyst upgrade; QSSR is a streamlined tier added for base PS5 in October 2026.
AMD FSR 4 and 4.1Radeon RX 9000 Series, RX 6000 Series, Ryzen AI Max/9/Z2, and upcoming Xbox Series X|SMachine-learning upscaler sharing roots with Sony’s Project Amethyst; unsupported GPUs fall back to analytical FSR 3.1.
Intel XeSSIntel Arc GPUs (XMX path) and non-Intel GPUs (DP4a path)Uses dedicated XMX matrix acceleration on Arc cards and a slower, softer DP4a shader fallback on competing GPUs; XeSS 2 improves motion-vector handling.

Sony’s console lineup split into two AI tiers during 2026. In February 2026, Sony announced an upgraded PSSR neural network for the PS5 Pro built through its Project Amethyst collaboration with AMD, debuting in Resident Evil Requiem before reaching games like Assassin’s Creed Shadows and Arc Raiders in April 2026. Ubisoft noted that the upgraded PSSR preserved fine moving grass and distant foliage with fewer manual developer tweaks. Then in October 2026, Sony introduced Quick Spectral Super Resolution (QSSR), a lighter neural network hand-tuned to bring AI upscaling to the standard PS5, starting with titles like Marvel’s Spider-Man 2.

AMD took the same Project Amethyst research to PC and Xbox. After launching machine-learning FSR 4 exclusively on RDNA 4 Radeon RX 9000 graphics cards in 2025, AMD’s October 2026 announcement expanded ML-based FSR Upscaling 4.1 via Adrenalin 26.10.1 to Radeon RX 6000 GPUs, Ryzen AI Max and Ryzen AI 9 processors, select Ryzen Z2 handheld chips, and upcoming Xbox Series X|S updates, broadening the FSR 4.1 rollout well beyond RDNA 4. Because version support shifts with driver releases, check your GPU vendor’s current release notes to verify exact card compatibility.

Most PC games expose these upscalers through a standard menu of presets: Off, Quality, Balanced, Performance, and Ultra Performance. Each step lowers the internal render resolution to gain more frame rate. On consoles, developers usually bind PSSR or QSSR automatically to the game’s Quality or Performance display modes rather than giving you granular scaling sliders.

Why does AI upscaling still show ghosting or shimmer?

Even strong neural models hit physical limits whenever the engine feeds them incomplete data. Disocclusion is the biggest culprit: when a moving character or car uncovers background pixels that were hidden in the previous frame, the history buffer has no valid samples for that newly exposed patch. Transparent objects like glass, smoke, or particles complicate things further because engines often exclude them from temporal motion vectors, which can leave faint ghost trails behind fast movement.

Low base resolutions also expose the ceiling of temporal reconstruction. If you pick Performance or Ultra Performance mode on a 1080p monitor, the GPU hands the neural network very few real pixels to work with, causing thin geometry like chain-link fences, power lines, and distant tree leaves to shimmer or soften in motion. Stepping the in-game preset up from Performance to Balanced or Quality feeds the network more raw geometry and usually clears up that instability right away. Keep in mind that running the AI model carries a fixed GPU time cost every frame, so an upscaler only boosts speed when the time saved by shading fewer pixels beats the time spent running the network.

Several common misconceptions also trip players up when troubleshooting image quality:

  • AI Super Resolution is not the same thing as Frame Generation. Upscaling rebuilds each rendered frame at a higher resolution, whereas features like DLSS Multi Frame Generation (available on RTX 50 Series GPUs, generating up to three extra frames in DLSS 4 or up to five in DLSS 4.5’s 6X mode) synthesize entire intermediate frames between rendered ones.
  • DLSS no longer requires NVIDIA to train a separate model for every individual game. Per-game training ended with DLSS 1; since DLSS 2.0 in 2020, NVIDIA has shipped generalized neural networks that work across titles.
  • Temporal upscalers are not post-processing sharpening filters. Sharpeners only increase contrast between existing pixels, while temporal upscalers pull genuinely new sub-pixel geometric detail from jittered multi-frame data.
  • Not every motion smear comes from your upscaler. Slow LCD monitor pixel response times physically blur moving edges on the panel itself, which can mimic or worsen temporal ghosting from the game engine.

Frequently asked questions

Does AI temporal upscaling add input lag?

No. Temporal super resolution usually lowers input latency compared with native resolution because rendering fewer pixels lets your GPU finish each frame faster. Latency increases come from separate Frame Generation features that buffer frames to insert extra ones—which is why vendors pair Frame Generation with latency-reduction tools like NVIDIA Reflex or Intel XeLL.

Can AI upscaling look sharper than native resolution?

In some scenes, yes. Native rendering still needs anti-aliasing (usually TAA) to smooth jagged edges, and traditional TAA often blurs fine textures while the camera moves. Modern transformer-based DLSS models and Sony’s 2026 upgraded PSSR can preserve thin lines, grass, and distant foliage more cleanly than standard native TAA.

Why should you turn off TAA when using DLSS, FSR, or XeSS?

AI temporal upscalers handle anti-aliasing directly inside their temporal reconstruction pass. According to Intel’s 2025 XeSS developer guide and Unity’s 2026 documentation, leaving a separate TAA pass active wastes GPU time and creates conflicting camera jitter or double-blended softness. Most games disable TAA automatically when you enable an upscaler, so check your graphics menu if the image looks unusually blurry.

Recent Articles

Related News

Leave A Reply

Please enter your comment!
Please enter your name here