Introducing Neural Frame Rate Upscaling in Arm Neural Graphics SDK for Game Engines and UE plugins

Last October, Arm introduced the Arm Neural Graphics SDK for Game Engines and the Neural Super Sampling (NSS) plugin for Unreal Engine 5.4. That release established the SDK as an open, engine-agnostic foundation for bringing Arm neural graphics technologies into real game-engine workflows. It also gave Unreal Engine developers a direct path to try NSS through a plugin-based integration.

Today, we continue that journey with a new use case: Neural Frame Rate Upscaling (NFRU). Developed by Arm, NFRU is an AI-powered frame generation technology for Neural Graphics use cases. NFRU uses a neural network to generate intermediate frames between conventionally rendered frames. This increases the perceived frame rate and improves visual smoothness.

Why Neural Frame Rate Upscaling matters

Modern games must constantly balance visual fidelity, performance, thermals, and battery life. Developers want richer lighting, denser worlds, higher-quality effects, and smoother motion. However, each goal increases pressure on the GPU and the rest of the graphics pipeline.

While NSS reduces pixel rendering costs, it provides limited benefits for geometry-bound workloads. NFRU complements NSS by delivering smoother motion when geometry processing constrains graphics performance. NFRU addresses one part of that challenge by using a neural network to synthesize an intermediate frame from existing rendered frames and motion information. This is instead of asking the GPU to brute-force more complete frames.

By combining NFRU and NSS with 2X upscaling ratio, developers can render only 1/8^th of the pixels. AI generates the remaining seven-eighths. This approach is especially relevant for mobile games. A game that renders fewer full frames while presenting smoother motion can use the recovered capacity in different ways. Developers can improve sustained performance, reduce power, or allocate more resources to visual features. They can also provide a smoother and more responsive experience.

NFRU support in Neural Graphics SDK for Game Engines

In the Neural Graphics SDK for Game Engines, NFRU is as a modular use-case component rather than a monolithic engine integration. The API layer exposes NFRU through the SDK use-case provider model. The NFRU component manages the frame-rate upscaling workflow and delegates GPU work to the Vulkan backend. This modular architecture enables lightweight integrations while the SDK encapsulates the underlying NFRU implementation. At runtime, NFRU operates on two neighboring rendered frames around the target time T. The engine provides the rendered color frames, depth, motion vectors, and other required metadata. The SDK handles motion processing, candidate-frame construction, and neural composition to generate the intermediate frame.

Internally, NFRU has three major stages: optical flow estimation, frame generation, and frame pacing. The optical flow estimates dense pixel-level motion between neighboring rendered frames. Hardware acceleration supports this process where available. NFRU uses optical flow and motion vectors to build motion-aligned frame candidates. A lightweight neural network then combines these candidates to produce the generated frame. An optional frame-pacing part coordinates the presentation of rendered and generated frames. Alternatively, developers can use the game engine's native frame-pacing solution.

NFRU support for Unreal Engine workflows

The NFRU plugin for Unreal Engine uses the Arm Neural Graphics SDK for Game Engines and the native frame-pacing solution in Unreal Engine. We are also enhancing the NSS plugin for Unreal Engine by creating a unified Neural Graphics plugin that combines NSS and NFRU.

This unified plugin streamlines integration for Unreal Engine developers. It supports UE5.4 and UE5.6 versions. The plugin gives developers access to advanced neural graphics technologies for super sampling and AI-powered frame interpolation within their projects. This unified approach simplifies development and reduces integration effort. It also keeps the plugin aligned with the Arm open model-development ecosystem for neural graphics use cases.

The Neural Graphics plugin uses Vulkan® as the default Rendering Hardware Interface (RHI). It supports the ES3.1, SM5 shader profiles. The plugin supports Windows through the Vulkan emulation layer and Android platforms.

NSS enhancement continues alongside NFRU

Over the past few months, we have collaborated closely with partners through our early access program to improve the NSS models. These improvements increase image quality and give developers more flexibility to optimize for visual fidelity and power efficiency.

We have introduced three distinct quality modes for different device capabilities, content targets, and frame-time budgets: Quality, Balanced, and Performance. Each mode gives developers control over the balance between image quality and performance. Developers can select the mode that best fits the requirements and constraints of their game. This enhancement is now supported by the latest SDK and UE plugin release.

How to get started?

The Arm Neural Graphics SDK for Game Engines and NSS plugins for Unreal are open-source and distributed under the permissive MIT license. Developers can access the latest source code and pre-built binaries from our official GitHub repositories. Use the GitHub links below to download the resources. Follow the developer guide and learning path materials for integration guidance:

Arm Neural Graphics SDK for Game Engines  Neural Graphics Plugin for Unreal Engine

We want to hear your feedback

If you have a feature request or encounter an issue, contact your Arm representative or submit an issue on GitHub. Your feedback is highly valued and helps us improve our tools for the entire developer community.

You can also explore our extensive tools in the Neural Graphics Development Kit on the Arm developer website.