Making Tencent neural image coding practical on mobile with Arm SME2

Learned models are increasingly complementing traditional algorithms in established computing workloads. Image compression is a notable example. Neural approaches have advanced from research to formal industry standardization, including JPEG AI (ISO/IEC 6048 | ITU-T T.840) and IEEE 1857.11-2024 for neural network-based image coding.

At the forefront of this progress, Tencent’s neural image codec, which is also an implementation of the IEEE 1857.11.1 standard currently under development, achieves around 30% better compression efficiency than a HEIF encoder implementation (a modern image format widely used in smartphone photography) at comparable visual quality, depending on image content and encoding settings. It uses learned neural networks to build a compact image representation with fewer bits. For users, that can mean smaller image files and less network bandwidth when photos are stored or shared.

Realizing those benefits on mobile also means making neural decoding fast enough for everyday use. When an image is opened, its compact representation must be reconstructed into millions of pixels. A neural decoder does this by running learned neural networks rather than relying only on conventional, rule-based codec operations. Neural decoding adds significantly more computation between compressed data and a viewable image. On a phone, decoding still must happen fast enough that opening a photo or swiping to the next one feels immediate.

Tencent Media Lab and Arm collaborated to address this compute challenge, bringing together Tencent’s work on neural image coding with Arm’s CPU optimization experience and Arm Scalable Matrix Extension 2 (SME2). The teams focused on making neural reconstruction fast enough for practical mobile use while keeping the implementation reusable across a broad range of devices. Tencent’s image-browsing demo below shows how that acceleration translates into the viewing experience.

Tencent’s 2K image-decoding demo with SME2 enabled on the left and disabled on the right, using two CPU threads. Playback is slowed by 5× to make the decode wait visible; swipes are aligned and red rings mark touch input. 

Image reconstruction in this workflow has three main stages. Entropy decoding recovers the encoded features, the neural decoder transforms those features into reconstructed pixels, and the result is passed on for display. The neural decoder is the most compute-intensive stage, and the detailed analysis that follows focuses on its most time-consuming module as an example of the broader optimization effort. The latency results reported below refer to this module.

Figure 1. Image reconstruction flows from entropy decoding through the neural decoder to display. The module analyzed and measured in this article sits within the highlighted neural-decoder stage.

Accelerating the most time-consuming decoding module to 30 ms

For a 2048 × 1344 image, which contains around 2.8 million pixels, the most time-consuming module within the neural decoder runs in approximately 30 ms on a commercially available SME2-enabled smartphone used as the reference device for this study using two CPU threads. With SME2 disabled, the same module takes 61 ms. Using one CPU thread on the same device, enabling SME2 reduces module latency from 95 ms to 54 ms.

Figure 2. SME2 reduces latency of the measured decoding module on the reference device using one and two CPU threads. Speedup may vary between runs and across device configurations.

Bringing these gains to users at scale also requires the decoder to work efficiently across the devices they actually use. A common CPU path helps make that practical: the same module implementation and model-preparation flow can be reused broadly, while runtime dispatch selects kernels for the capabilities of each processor.

Optimizing the complete neural image decoder path

Accelerating the matrix-heavy core

Profiling made the first optimization target clear. Matrix multiplication and convolution accounted for the largest share of execution time, making them the natural place to apply SME2. On the Arm CPUs used in this work, SME2 drives a dedicated matrix engine that performs many multiply-and-accumulate operations in parallel, accelerating the decoder’s INT8 matrix-heavy computation.

The gain was substantial, but it shifted the performance bottleneck. As matrix computation became faster, tensor layout conversion, intermediate memory traffic, and fixed-point processing accounted for a larger share of latency. Accelerating the dominant compute block had shifted the bottleneck, so the teams continued outward into the rest of the neural-decoder path.

Removing work around the matrix kernels

Figure 3 shows that shift at the data-flow level. In the baseline path, values are rearranged into the layout expected by the matrix kernel. The matrix result is then written as an intermediate tensor and read back for fixed-point output processing. In the optimized path, the producer already supplies the layout the next kernel needs, SME2 accelerates the matrix computation, and the required output processing happens before the result is stored. The model still produces the same output but avoids extra movement and repeated handling between those steps.

Figure 3. The optimized neural-decoder path removes work around matrix computation. The baseline rearranges input data and makes an intermediate tensor round trip before output processing; the optimized path keeps a compatible layout, uses SME2 for matrix computation and processes the result before the final store. Dimensions, block sizes and animation timing are illustrative.

Some of this work can be removed during ahead-of-time model preparation. If one operation produces data in the layout required by the next kernel, a separate conversion pass is no longer needed. Likewise, when an intermediate tensor exists only to be copied or immediately consumed by the next operation, that boundary can sometimes be simplified to avoid another pass through memory. This is why graph preparation matters alongside kernel optimization. The kernel determines how efficiently the computation itself runs; the graph determines how much work is needed to prepare its inputs and handle its outputs.

Numerical processing has a stricter constraint. Tencent’s decoder uses explicit fixed-point scaling, rounding and clipping, and those operations are part of its defined behavior. They cannot simply be removed or reordered, because once rounding is involved, changing the arithmetic order can change the final integer result.

The optimized implementation therefore moves the required work without changing the result. The matrix kernel applies output scaling and floor rounding after accumulation and before storing the output. This removes the need to write an unprocessed intermediate tensor and read it back for a separate graph-side pass. It preserves the required arithmetic while eliminating the extra movement around it.

Measuring the combined gains

Figure 4 shows how these changes accumulated during development. Starting from an earlier 99.2 ms development baseline on the reference device using two CPU threads, matrix acceleration produced the largest initial reduction, followed by numerical and data-flow optimizations. Together, these changes reduced module latency to 30.0 ms.

Figure 4. Cumulative optimization of the neural decoding module on the reference device using two CPU threads, reducing latency from an earlier 99.2 ms development baseline to 30.0 ms through matrix acceleration, numerical optimization, and data-flow optimization. The stages are cumulative, with SME2 enabled from the matrix-acceleration stage onward.

Looking inside the final optimized module shows where those gains come from. Figure 5 breaks down the workload with SME2 disabled and enabled for both one- and two-thread configurations. In each case, SME2 sharply reduces matrix-compute time, while data movement and fixed-point processing become a larger share of the latency that remains.

Figure 5. Execution-time breakdown of the optimized neural decoding module on the reference device, with SME2 disabled and enabled using one and two CPU threads. SME2 significantly reduces matrix-computation time, making data movement, conversion, and fixed-point arithmetic a larger share of the remaining latency.

The bottleneck shift is visible in both configurations. With one CPU thread, matrix computation falls from 75 ms to 35 ms, while data movement and fixed-point processing remain largely unchanged. With two CPU threads, matrix computation falls from 46 ms to 18 ms as total module latency drops from 61 ms to 30 ms. The optimization loop is therefore straightforward: profile the complete model, accelerate the dominant cost, measure again, and follow the bottleneck as it moves.

Extending SME2 acceleration across mobile devices

Beyond the reference device used for detailed optimization, we evaluated the neural decoding module across several recent commercially available smartphones. Figure 6 shows that enabling SME2 reduces module latency on each tested phone, using one and two CPU threads.

Much of the decoder, graph preparation, and kernel optimization can be reused across devices. Runtime kernel selection, build configuration, and validation are adapted to each target. This reuse reduces the amount of device-specific optimization required across different processors and operating systems.

Figure 6. Neural decoding module latency with SME2 disabled and enabled across recent commercially available smartphones, using one and two CPU threads. Numerical latency labels are intentionally omitted to emphasize SME2 performance improvements across devices rather than rank individual products.

Conclusion

Neural image coding can reduce the number of bits needed to represent an image while preserving comparable visual quality. However, practical mobile deployment also depends on reconstructing that compact representation quickly when the image is viewed. Making that reconstruction fast requires more than accelerating one class of operation.

Tencent and Arm optimized the neural image decoder from the largest compute block outward. SME2 accelerates the matrix-heavy core, while graph, runtime, and kernel changes reduce layout conversion, intermediate data movement, and repeated numerical processing around it. Together, those changes bring latency in the most time-consuming decoding module to approximately 30 ms for 2K images on the reference device using two CPU threads, while retaining a common CPU path that can be reused across devices. In the full decoding pipeline, these optimizations translate into an overall decoding time of approximately 76 ms on the same device using two CPU threads, as illustrated by the video demo.

The engineering pattern is to follow the bottleneck as it moves. Faster matrix computation exposes costs that were previously hidden behind it, and reducing those costs reveals the next constraint. Repeating that cycle across the complete neural decoder is what turns hardware acceleration into lower model latency.

To learn how to profile models with SME2, start with the model-profiling examples on GitHub, which demonstrate an end-to-end profiling workflow using public open-source models.

Acknowledgments

This work was jointly developed by Jason Zhu, Jackie Lee, and Damien Dooley from Arm, and Greyson Xu, Neo Zhang, and Yiming Li from Tencent Media Lab.

We would like to thank Shan Liu from Tencent Media Lab for the support and direction provided throughout this work. We also thank Stephen Peng, Cherine Gu, Tyler Mullenbach, Ronan Naughton, and Shantu Roy from Arm for their support and guidance throughout this collaboration.

For further information or any questions about this work, please contact:

• Arm: chinadeveco@arm.com
• Tencent Media Lab: medialab@tencent.com