Numerical Precision

Data-plane processing is about processing a large amount of data and computation as efficiently as possible. Using algorithms in narrow data types through the processing path is one of the main tricks used improve efficiency.

For most graphics algorithms, the eventual color outputs from a render are likely to be RGBA8 surfaces that only store 8 bits per color channel. Therefore, it is not necessary to use the full FP32 floating-point value.

A FP16 value is a viable alternative for color computation. In many compute problem domains, it is possible to go even further towards smaller data types.

Previous Next