Shader precision
The OpenGL ES and Vulkan graphics standards have their own definitions of data types. Variables are declared using a precision qualifier that defines the minimum precision that the implementation can use.
However, an implementation can substitute a more precise variable if required. As shown in the table below, the specification for the OpenGL ES shader language for floating-point values states that:
| Declaration | Range | Magnitude | Precision |
lowp |
[-2, 2] | [2^-8, 2] | Absolute 2^-8 |
mediump |
[-2^14, 2^14] | [2^-14, 2^14] | Relative 2^-10 |
highp |
[-2^62, 2^62] | [2^-62, 2^62] | Relative 2^-16 |
lowp float values can be stored using a 10-bit fixed point, mediump float values can be stored using a 16-bit floating point, and highp float values can be stored using a 32-bit floating point. However, the resulting output depends on the underlying GPU.
Mali GPUs do not distinguish between lowp and mediump variables. This means that both are mapped to 16-bit data types, and highpvariables are mapped to 32-bit data types.
Note: The older Mali-400 series GPUs, which are based on the Utgard architecture, do not support highp processing in fragment shaders. Therefore, all variables will be treated at 16-bit variables when using Mali-400 series GPUs.
Rounding modes
In addition to the floating-point precision differences, consider how floating-point values are rounded if a result cannot be exactly represented. There are broadly three main categories of floating-point rounding available:
- Round to nearest.
- Round toward 0.
- Round away from 0.
Round to nearest gives the least additional error, up to 0.5 ULP, compared to up to 1ULP for round toward 0 and round away from 0. However, round to nearest is slightly more expensive to support in hardware. As with numerical precision, the OpenGL ES graphics specifications does not tightly define the implementation.
Mali GPUs default to rounding to nearest, although this can be overridden by the application in some versions of OpenCL.