Aliasing occurs when you try to capture information without using enough samples to faithfully recreate the object. For most practical uses, aliasing is an aspect of signal processing that cannot be ignored.
In audio and video, you get aliasing as you sample the raw data of sound or color to turn it into a discrete digital signal. Aliasing is more obvious at higher audio frequencies and in finer details of a video.
Aliasing occurs when you sample the raw data too infrequently. The threshold for aliasing is when the sampling rate is less than half of the highest frequency in the content. For graphics, aliasing is most obvious in the rasterization process of displaying objects from a camera that is in motion and is going against the discrete grid of pixels which form the display.
In a VR environment, there are two specific aliasing variants which are especially apparent:
- Geometric aliasing
- Specular aliasing
Geometric aliasing is most visible when a scene contains high-frequency input signals in color space that are displayed on the relatively low frequency of pixel samples, for example, a rapid transition between two contrasting colors.
This aliasing is especially visible in straight lines that appear to crawl when moving the camera. The following image shows an example of this geometric aliasing on the red edges of the shapes:
Geometric aliasing can also occur when meshes with high polygon counts are rendered onto a low-resolution display, resulting in too much detail due to the quantity of polygons with the width of a few pixels. Lots of polygons means lots of edges, and with few pixels to render them in, this results in more aliasing.
Also, a high density of triangles is inefficient for rasterizing in the GPU, with the GPU spending many cycles on geometry that is too small to make any significant difference to the final picture. Additionally, the excess cycles the GPU spends on rasterizing geometry means that there are fewer cycles left to execute fragment shaders. This is because the GPU has to spend many cycles rasterizing the geometry rather than on executing the fragment shaders.
Specular aliasing results from objects with sharp highlights which 'pop' in and out of existence as either the camera or the objects move. This manifests itself as pixels which appear to shimmer across frames as a result of the specular effect being present in one frame but not in the next. The shimmering effect is annoying for the user as it distracts their attention, impacting the sense of immersion and reducing the quality of the user experience. The ridges in the wall on the following image demonstrate specular aliasing.
Common anti-aliasing techniques such as supersampling are not feasible on mobile due to the computational complexity. While Multisample Anti-Aliasing (MSAA) is both effective and performant, it only acts on geometric aliasing. MSAA is therefore ineffective against under-sampling artifacts that occur within shaders such as specular aliasing.
Various techniques are required to mitigate each variation of aliasing, due to the many types of aliasing that occur within computer graphics, and more specifically VR.