20170315 - Custom Resolves and Sample Patterns


Shadertoy
Direct Link to the Shadertoy
Change the #define MODE in the source to try different options, would have made it a slideshow but the shader explodes.

Warning, this is a post about temporal effects, so actually running the shadertoy and trying options is required to see what is being described. The stills below look quite different than the live version.

Background
The shadertoy is a moving simulation which looks about right when windowed with the source editor visible on a 1920x1080 display. Higher resolutions like full-screen and most of upper-right region is massively under-sampling. The image is a moving pattern designed to be challenging to resolve well. The resolve is spatial only, there is no temporal filtering. Both the resolve and grain are linear and energy conserving, assuming an sRGB output display target.

Note for noise in the shadertoy and for the jitter pattern, it just isn't practical or possible to have good patterns, because there is no way to load images in shadertoy. So instead there is reduced image quality, and some rather bad clumping for the jittered sample pattern. Doing this properly would yield a much better result.

Only Got One Sample Per Pixel, Use It Wisely
#define MODE 0
Starting with the worst possible option, a simulation of what the image looks like when rendering all the geometry with triangles, with no anti-aliasing, and the worst possible sampling pattern, the regular grid (ie the default practice for GPUs). The output is sharp, aliased, and quite wrong visually.



#define MODE 1
Same geometry rendered with a simulation of what is possible with programmable sample positions for the 2x2 pixel quad using this pattern (which looks like the 4xMSAA pattern but applied for the 2x2 pixel quad instead),

+-------+-------+
|       | 1     |
|   .   |   .   |
| o     |       |
+-------+-------+
|       |     2 |
|   .   |   .   |
|     3 |       |
+-------+-------+

The pattern has a symmetrical change to the ddx and ddy, possible to manage with a single descriptor LOD bias? The resolve in this case uses a simple filtered Gaussian, with just enough kernel width to make the checker pattern go away. The result is something much closer to the ground truth, which is much more temporally stable even at one sample per pixel. Note it is impossible to get rid of the moire pattern.



#define MODE 2
Now onto something not possible with GPU triangle rendering, a jittered sample pattern, but still at one sample/pixel, and this time no resolve kernel, but with peak added grain. Only way to view this correctly is inside shadertoy in the live version. Also can do much better with stratified sampling, but not practical in shadertoy. Moire pattern goes away, and under motion the image no longer looks like features are locked to pixels, instead the pattern appears to have smooth sub-pixel motion. Note the grain serves a strong purpose, to mask the artifacts of rendering with one sample per pixel.



#define MODE 3
Same as before, but re-introducing the gaussian resolve filter. In motion this now looks like a noisy version of the ground-truth, meaning it shares the same quality in motion and visual anti-aliasing. Note again this is a spatial-only resolve, temporal noise reduction can work wonders to clean this up.



With Two Shaded Samples Per Pixel
#define MODE 4
Typical 2xSGSSAA (shading both of the 2xMSAA samples), then applying the default box filter. Output is garbage in motion. Moire, aliasig, etc.



#define MODE 5
Switching back to the jittered sampling at 2 samples/pixel, with Gaussian resolve and grain, much closer to the ground-truth, not at much grain is required to mask the lack of samples/pixel.

For non-triangle rendering with cached texture space shading, getting to 2 shaded samples per pixel on average can yield quite nice results with only proper spatial filtering. This is important in that even with added temporal filtering, disoccluded areas only have capacity for spatial filtering.



On to Four Samples Per Pixel
#define MODE 6
Typical 4xSGSSAA (shading all of the 4xMSAA samples), then applying the default box filter.



#define MODE 7
Switching back to the jittered sampling at 4 samples/pixel, with Gaussian resolve and minor grain, very close to ground-truth live in motion.