20090705 - Odd Transparent Rendering Ideas


Transparent content tends to require a trade-off between overdraw and sharpness. The ideal solution in my mind is to able to have sharp features (high resolution) with low amounts of overdraw (more opaque) intermixed with smooth features (low resolution) and high amounts of overdraw (more transparent).

With 100% order-independent rendering, such as only additive transparency, one possible solution would be as follows,

(1.) Have a set of accumulation buffers at reducing resolutions.
(2.) Distribute draws to buffers based on sharpness and overdraw.
(3.) Combine all buffers.

Combine buffers requires repeated passes of up-sample and merge in order of the smallest buffer to the largest buffer. Note it might be a good idea to filter better in the up-sample passes to avoid the typical mip maximize artifacts. Also note these combine passes are going to be bandwidth bound, so its a good idea to pair with some ALU bound work in the same kernel if possible!


TANGENT ON BLEND MODES

Typical blend modes used in order-independent rendering are ADD and MUL. ADD to lighten, and MUL to darken. Often using both requires 2 passes, and when mixed with multiple resolutions, artifacts are found such as background MUL particles (say smoke) drawing over foreground ADD particles (say fire).

Another issue is gamma-correct (linear space) blending, are transparent surfaces combined in linear space or after the linear to gamma transform?

In the case of gamma space (non-linear) blending, I've never liked ADD blending at all for additive transparency. Perhaps this is because of years of photography where I go to complex lengths to avoid over-exposure. HDR in games is often (IMO) ridiculously overdone (really, should 25-50% of the frame be white). A visually better option (IMO) is Photoshop's SCREEN blend mode. SCREEN blending results in a soft approach to white, with blending limited at white. From a photography perspective, SCREEN works more like the way film responds to over-exposure. SCREEN is an inverted multiply,

d = 1 - ( (1-a) * (1-b) )

The inversion required by SCREEN can be factored out to doing MUL blending with an inverted framebuffer when attempting to port this over to DX or GL or GCM.

In the case of linear blending, at least in Photoshop, I've found MUL for darkening unacceptable with both 8-bit and 16-bit modes. To test this in Photoshop, one must actually build a linear color profile (which is quite easy). The problem is the loss of precision (banding, etc) when the MUL results in colors near black (even in 16-bit mode). Perhaps with 16-bit float framebuffer outside of Photoshop this isn't a problem.


BILLBOARDS AND SPHERICAL PARTICLES

Particle billboards suffer from the problem of hard intersection with solid geometry, and in the case of sorted billboards, pop when sort order changes.

Both of these problems have similar solutions, dynamically adjusting transparency across the particle to avoid the artifact. Spherical (or Z buffer aware) particles easily solves the first problem of intersecting with solid geometry. The second problem of pop on order change is more complex.


TRANSFORMING ORDER DEPENDENT TO ORDER INDEPENDENT?

This is a wild completely untested idea I haven't had time to try. Note the application of this would be for a non-triangle renderer which doesn't have standard opaque triangle surfaces. Application of this to a typical game engine is likely not worth it (requires two alphas, one for weight and one for transparency, or other complicated things)!

The idea is to solve the following problems in one go,

(1.) Removal of the order change pop on billboards.
(2.) Ability to easily work in multiple resolutions for performance.
(3.) Ability to do blend, lighten and darken.
(4.) Ability to do all this in one "pass".

The idea is similar in function to front first blending in which RGB accumulates color and A accumulates coverage. Except now RGB accumulates weighted color, and A accumulates weight. Actual RGB color is weighted RGB divided by weight A (assuming a FP16 buffer).

However, drawing would be order-independent, billboards rendered by computing a weight factor of visibility across the billboard's surface (or perhaps for smaller billboards, constant weight factor across the billboard's surface). This visibility factor would take into account both the billboard's transparency and expected occlusion by other billboards and surfaces. RGB would be billboard color times weight. Also for lighten and darken, weight stored in A could be adjusted.

Clearly this is adding another problem to solve (visibility estimation for weight factor). However I believe the weight factor could easily be computed at a lower effective frame rate and at a much lower resolution, possibly leading to better performance overall.