Future work
Although the final deadline for this project has passed there are still a couple of things left to be done. I have split them into two main categories: implementation, improving and extending on the existing code / ideas and algorithm, introducing new ideas that should give better results.
- Implementation
The existing render manager can be improved by extending it to support:
- Multiple lights – at the moment only one light is supported. Multiple lights can be added by using a 3D texture, because every light generates a maximum of up to 10 textures.
- Multiple mesh types – for now only genmesh is supported. Although this is the most generic type of mesh used in Crystal Space, support for materials from other types of meshes has to be added as well.
- Rendering smoke and clouds – this will involve adding a volumetric rendering technique and apply the algorithm from the current render manager without any further modifications.
- Algorithm
A couple of ideas can be added to produce better renderings or make the application run faster.
- Different split ratio for each ray – at the moment the split ratio is computed globally and this causes problems when objects with different densities are present in the scene. One possible way of having a unique split function for each ray is by creating a split texture, storing the split ratios for each individual ray onto a corresponding pixel.
- Recomputing the split ratio in real-time – this is the current implementation bottleneck, because it is done on the CPU. If the split ratio will be computed for each ray, then no global information will be needed and the computation can be done faster on the GPU by adding a new image processing render pass.
- Compute the optimal number of layers – the number of layers is currently chosen by the user. However, a test scene can be created so that the maximum number of layers that keeps the application real-time will be automatically chosen.
Hybrid split
As I mentioned in the previous post a hybrid split between the linear and the logarithmic split can be a good idea because when the linear splitting scheme falls short the logarithmic one can be used and the other way around.
Because when multiple layers contain the same information artifacts may occur, the criteria for choosing the ratio between linear and logarithmic splitting is so that it produces consecutive layers as different from each other as possible. Or put in another way each new layer should bring new information. In terms of computer vision this can be translated to having the mutual information between these images as small as possible.
Two techniques of measuring mutual information were tested: sum of absolute differences and cross-correlation coefficient.
The sum of absolute differences is pretty straight forward to compute and it involves adding the absolute value of the difference between each two corresponding pixels from the two images.
The cross-correlation coefficient represents the ratio between the covariance of two images and the product of their standard deviation, and can be computed using the following formula:
where Ī(•) is the mean of image I. Another useful property about correlation is that it has values on a scale ranging from [-1, 1] and it gives a linear indication of the similarity between images.
As expected from the findings in the previous post the mutual information is smaller when choosing a more linear split for sparse objects and a more logarithmic one for denser objects (Figure 1).
Figure 1 Plot generated using gnuplot. Linear splitting corresponds to a split ratio of 0 while logarithmic splitting maps to the value of 1.
Because, as we can see from Figure 1, the cross-correlation coefficient (shown in green) covers a wider range of values, giving better estimate for each density value, it was chosen as the default method of computing the mutual information. The cross-correlation probably performs better due to the influence of standard deviation, which is completely neglected for the sum of absolute differences (shown in red).
Splitting scheme
- Linear splitting
The most frequently used splitting scheme for choosing the opacity maps’ position is the linear one. It has been used as the primary splitting scheme in both opacity shadow maps (OSM) and deep opacity maps (DOM).
However, if we were to look at the light’s distribution on real-world translucent objects such as clouds, trees or hair we can observe that for dense objects the lighting caused by self-shadowing changes only at the very beginning of the object (Figure 1).
Figure 1 Real-world photographs of clouds (a) and bushes (b). It can be observed that for these objects the lighting only changes at the very beginning of the object.
In such cases a linear distribution would produce layers that contain almost the same information from a certain layer onwards (Figure 2). A distribution that would have more layers near the beginning of the object and fewer at the end would probably give better results.
Figure 2 Layers obtained using linear splitting on a scene with a dense model. The last four layers contain almost the same information.
- Logarithmic splitting
The logarithmic distribution has a slower increase rate and therefore produces a splitting that has a higher density of layers at the beginning of the object (Figure 3).
Figure 3 Comparison between linear and logarithmic distributions. Linear increase, blue, versus logarithmic increase, green (a), linear split (b) and logarithmic split (c).
Obtaining layers that have different shadow information prevents artifacts like the ones shown in Figure 4.
Figure 4 Difference in rendering when using linear splitting (a) and logarithmic splitting (b). Linear splitting (a) produces incorrect self-shadows because most of the layers contain the same information (Figure 2).
- Hybrid split
Although logarithmic splitting produces good results on dense objects, it doesn’t preform well on sparse objects because the lighting caused by self-shadowing changes throughout the entire length of the object (Figure 5).
Figure 5 Real-world photographs of clouds (a) and trees (b). It can be observed that for sparse objects the lighting changes throughout the entire length of the object.
The rendering artifacts that occur when logarithmic splitting is performed on sparse objects can be seen in Figure 6.
Figure 6 Difference in rendering when using logarithmic splitting (a) and linear splitting (b). Logarithmic splitting (a) produces artifacts: the willow is incorrectly lit near the top, because the layers don’t uniformly cover the whole length of the sparse object.
However, because the linear splitting scheme can be used when the logarithmic one fails and vice versa, using a hybrid split between the two of them based on the given scene should produce artifacts free renderings. More on this hybrid split in the next post.