Views
What is lighting and why is it important
It might seem like a strangely obvious question as everyone should be familiar with the characteristics of light in the real-world and the way in which this is important to our daily survival. However if you take a few moments to contemplate what lighting really is and how its characteristics make it important then it becomes immediately obvious why simulating it as part of computer graphics is absolutely essential.
All materials in the universe emit, reflect or transmit electromagnetic radiation that can be recorded by wavelength. The human eye can only pick up a subset of the electromagnetic spectrum – specifically between 400 and 700 nanometre wavelengths. Fundamentally this section of the book will deal with modelling which wavelengths are reflected back and are visible to the eye, thus becoming the colours displayed on the computer’s display. Interesting effects can be attained by visualising wavelengths outside of this range (such as heat registering in the infrared wavelengths) but this section of the book focuses on the visible spectrum only. The way in which different materials reflect visible light energy can be subtle but extremely important. The human brain is exceptionally good at picking up these differences even if we’re not consciously aware of it. Computer graphics allows for many types of visual clues and if these don’t work in harmony or conflict with characteristics our brain is expecting it will greatly reduce the perceived realism of an image.
For example, if a cube model is given a wood texture yet exhibits plastic-like light reflection (typical of the simple Blinn-Phong model used by Direct3D’s old fixed-function pipeline) the perceived realism will be much lower than it could be. To generate good results a lighting model that captures the physical characteristics of the intended material is essential.
The ever increasing expectations of games players and users of graphics-oriented software combined with the importance of lighting and the possibilities available through Direct3D 10 requires an entire section of this book. By a similar measure it should also be clear that lighting models should be an important, core part of your graphics rendering process.
It is important to note that light, in the context of this section, is the presence of light energy and the algorithms are modelling its interactions with a given material. A key component of lighting is also its occlusion – a characteristic more generally known as shadowing which is covered in a later section of this book.
Outline for this section of the book
Both the theory and implementation of lighting in the context of computer graphics can get very complex, very quickly. With a little searching online it would not be difficult to find a research paper on the subject that has enough mathematical notations to convince most people that it is written in a completely foreign language.
The key emphasis with this section of the book is on practical and usable results. It is not possible to jump straight to the end-results without some coverage of theory, but it is possible to skip much of the mathematical derivation and focus on points of interest. By the end of this section you should have a good enough understanding of the concepts and theory to know how to use them effectively, yet also understand their implementations to build them into your own software with little (if any) further research.
Many of the concepts introduced whilst explaining lighting are fundamental to computer graphics; for those who are unsure or unfamiliar with these concepts [Akenine-Möller&Haines02] is highly recommended.
This first chapter will provide the basic knowledge required as a foundation to later chapters and give some insight into how the implementation of lighting models should be approached. The following two chapters set up the framework necessary to begin working with lighting models. The next six chapters cover specific lighting models implemented in the context of the framework previously discussed; this will provide a comprehensive suite of models for most practical graphics requirements. The final chapter in this section summarises all of the techniques covered in earlier chapters – as will be shown, much of the real-world usage comes down to a choice of appropriate models and combinations thereof and having a summary is important.
Prerequisite mathematics
Anyone familiar with computer graphics should be well aware that there is a heavy dependency on mathematics – the field of lighting is no exception. Luckily lighting makes use of many of the standard mathematical techniques used throughout computer graphics – vectors and matrices. Knowledge and experience of these from other areas of computer graphics should transfer across.
Vectors will primarily be in two or three dimensions and defined in either Cartesian or spherical coordinates. Basic operations such as addition, subtraction, scalar multiplication and normalization will be regularly used. In particular an understanding of how two vectors might compare with each other, such as the angles between them (dot, or inner product) and how basis vectors can be used to define a coordinate system (including cross-products). One aspect of vector mathematics that is heavily used in lighting is that of generating ‘face normals’. Given that the vertices making up a triangle are coplanar it is possible to generate a vector perpendicular to the face of the triangle. This vector describes the orientation of the triangle and is therefore very useful when determining whether it faces towards or away from a light source. However it is important to realise that triangle-based geometry is a convenience for computer graphics and can really be considered as an approximation to a much higher detail and probably continuous surface. This is supported by the way that most modelling packages as well as the Direct3D rendering API store normal vectors as per-vertex attributes. This higher resolution of normal vectors allows for a closer approximation to the true surface that is being represented. Consequently face normals are indirectly important and play more of a foundation role in support of the actual information used for rendering or storage.
Being able to generate and manipulate these vectors is important when it comes to the source data that is fed into the lighting equations; the remainder of this section will assume that this information is present where necessary – the exact methods aren’t directly covered.
Coordinate-space transformations are the main way that matrix mathematics gets involved with lighting computation; with Direct3D 10 being entirely shader-driven a good understanding of these types of transformations is essential. As will be shown in later chapters it is very common to be transforming data between coordinate spaces, more so than in other aspects of computer graphics. A common problem when implementing lighting in shaders is to compare two vectors that are in different coordinate systems – any expected relationship (such as the angle between them) becomes invalid and unexpected results are the dish of the day – these bugs can be very tricky to identify such that it puts extra emphasis on understanding the theory as well as the actual code.
If any of the terms in the previous paragraphs are alien to you, or if it’s been a while and you’re a bit rusty then it is highly recommended that you do a bit of extra reading to get yourself up to speed. Excellent texts like [Lengyel04] and [Farin&Hansford98] are recommended or for a free and online alternative [Mathworld] is a good choice.
What are lighting models?
Simulating realistic lighting in computer graphics is not a new thing and has been researched, discussed and implemented for decades. With the importance of lighting combined with this amount of time it should come as no surprise that there are a huge number of approaches to implementing lighting effects. Fundamentally there are two motivations when deriving simulations of the lighting contribution in a 3D scene – physical and observational.
Physically based approaches tend to be the most realistic as they attempt to simulate the real-world physics behind the interaction of light energy with surfaces and materials; they typically rely on the accepted principles and laws of physics.
Observational approaches start with an observation (“gold metal is shiny” for example) and attempt to derive a way to simulate the same results. Research papers covering observational approaches often include discussion regarding the methodology and instrumentation used to measure the observed effect and later on show how their method matches the real-world results.
Later chapters covering individual lighting models fall into both categories. Robert Cook and Kenneth Torrance’s model is heavily based on physical principles whereas Gregory Ward’s is based on observed characteristics. Whilst both models easily map to programmable GPU’s a key motivation behind empirical approaches is that they result in simpler equations which correspond to a simpler implementation and faster performance.
Global and local illumination
Lighting in computer graphics can be simplified as a two-phase algorithm. The first phase deals with transporting light energy from the source to the destination and the second phase deals with the results of this light energy interacting with a particular point on a surface. The two phases will overlap, but broadly speaking they can be considered separately.
The previously discussed physical and observationally motivated modelling describes the second phase. The computations at this stage are not interested in how the energy arrived at this particular point, all that matters is that some did and that reflectance should be computed. Consequently it would be possible to have a physically accurate transport model tied to an observationally motivated reflectance model (or vice versa). However the first phase is of considerable importance to the end result, specifically the complexity of its simulation defines the global or local illumination categories. It is quite conceivable that the light energy emitted from the source will be affected by interactions with the environment before it reaches the point where reflectance is to be calculated.
Global illumination is a class of algorithms that considers most (if not all) of the intermediary interactions before passing a quantity of incoming energy to a lighting model for the second phase. These steps are often referred to as “bounces” given that the global illumination algorithm will bounce light energy around the environment until a condition causes it to stop. How energy is absorbed or reflected at each bounce and the number of bounces used has a huge impact on the results of the final image. Various algorithms in this category attempt to get the maximum visual quality for the minimum amount of intermediary computation, but it is still common to find global illumination algorithms taking minutes or hours to generate a single image. For a reasonably complex scene rendered at a medium display resolution it is not hard to end up computing many millions of bounces.
At the other end of the spectrum is local illumination which vastly simplifies the transport of light from the source to the destination. Local illumination is implemented in terms of the relationship between the two regardless of the environment they both exist in. This cuts out a huge amount of work and makes the whole process of lighting a great deal faster and easier to implement – to the extent that the vast majority of real-time computer graphics will be using a local illumination algorithm.
The opening section of this chapter commented on shadowing being due to the occlusion of light energy. This is one of the most obvious errors introduced via a local illumination algorithm. By considering only the relationship between source and destination it can’t determine if there is any geometry blocking the energy being emitted by the light source; consequently there are no shadows in a pure local illumination system. By contrast, shadows are implicitly included in a global illumination system as the entire environment is considered whilst evaluating the energy received at the destination.
Diagram 1.1
Point X in the preceding diagram is an example of where global and local illumination will differ. On the left up to 3 bounces are shown for the energy transmitted between the source and destination whereas on the right only the relationship is considered and the surrounding geometry is completely ignored.
In most cases each bounce will reflect less energy such that point X still receives light energy despite appearing dark. A very important observation is that this energy cannot be expressed via a single vector (as is the case with local illumination) – instead all energy over the hemisphere surrounding the point in question must be considered. Crucially this allows for the many small (and, at an individual level, insignificant) pieces of energy to be factored into the final result. If it weren’t obvious, this scenario is demonstrating the previous point about shadowing. On the left the occluding geometry is factored in as part of the environment, whereas it is completely ignored for local illumination. Refer to the later section in this book that covers shadowing techniques to see how this problem can be remedied.
Global illumination is able to represent several other surface and environment characteristics that are very difficult to achieve via local illumination:
- *Semi-transparent surfaces* will allow light to pass through them. This opens up a lot of complex interactions – think about glass prisms that can split light into its component wavelengths and stained glass windows for examples.
- *Sub-Surface Scattering* is related to semi-transparent surfaces but adds an additional complexity in return for being able to represent a class of materials that local illumination would struggle with. Diagram 1.1 simplifies a surface as a depth-less boundary when in truth the material being represented may well have thickness (consider skin as an example). Depending on the composition of this surface the light energy may enter (in the same way as semi-transparent surfaces) but get reflected and scattered internally and exit at a different point on the same surface.
- *Surface bleeding* is similar to the stained glass window example used previously. It can be noticeable that energy reflected during one of the intermediary bounces will slightly alter the colour of the light. This altered colour will affect the contribution made at the destination. For example, a bright red object sitting on a flat white surface may result in a hint of red colour on the white surface due to the intermediary bounce of the red object onto the white surface “bleeding” its colour.
The above is not an exhaustive list, but should give a good idea of the types of effects that become possible with a more involved light transport algorithm. In conclusion, consider the following image:
Image 1.2
The “PRTDemo” sample provided as part of Microsoft’s DirectX SDK was used to generate image 1.2. Whilst these images are achieved via their Direct3D 9 API the comparison is still relevant for Direct3D 10.
The immediate observation is that a global illumination system generates softer and more natural looking results versus local illumination’s harsh and artificial appearance. Each intermediary computation for global illumination may have a seemingly insignificant contribution but the combined results of thousands or even millions of intermediary computations make a huge difference to the final image.
The savvy reader will notice that the previous images are from a real-time graphics sample which is counter to the previous comments about global illumination not having real-time performance characteristics. The following section will cover this crucial point.
Emphasis on dynamic lighting
For software making use of high-end graphics (especially computer games) the all important term is likely to be ‘interactive’. Over the years applications have had better and better physics simulations combined with increasingly life-like animations; very few people would disagree that this trend is likely to continue. Various hardware limitations have often forced applications to only make important parts of the environment interactive (a simple way to reduce the amount of work the physics simulation has to do) but as the hardware gets more powerful and more expressive this becomes less of an issue. It is now at a stage where almost every single object in the environment can be completely dynamic.
As has been highlighted, global illumination has high computation costs – but through techniques such as “light mapping” (aka “light baking”) it is possible to perform this expensive computation once and then re-use the results in a real-time environment. If any of the geometry or lights used to create these scenes change then some or all of the light maps must be discarded and recomputed; this is not a viable option and thus the application developers artificially force the environment to be completely static.
Techniques such as Pre-computed Radiance Transfer and Spherical Harmonics (see [Green03] for more details) do a good job of eliminating this characteristic and still generate results comparable to the best global illumination algorithms. However these techniques still require an amount of pre-processing and storage such that they still impose restrictions on some (or all) of the environment being static.
Forcing the environment to be static conflicts with the opening assertion that environments are increasingly dynamic but that should not rule the option out. In some situations parts of the environment may not need to be dynamic either for realism or for interactivity purposes (e.g. you don’t want a player destroying the entire game level). Equally it is possible that dynamic lighting may be overkill. Whilst it might technically be more accurate to be dynamic there may be such a subtle difference between dynamic and static that the extra computation involved in dynamic lighting simply isn’t worth the gain and pre-computed lighting becomes the winning candidate.
This idea of pre-processing part of the global illumination term and storing only the coefficients is a very powerful one. The general lighting model has a placeholder for this class of information – the ambient term. In common practice a constant ambient term is used and it defines the lowest level of lighting at any point in the image, but there is no requirement for this term to be constant. Per-object, per-material, per-vertex or even per-pixel ambient information is entirely possible – in most cases it is a trade-off of storage space against quality.
An important consideration is that the ambient term is not suited to high-frequency lighting. More, as the very definition of the word implies, it should be for the general lighting environment and not for specific details such as pixel-precise shadows or lights that have a sharp and noticeable shape or areas of influence. Direct3D 10 relaxes many of the shader-oriented constraints that existed in previous iterations of the API such that there is much more room for the graphics programmer to implement effective ambient terms should they be appropriate. The increased dimensions of constant buffers allows for more per-draw-call constants (suitable for per-scene, per-object or per-material ambient terms) and system generated values and load-from-buffer features are suitable for per-object (or instance of an object). Increasing the size of a vertex declaration can cause bandwidth problems but in theory the D3D10 API is very flexible in this regard such that appending a per-vertex ambient term is definitely possible.
When designing a hybrid system along these lines (part pre-computed, part dynamic) it is important to consider the interaction between different lighting algorithms. If the results are mismatched then the overall image may appear to be incorrect. Consider the case where the background uses a highly realistic global illumination approach yet the dynamic objects use an empirically derived approximation – it is conceivable that the dynamic objects will “float” over the background. This is purely an optical illusion as the observer’s brain spots the differences between the background and the foreground and finds it hard to merge them together as one consistent image. Unfortunately there are no easy solutions to matching different lighting models (even if they are of a similar algorithmic class) such that experimentation is crucial. Regardless of this, it is possible to mitigate this downside but it is important to consider it at design-time. The remainder of this section of the book will focus entirely on dynamic, local illumination lighting models. Whilst this eliminates an entire class of lighting solution it does tie in with the original assertion that environments are becoming increasingly dynamic.
It is also worth noting that the majority of the interesting (as well as extremely complex) global illumination algorithms are not shader oriented. Essentially most viable real-time global (and pseudo-global) illumination algorithms only use shaders to decode existing pre-computed results rather than doing any primary calculations in their own right.
Another key factor in this decision is that lighting is a field with a huge scope. It is possible to write not just one, but several books covering the topic of computer graphics lighting. Fitting all of this into a single section of a single book would simply not do the topic justice and would not cover it in anywhere near enough depth to be practically or theoretically useful.
For those who want to explore global illumination further [Dutré et al 03] and [Shirley&Morley03] are recommended.
BRDF’s and the rendering equation
The rendering equation is a simple and intuitive formula that underpins much of computer graphics – especially the field of lighting. It provides the basic framework for computing the colour contributed to the final image given an amount of incoming light energy and a function to describe how it is reflected:
The emitted term allows for the material in question emitting light energy without any input – it is a light source in its own right. Relatively few materials require the use of this term and in most implementations it will be left as a simple constant – for brevity it will be omitted in later sections. The previous equation is simplified down to a single light’s contribution. The full rendering equation needs some extensions:
For a given fragment the light from all possible sources (N) must be summed together using part of the previous equation. The ambient term is included as a simple constant to express the indirect lighting of a global illumination algorithm discussed previously in this chapter. Whilst both this and the emissive term are simplifications they can still have an important influence as well as being conveniently and easy to implement.
The Bidirectional Reflectance Distribution Function (BRDF) term is the key to this section of the book. In this context it is a placeholder for much more complex equations such as those covered in chapters 4 through 9 later in this section. Simply put, a BRDF models the way in which surfaces appear differently if we change the lighting or viewing direction.
Diagram 1.3
A BRDF is defined over a hemisphere bound by the plane created between the point being considered (a vertex or pixel for example) and its normal. The possible energy that could be received and reflected will always be a point on the surface of this hemisphere.
The traditional way of addressing points on this hemisphere is to use spherical coordinates – r, θ and Φ. Because the hemisphere is of unit dimensions we can omit r (which would be 1.0) and address the hemisphere simply in terms of θ and Φ. As will be shown in later chapters it can be convenient to replace spherical coordinates with Cartesian vectors (more commonly available in computer graphics).
A lot of materials will scatter much of the incoming light energy, but for the purposes of generating an image as the camera or eye would see it a single vector can be used – only light reflected along this direction will actually be recorded and contribute to the final image. An expensive part of global illumination renderers is evaluating the light scattered in other directions as, within a number of “bounces” (the process of incoming energy being reflected is sometimes referred to as a bounce) a small amount of the light energy may in fact reach the observer.
Whilst a lot of lighting models waiver physical accuracy in favour of visually acceptable results a true BRDF should hold to two fundamental properties:
- Bi-directionality: If the incoming and reflected directions are exactly swapped then the result should be the same.
- Conservation of energy.
The exact make-up of the BRDF is the subject of later chapters and varies from model to model but a common break-down is into diffuse and specular terms. These two concepts are fundamental to lighting models such that understanding their definition is essential.
Diffuse reflection can be considered as a special case of sub-surface scattering within the material; one where the light enters and leaves at the same point due to very small-scale internal scattering. It is also important to realise that diffuse reflection is view-independent which further simplifies its evaluation. If the surface layer generates a very strong specular reflection then little or no energy may penetrate the surface thus not all materials exhibit inner-scattering. Metals, for example, often demonstrate this characteristic.
Specular reflection occurs on the surface of the material and is view-dependent. This component of a BRDF is closer to the concept of reflected energy as, unlike diffuse reflection, it hasn’t been absorbed by the material before being reflected.
Most lighting models allow for these two terms to be controlled by coefficients that can be used to generate a better representation of the target material. For example, a snooker ball might have a red diffuse coefficient and a white specular coefficient.
Materials can either be homogenous or non-homogenous and for realistic results it is important to adhere to these. Breaking these constraints tends to break the conservation of energy (for large coefficients more energy will be reflected than was received!) and break the perceived realism of an image. A homogenous material is made up of a single constituent such that the specular and diffuse layers are the same. In this case the diffuse and specular coefficients must be linked together – any incoming energy not absorbed into inner-scattering will be reflected via the specular component. Specifically the specular and diffuse components should sum up to 1.0.
Alternatively a non-homogenous material will have separate constituent parts – a surface layer of varnish over wood for example. Technically this means that the lighting model is attempting to compute reflection for two different materials (the varnish and the wood) which can cause problems for realism. At the implementation level the specular and diffuse coefficients will be independent of each other.
In practical terms these two coefficients don’t always adhere to these rules due to artistic license. Whilst physically incorrect it may well be that a better lighting environment is created via higher (hence summing to greater than one) value coefficients.
Incoming light intensity is proportional to outgoing light intensity. Simply put, the more incoming light energy the larger the amount of reflected energy – for example: twice the incoming energy would generate twice the reflected energy.
Diagram 1.4
The final term, Cos(θ), is from Johann Heinrich Lambert’s cosine emission law – published almost 250 years ago. The preceding diagram shows why this is a crucial scalar of incoming energy. Considering a beam of light (from a torch for example) that is emitting energy equally within the bounds of the beam it is possible to measure how much energy is being transmitted and received per unit area. If the receiving geometry is facing the beam of light then the area receiving light energy will be an exact projection of the beam of light itself (shown at the top of diagram 1.4). If the receiving geometry were rotated, as shown on the bottom of diagram 1.4, then the projected shape of the beam changes – it gets larger. In the preceding diagram the receiver on the bottom is approximately 40% longer than the emitter.
The key observation is that the same amount of energy is now hitting a much larger surface area, thus the potential energy to be reflected back at any given point is less than it was before. Lamberts law for illumination models this characteristic by taking the cosine of the angle which gives results between -1 and +1 and scales accordingly. When the rays of light are perpendicular to the surface the result will be +1 (full energy) and when parallel the result will be 0 (no energy). It is possible for the value to be negative when the surface actually faces away from the light – which is when the surface should be in shadow. It doesn’t make much sense to allow negative values as this generates negative energy being reflected so a simple solution is to clamp the term to the range 0 to +1. This work-around can be very useful when implementing lighting shaders as it allows for dynamic branching to skip any lighting calculations for surfaces facing away from the light source.
The Fresnel Term
Several lighting models discussed later in this section utilize the ‘Fresnel term’ to determine how much of the incoming light energy is reflected. The exact usage will be discussed in the appropriate sections, but due to it being a common term across several models it warrants discussion here.
Diagram 1.5
The preceding diagram shows examples of light reflection from three different incoming angles. At a shallow angle the majority of the light energy will be reflected away from the surface, moderate angles will reflect and refract similar amounts of light energy and steep angles will reflect very little.
Materials have an ‘index of refraction’ associated with them that, along with the appropriate angular measurements, can be used to determine the ratio of reflected to refracted energy. As previously stated the lighting models discussed in this section focus on the reflected light energy such that the Fresnel term becomes a very important factor.
Sources of refraction indices are not always easy to find and is further complicated by the fact that it is wavelength dependent. The majority of visible light is a combination of multiple wavelengths – a fact reflected by the traditional red (650nm), green (550nm) and blue (440nm) representation of colours in computer graphics. Rich visual results can be achieved by evaluating the Fresnel term using a separate index of refraction for each colour channel’s wavelength, but this is often overkill for real-time graphics. It can be easier to determine a dominant wavelength and have a single index of refraction – conversion from the RGB colour space to HSV or HSL yields the Hue which can be mapped to the physical wavelengths of the visible spectrum with good accuracy.
However there is still the problem due to indices of refraction being looked-up based on wavelengths and known material types. Theoretical computation is difficult if not impossible such that real-world results in tabulated form become a key source of information. This, for obvious reasons, makes an entirely dynamic system difficult to implement.
Due to this complexity there isn’t really a single correct way of providing input data into Fresnel-based effects. With a substantial amount of work it is possible to generate very accurate results, but with much less work it is still possible to generate plausible images. Dominant wavelengths and looking-up against a small set of significant materials is perfectly valid as is simple empirical testing (most indices of refraction will be between 1.0 and 5.0).
Where and when to compute lighting models
Direct3D 10’s biggest departure from its predecessors is that it dropped the “fixed function” pipeline – legacy functionality from before the programmable pipeline was available. The last version of the API to exclusively use the fixed-function pipeline was 7 (released in 1999), since then there has been a gradual move towards the richer and more powerful programmable approach.
Given that Direct3D 10 is entirely shader based it might seem compulsory to implement all parts of a rendering effect as a shader. Technically this might be the case, but it doesn’t cover every possibility. There is no specific reason why the shader must perform each and every complex operation – it could simply look up the result from a pre-created table. The resource model in Direct3D 10 is much more flexible than in previous versions such that it is possible to access results computed by the application in several ways and across any stage of the pipeline.
It is often necessary to fine-tune graphics algorithms in order to get them to fit the desired performance profile; shaving off a few instructions in a pixel shader might give a noticeable speed increase when multiplied by the thousands or millions of times it is executed. There are fundamentally two types of performance cost when looking at shaders – arithmetic or sampling. Arithmetic instructions are those that work through the equations computing intermediary and eventually final results, sampling is where data must be pulled into the shader processor from video memory and is therefore a heavy consumer of memory bandwidth. The balance between these two categories can get incredibly complicated (especially given that GPU’s are massively parallel in nature) and is well beyond the scope of this book. The reason this is brought up is that it offers lighting models a neat possibility for optimization.
As will be shown throughout the following chapters the lighting models can be very heavy on arithmetic and often outnumber sampling operations by an order of magnitude (although it is worth noting that some GPU architectures are optimal at around 10:1 ratios of arithmetic to sampling). This doesn’t tend to work in the favour of a balanced processing load at the hardware level and can mean that parts of the GPU will be extremely busy (and hence become a bottleneck) whereas other parts will be idle.
Factorization can counter this characteristic. By looking at the arithmetic equations it is sometimes possible to extract common terms with only one or two well defined ranges of input. These common terms could easily be computed by the application, stored in a 1D or 2D texture and uploaded to the GPU. The shader need only calculate the input(s) and read the appropriate value rather than compute it manually every single time. If this common term being factored out is expensive to compute then a one-time computation by the application is much more compelling than repeating the same computation thousands or millions of times for every image rendered.
Examples of factorization are included throughout the later chapters in this section, but in general it is always worth considering three questions:
- What range of inputs is needed? Textures are addressed with 0.0-1.0 coordinates, buffers with integer offsets, the more complex it is to map an input to these constraints the more work a shader must do before it can fetch an answer.
- What range of output(s) is needed? If only a small range of answers need to be encoded then information theory (such as concepts introduced by Claude E. Shannon) can be used to determine the minimum number of bits required to express the required values.
- What is the required resolution of the output? For a slowly changing sequence of answers accuracy to 1 decimal place might be sufficient, conversely a rapidly (or even randomly) changing sequence of answers might required much high resolution.
By considering these questions it is easier to make sensible choices about the resource size (obviously affecting storage requirements but also impacting on memory bandwidth usage) and storage format (an 8bit integer format requires less storage and bandwidth than a 32bit floating point format).
Complex arithmetic like raising numbers to a power and trigonometric functions are prime candidates for this approach. However, due to the aforementioned balances involved this form of factorization should be used with care. It is possible to attain worse performance due to shifting too much work over to the sampling units and thus leaving arithmetic capability dormant – profiling and measurement are essential in working out where this balance lies.
Image 1.6
The above image utilizes a lighting model demonstrated in the later chapter on Michael Oren’s and Shree Nayar’s lighting model. A simplified model presented as part of their paper can be easily factored to a 2D look-up texture, and the above image shows the comparison between a purely arithmetic approach (top-right) and differing resolutions of look-up texture down the left-hand side. The black and white images are a difference operator with a 1% error threshold – any white pixels differ from the pure-arithmetic reference image by at least 1%. As the resolution of the look-up texture increases the error decreases, as summarised in the following table:
Graph 1.7
This sort of analysis is not perfect, but it can give reasonably accurate guidance as to an ideal size for a look-up texture. It is immediately clear that the jump from 128x128 (32kb) to 512x512 (512kb) requires substantially more storage for only a 0.7% improvement!
Another important aspect is that of the performance versus quality trade-off. In general a higher quality technique tends to require more work and consequently ends up being slower to execute. The question therefore is whether the decrease in performance is worth the increase in quality.
An interesting avenue to explore is the use of ‘mix-and-match’ terms; as will be shown in later chapters the majority of models break down into both a diffuse and specular term. Ideally these should be matched, but there is no requirement for this to be the case. When performance is important it may be useful to swap an expensive term for a simpler and cheaper evaluation until the object being rendered becomes more influential in the final image.
Extending this leads to Level Of Detail (LOD) lighting algorithms. Conventionally LOD algorithms are employed in geometric contexts such as reducing the number of triangles in a mesh, but it is perfectly possible to apply the same concepts to lighting. By using shader model 4’s dynamic branching it is possible to select an evaluation based on a suitable metric – distance into the scene or mip-map level for example. With simple interpolation it becomes possible to smoothly transition between an expensive and high-quality evaluation where it matters and a simple low-quality evaluation where it may not be so noticeable.
Single or multi-pass rendering
A ‘pass’ in Direct3D terminology describes a specific set of data’s journey through the entire graphics pipeline. Direct3D 10 blurs this slightly with the stream-out technology that effectively allows for dual-pass processing in what appears to be a single pass from the application’s perspective. In the context of lighting models a pass would be the necessary steps to configure the pipeline for a particular lighting environment and then sending an object to be rendered with the result being the final, correctly lit, image.
As described previously light is simply visible energy and a key observation is that more lights mean more energy – an additive operation. This plays perfectly into the hands of a multi-pass rendering architecture and allows for very simple application code. By configuring the Output Merger pipeline stage for additive blending using the following code the value in the frame-buffer will effectively accumulate all energy contributed to each pixel:
D3D10_BLEND_DESC omState; ZeroMemory( &omState, sizeof( D3D10_BLEND_DESC ) ); omState.AlphaToCoverageEnable = FALSE; omState.BlendEnable[0] = TRUE; omState.RenderTargetWriteMask[0] = D3D10_COLOR_WRITE_ENABLE_ALL; omState.SrcBlend = D3D10_BLEND_SRC_COLOR; omState.DestBlend = D3D10_BLEND_DEST_COLOR; omState.BlendOp = D3D10_BLEND_OP_ADD; omState.SrcBlendAlpha = D3D10_BLEND_ONE; omState.DestBlendAlpha = D3D10_BLEND_ONE; omState.BlendOpAlpha = D3D10_BLEND_OP_ADD; ID3D10BlendState *pAdditiveBlending = NULL; g_pd3dDevice->CreateBlendState( &omState, &pAdditiveBlending ); FLOAT factors[] = { 0.0f }; g_pd3dDevice->OMSetBlendState( pAdditiveBlending, factors, 0xFFFFFFFF );
The potential problem with multiple light sources is that values can easily go beyond the displayable 0.0 to 1.0 range and it becomes necessary to utilize High Dynamic Range Rendering (see later chapters on HDRI) to generate a plausible final image.
Whilst it is logical and simple to take the multi-pass additive approach it is worth realising that there is a substantial overhead involved. Each pass will have to duplicate many calculations performed as part of a previous pass – even if it’s just transforming the same vertices from model to projection space. There’s also the potential for eliminating parallelism between the CPU and GPU and thus severely hurting performance – ideally the GPU will work as independently as possible.
Previous shader models available as part of Direct3D 9 were not particularly good at dynamic flow control such as that required by looping constructs (“for” and “while” constructs). Shader model 4.0 is much better equipped for these constructs although the actual performance is still very dependent on the underlying hardware. It does allow for having multiple lights in a single pass and therefore improving parallelism (the GPU can do more work before requiring the CPU to intervene) and reducing overhead by not repeating any unnecessary processing.
This gives rise to three possible approaches to implementing multiple lights:
| Approach | Characteristics |
|---|---|
| One light per pass, multiple passes | Simple to implement, high overhead and potential loss of parallelism |
| Multiple lights, single pass | Either implemented via looping constructs (simple to implement but potentially sub-optimal performance) or by writing the code once for an arbitrary number of lights and compiling all possible permutations (complex application-side management but optimal performance) |
It is worth noting that a third category, “multiple lights and multiple passes,” was favoured by some. This was born of the more restrictive limits imposed by earlier shader models. It may not have been possible to encode more than a few lights in a single pass, therefore 20 active lights might have been 4 lights per pass and 5 passes. Direct3D 10 relaxes the constraints such that this scenario shouldn’t be required anymore – but it is always worth considering should this part of the lighting engine prove to be a bottleneck.
There is an alternative approach to rendering multiple lights with a very favourable performance profile – deferred shading. It is a substantial departure from forward rendering (the easiest approach and the one used throughout this section of the book) and requires a different architecture completely. As the name suggests the actual shading of pixels in the final image is deferred until the last possible moment. The advantage is that any expensive shading operations are only performed for pixels that contribute to the final image and thus wastes no time computing unnecessary results. However, the initial overhead for deferred shading is high and any performance gains aren’t seen until a substantial workload (such as a large number of lights or a scene with a lot of overdraw) is being rendered. [Calver03] contains a good overview of how deferred shading works, its strengths and its weaknesses.
Sample Code
Each of the subsequent chapters includes demonstration code for reference. This sample code can be used for as little as an interactive demo or it can provide code for you to use and abuse in your own projects.
Graphics samples require example models to demonstrate their algorithms and features with, for this section the Brown mesh set is used [Brown] which provides data in a simple “indexed face set” (IFS) format. Code is provided in the samples for loading this data into a format that Direct3D 10 can use.
References
[Akenine-Möller&Haines02] Real-Time Rendering Second Edition by Tomas Akenine-Möller and Eric Haines. ISBN 1-56881-182-9
[Lengyel04] Mathematics for 3D Game Programming & Computer Graphics 2nd Edition by Eric Lengyel. ISBN 1-58450-277-0
[Farin&Hansford98] The Geometry Toolbox For Graphics And Modelling by Gerald Farin and Dianne Hansford, ISBN 1-56881-074-1
[Mathworld] http://mathworld.wolfram.com
[Green03] Spherical Harmonic Lighting: The Gritty Details, Robin Green, Sony Computer Entertainment America – presented at GDC 2003.
[Calver03] Photo-realistic Deferred Lighting, Beyond3D.com, Dean Calver 31st July 2003. http://www.beyond3d.com/articles/deflight/
[Dutré et al 03] Advanced Global Illumination by Philip Dutre, Philippe Bekaert and Kavita Bala. ISBN 1-56881-177-2
[Shirley&Morley03] Realistic Ray Tracing (second edition) by Peter Shirley and R. Keith Morley, ISBN 1-56881-198-5
[Brown] The Brown Mesh Set, created by Morgan McGuire and Nick Musurca of Brown University. http://graphics.cs.brown.edu/games/brown-mesh-set/index.html