Rendering an Image of a 3D Scene

Distributed under the terms of the CC BY-NC-ND 4.0 License.

  1. It All Starts with a Computer and a Computer Screen
  2. And It Follows with a 3D Scene
  3. An Overview of the Rendering Process: Visibility and Shading
  4. Perspective Projection
  5. The Visibility Problem
  6. A Light Simulator
  7. Light Transport
  8. Shading
  9. Summary and Other Considerations About Rendering


Reading time: 17 mins.

While everything in the real world results from light interacting with matter, some interactions are too complex to simulate using the light transport approach. This is when shading becomes essential.

Figure 1: This image shows two objects under the same lighting conditions appearing to have the same color (hue), yet one is darker than the other. This difference in brightness is not due to the amount of light falling on the objects but rather how much light each object reflects into its environment.

As mentioned in the previous chapter, simulating the appearance of an object requires computing the color and brightness of each point on the object's surface. Color and brightness are closely related. It's crucial to distinguish between the brightness caused by the amount of light falling on an object's surface and the brightness of an object's color (sometimes referred to as the color's luminance). The brightness of color, along with its hue and saturation, is a property of color itself. If you observe two objects under identical lighting conditions and they seem to have the same color (same chromaticity), but one is darker than the other, it indicates that their brightness is not solely a result of the light they receive. Instead, it depends more on how much light each object reflects into their environment. In essence, these two objects have the same color (chromaticity) but differ in how much light they reflect (or absorb). Therefore, the brightness (or luminance) of their colors varies. In computer graphics, this characteristic color of an object is termed albedo. Objects' albedo can be measured precisely.

Note that an object cannot reflect more light than it receives (unless it emits light, as light sources do). The color of an object, particularly for diffuse surfaces, can generally be calculated as the ratio of reflected light to the amount of incoming (white) light. Since an object cannot reflect more light than it receives, this ratio is always less than 1. Consequently, colors of objects are defined in the RGB system as values between 0 and 1 for floating-point encoding, or 0 and 255 if using a byte. It is more practical to express this ratio as a percentage. For example, if the ratio—also referred to as color or albedo (these terms are interchangeable)—is 0.18, the object reflects 18% of the incoming light back into the environment. Learn more about this topic in the lesson on Colors.

Defining the color of an object as the ratio of the amount of reflected light over the amount of light incident on the surface, as explained previously, means that color cannot exceed one. However, this does not imply that the quantities of incident and reflected light off the surface of an object cannot be greater than one; it is only the ratio between them that cannot exceed one. What we perceive with our eyes is the product of the amount of light incident on a surface and the object's color. For example, if the energy of the light impinging upon the surface is 1000, and the color of the object is 0.5, then the amount of light reflected by the surface to the eye is 500. This simplification is technically incorrect from a physics perspective, but it serves to illustrate the concept. In the lesson on shading and light transport, we will delve into what these values of 1000 or 500 mean in physical units and understand that the interaction is more complex than merely multiplying the number of photons by 0.5 or the albedo of the object.

Thus, assuming we know the color (albedo) of an object, to compute the actual brightness of a point \(P\) on the surface of that object under certain lighting conditions (referring to the actual amount of light energy reflected by the surface to the eye, not the brightness or luminance of the object's albedo), we need to consider two factors:

It is important to remember that for specular surfaces, the amount of light reflected depends on the viewing angle. Moving around a mirror, for instance, changes the image reflected; the amount of light reflected towards you varies with the viewpoint.

Figure 2: To compute the actual brightness of a point \(P\) on the surface under certain lighting conditions, we must consider how much light falls on the object at this point and how much light is reflected in the viewing direction. Calculating the light arriving at \(P\) involves summing up contributions from light sources (direct lighting) and other surfaces (indirect lighting).

Knowing an object's color (its albedo), we must determine how much light arrives at point \(P\) and how much light is reflected in the viewing direction, from \(P\) to the eye.

First, it's essential to understand that light reflected into the environment by a surface results from complex interactions between light rays (or photons, for those familiar with the term) and the material comprising the object. There are three critical points to note:

Figure 3: For mirror surfaces, if the view direction and the reflected ray direction are not the same, the reflected ray of light is not visible. The amount of light reflected is a function of both the incoming light direction and the viewing direction.

Let's summarize what we know:

  • Simulating light-matter interactions (happening at the microscopic and atomic levels) is too complex, necessitating alternative solutions.

  • The amount of light reflected from a point varies with the viewing direction \(\omega_v\).

  • For a given viewing direction \(\omega_v\), the amount of light reflected depends on the incoming light direction \(\omega_i\).

Shading, which is responsible for computing the amount of light reflected from surfaces to the eye (or other surfaces in the scene), depends on at least two variables: the incident light direction \(\omega_i\) and the outgoing or viewing direction \(\omega_v\). The origin of light is independent of the surface, but the amount of light reflected in a given direction varies with the surface type (diffuse or specular). As previously mentioned, gathering light at the incident point is more a light transport problem. Regardless of the technique used to gather light at point P, it is crucial to determine the light's origin, i.e., from which direction it comes. A shader takes the incident light direction and the viewing direction as input variables and returns the fraction of light reflected by the surface for these directions.

$$\text{ratio of reflected light} = \text{shader}(\omega_i, \omega_o)$$

Simulating light-matter interactions to obtain a result is complex, but the outcome of these numerous interactions is predictable and consistent, allowing for approximation or modeling with mathematical functions. We will explore the origins, nature, and discovery of these functions in lessons devoted to shading. For now, let's intuit how and why this works.

The law of reflection, introduced in a previous chapter, can be expressed as:

$$\omega_r = \omega_i - 2(N \cdot \omega_i) N$$

In plain English, this equation means that the reflection direction \(\omega_r\) can be calculated as \(\omega_i\) minus two times the dot product of N (the surface normal at the point of incidence) and \(\omega_i\), multiplied by N. This formula relates more to computing a direction than the amount of light reflected. However, if for any given incident direction (\(\omega_i\)), \(\omega_r\) coincides with \(\omega_v\) (the viewing direction), then the ratio of reflected light for this particular configuration is 1 (Figure 3 - top). If \(\omega_r\) and \(\omega_v\) differ, the amount of reflected light would be 0. To formalize this concept, you can write:

$$\text{ratio of reflected light} = \begin{cases} 1 & \text{if } \omega_r = \omega_o \\ 0 & \text{otherwise} \end{cases}$$

This is just an example. For perfectly mirror-like surfaces, we never proceed in this way. The point here is to understand that if we can describe the behavior of light with equations, then we can compute how much light is reflected for any given set of incident and outgoing directions without having to run a complex and time-consuming simulation. This is precisely what shaders do: they replace complex light-matter interactions with a mathematical model, which is fast to compute. These models are not always very accurate or physically plausible, as we will soon see, but they are the most practical way of approximating the results of these interactions. Research in the field of shading primarily involves developing new mathematical models that match as closely as possible the way materials reflect light. As you may imagine, this is a challenging task: it's difficult on its own, but more importantly, materials exhibit very different behaviors, making it generally impossible to simulate accurately all materials with a single model. Instead, it is often necessary to develop one model that works, for example, to simulate the appearance of cotton, another to simulate the appearance of silk, etc.

What about simulating the appearance of a diffuse surface? For diffuse surfaces, we know that light is reflected equally in all directions. The amount of light reflected towards the eye is thus the total amount of light arriving at the surface (at any given point) multiplied by the surface color (the fraction of the total amount of incident light the surface reflects back into the environment), divided by some normalization factor that is necessary for mathematical/physical accuracy reasons. This will be explained in detail in the lessons devoted to shading. Note that for diffuse reflections, the incoming and outgoing light directions do not influence the amount of reflected light. However, this is an exception. For most materials, the amount of reflected light depends on \(\omega_i\) and \(\omega_v\).

The behavior of glossy surfaces is the most difficult to reproduce with equations. Many solutions have been proposed, the simplest (and easiest to implement in code) being the Phong specular model, which you may have heard about.

The Phong model computes the perfect mirror direction using the equation for the law of reflection, which depends on the surface normal and the incident light direction. It then computes the deviation (or difference) raised to some exponent, between the actual view direction and the mirror direction (by taking the dot product between these two vectors) and assumes that the brightness of the surface at the point of incidence is inversely proportional to this difference. The smaller the difference, the shinier the surface. The exponent parameter helps control the spread of the specular reflection (check the lessons in the Shading section to learn more about the Phong model).

However, good models follow some well-known properties that the Phong model lacks. One of these rules, for instance, is that the model conserves energy. The amount of light reflected in all directions shouldn't be greater than the total amount of incident light. If a model doesn't have this property (the Phong model lacks this property), then it would violate the laws of physics. While it might provide a visually pleasing result, it would not produce a physically plausible one.

Have you already heard the term physically plausible rendering? It designates a rendering system designed around the idea that all shaders and light transport models comply with the laws of physics. In the early days of computer graphics, speed and memory were more important than accuracy, and a model was often considered to be good if it was fast and had a low memory footprint (at the expense of accuracy). But in our quest for photorealism, and because computers are now faster than they were when the first shading models were designed, we don't trade accuracy for speed anymore and use physically-based models wherever possible (even if they are slower than non-physically based models). The conservation of energy is one of the most important properties of a physically-based model. Existing physically based rendering engines can produce images of great realism.

Let's put these ideas together with some pseudo-code:

Vec3f myShader(Vec3f Wi, Vec3f Wo) 
    // Define the object's color, roughness, etc.
    // Do some mathematics to compute the ratio of light reflected
    // by the surface for this pair of directions (incident and outgoing).
    return ratio; 
Vec3f shadeP(Vec3f ViewDirection, Vec3f Point, Vec3f SurfaceNormal) 
    Vec3f totalAmountReflected = 0; 
    for (all light sources above P [direct|indirect]) { 
        totalAmountReflected += 
            lightEnergy * 
            shaderDiffuse(LightDirection, ViewDirection) *
            dotProduct(SurfaceNormal, LightDirection); 
    return totalAmountReflected; 

Notice how the code is divided into two main sections: a routine (line 11) to gather all light coming from directions above point P, and another routine (line 1), the shader, which computes the fraction of light reflected by the surface for any given pair of incident and viewing directions. The loop, often referred to as a light loop, sums up the contributions from all possible light sources in the scene to the illumination of P. Each light source possesses a certain energy and originates from a specific direction (defined by the line between P and the light source's position in space). All that's needed is to forward this information, along with the viewing direction, to the shader. The shader then calculates the fraction of light from that specific direction that is reflected towards the eye, and this result is multiplied by the amount of light produced by the light source. Summing these results for all possible light sources in the scene yields the total amount of light reflected from P towards the eye, which is the desired outcome.

Note that in the sum (line 18), there is a third term: a dot product between the normal and the light direction. This term is crucial in shading and is related to the cosine law. It will be elaborated on in sections about Light Transport and Shading. For now, it's important to understand that it accounts for the distribution of light energy across the surface of an object as the angle between the surface and the light source changes.


A fine line exists between light transport and shading. As we delve into the section on Light Transport, it becomes clear that light transport algorithms often depend on shaders to determine the direction in which to spawn secondary rays for computing indirect lighting.

Two key takeaways from this chapter include the definitions of shading and shaders:

Historically, techniques for rendering 3D scenes in real-time were largely predefined by the API, offering little flexibility for users to modify them. Real-time technologies have evolved beyond this paradigm to provide a more programmable pipeline, where each step of the rendering process is controlled by separate "programs" called "shaders." The current OpenGL APIs now support four types of shaders: vertex, geometry, tessellation, and fragment shaders. The shader responsible for computing the color of a point in the image is the fragment shader, while the other shaders play minor roles in defining the object's appearance. It's important to note that the term "shader" has thus come to be used in a broader sense.