Transformation Pipeline

Matrices are everywhere in the world of computer graphics, and it is not only because they help us describe transformations in the world, such as translations and rotations, but because they do so in an elegant (and composable!) fashion. Coming from the perspective of Linear Algebra, matrix multiplication of a vector is just a transformation of it from one space to another.

Did I lose you already? Sure, math can be complicated, but the way that compute graphics thinks about this makes it less like the math you're used to and more like a tool, a process... a pipeline.

Like everything in computing, the power of composition is what allows us to come up with ever more complex systems, as we clearly have in the realm of visuals. In particular, and by forming some categories we’ll describe ahead, a pipeline has been built of a sequence of matrix operations (i.e. their composition by multiplication) which makes the rendering process easy to think about while encapsulating major complexity.

The Transformation Pipeline

Just like in OOP everything is an object, and like everything is a function in functional programming, everything is a vector space in computer graphics.

Every rendered image on the screen start with a digital model (let’s say a three-dimensional one). It is nothing more than a set of vertices in space. Which space you ask? One we know as “Object Space”. In this space, picture the center of the object (conceptually) as being in the origin of it, and the vertices placed with respect to that.

Note that lengths don’t really mean much right now, only in relative terms, as we can just scale the model later on in this next step. By taking each object, abstractly defined, and placing it on the scene in the correct position, and with the correct scale and orientation, we have moved each of its vertices from "Object Space" to what is called “World Space”. We could have, in principle, many matrices to achieve this, first rotating the object, then performing a translation to its correct location and such, but through composition by multiplication we can determine a single matrix that sets the object as we want it in the world.

You've Got the Whole World in Your Screen

If you're keeping track, we have, for now, performed just a single transformation. For each of our objects (their coordinates in "Object Space") we transform them into "World Space", scaling, rotating and positioning them accordingly. We're still in three dimensions.

Now the camera enters the game. The camera (like a real one) is the way we capture the world into two dimensions. We could place the camera in World Space and then try to map the rays into the camera's lens. To make the process simpler, however, we'll continue with our "Space" philosophy, as follows. We will create a new space, called Camera Space. On it, we can think of the camera as being fixed on its origin, pointing along one of the axis.

Where are our object? Well, they're still on World Space. We will now bring them into Camera Space by placing them around the camera, placing them on a position relative to it as if we had put the camera in the World Space instead.

To make it easier on the math, however, we fix the camera on the origin, pointing usually oriented along the Z axis, and move everything else in the scene to correct their relative position to it. This sounds like extra work, sure, but it ends up being much simpler. Realize that the only thing we've done is an additional transformation, just like the last one (into World Space), by multiplying just another matrix.

And so the final and expected step is here and that is a transformation into two dimensional Screen Space, that will actually be on the screen. The details of this transformation are a whole thing on its own, but, as you might imagine, it sums up to just another space transformation, via matrix multiplication.

Conclusion

Through this architecture a well-structured system has emerged, where each part of it has a strictly defined responsibility. But, at the same time, every part is the same. By moving through different spaces, the vertices first put in place by the model's designer end up in the screen, conforming a two dimensional picture of the scene.

The fact that it all comes down to matrix multiplication is great for the hardware designers, as it allows GPUs to be hyper-specialized systems that multiply matrices for a living, and it allows developers to always know where to find what they’re looking for (by thinking of the different spaces) and to know where to make the changes required.