VFX Back to Basics Series: 8. What is compositing?

This is part of a series on the basic elements of Visual Effects. Each post will talk about a certain element which is one of the basic bricks used for building VFX shots.

In this eighth and final post in the Back to Basics Series I will talk about compositing.

A CGI image with a split trough the middle which shows color channels in the top part and the alpha channel in the bottom part. Alpha channels are the key to compositing.

Where all the previous posts talked about generating elements, this one will talk about combining these elements into a final image. For people who don't know the word compositing I always compare it to "Photoshop with moving images". We layer up different images and combine them into one with the illusion that the final image looks as if everything was filmed at the same time.

Back in the old days

Although digital compositing is a very powerful tool in filmmaking, compositing itself did start out as an analog process with the use of the optical printer. The first optical printers were created in the 1920's and were improved up till the 1980's. At first the effects were quite simple like a fade in or fade out but the effects became increasingly complex with the addition of things like matte paintings and blue screen effects. All these elements were combined by exposing the film several times.

This also explains the necessity of using mattes. If the film would be exposed twice it would create a double exposure unless the area where the second element needs to come is masked out. For blue or green screen shots these mattes were made with high contrast film and the use of a color filter to separate out the background color. In some other cases mattes were painted by hand which could result in chattering edges.

Films like 2001: A Space Odyssey and Star Wars had a staggering amount of success with this optical workflow.

The digital era

At the end of the 1980's digital compositing started to take over. It has two major improvements over its optical counterpart:

• In the optical process it is necessary to create the several layers like the mattes by copying it from one film strip to another. Each copy degrades the image and adds extra noise to the result. In the digital realm a copy is 100% the same as the original (unless you resample or recompress them but that is usually not done).
• The second problem is that film going trough a machine can drift a bit which could result in chattering mattes or halos. If done correctly digital compositing does never suffer from this.

When we need to composite for film a digital intermediate is created by scanning the film. When shot on digital cameras the source is already digital and does not need scanning.

Popular compositing packages today are Nuke, Digital Fusion and After Effects. The first two are node based packages while After Effects is a layer based package and works in a similar way as Photoshop does. Node based systems are represented by a flow chart where every node applies an operator to tree. Life action and CGI compositing do benefit more from this method while motion graphics are usually done in layer based packages.

A Nuke node based network.

Layered based compositing in After Effects.

Color and Alpha channels

Let's have a look at how compositing works in the digital realm. A digital image consists out of pixels which are represented by a combination of the three primary colors red, green and blue. This combination can create a broad amount of other colors available in the visible spectrum. Every color is stored in a channel.

In order to combine different images, it is necessary to have a fourth channel which is called the alpha or matte channel. It contains the transparency information of each pixel and this dictates how the result looks like when one image is put atop another image. It works just like its analog counterpart.

There are two ways to show an image with an alpha channel. The premultiplied one where the value of the alpha channel is already multiplied by the color values or the unpremultiplied one where the color values have their original values. This is very important to understand as failing to grasp this concept will possibly make composites look horrible with weird matte lines as a result. Check what your software package expects you to work with.

CGI images are usually premultiplied automatically when rendered.

A magnified CGI image. Look at the anti-aliased edges. This image is premultiplied.

The corresponding alpha channel image of the above image which has the same anti-aliased edges.

When unpremultiplied the edges become aliased.

The Extra's

Composting packages do not only put one image over another but have a vast set of tools available:

• Rotoscope tools: With these it is possible to create mattes to isolate parts of the image. These include also operations to modify edges or to paint over existing imagery.
• Retiming tools: All kinds of operators to manipulate the speed of the sequence.
• Color correction tools: Operators to analyze and change the color of the image. Very useful to match foreground and background elements.
• Filters: All kind of filters like blur, motion blur, noise, denoise and many more.
• Keying tools: These are useful for matte extraction from images with blue or green screen but also other methods are available like luma keying.
• Layering tools: from simple operators to put one image over another to operators for premultiplication.
• Transformation tools: Operators such as translate, rotate, scale, resize and distortions.
• Tracking tools: Tools to track objects in a scene or to stabilize a shot.

Although compositing is a 2D process, nowadays most packages support 3D environments. This adds extra functionality as it is possible to have proper parallax, use projections and makes the integration of particle systems really useful.

Another rather recent development is stereo conversion tools. It allows regular images to be converted to stereographic images. Note that I use the term stereographic instead of more common and commercial term 3D. Stereographic images are not truly 3D as the audience can not look behind objects by moving their head. I therefor prefer the term stereographic.

This concludes the eighth part of the VFX Back to Basics series.

Make sure to subscribe (at the top) or follow me on Twitter (check the link on the right) if you want to stay informed on the release of new posts.

In this series:

1. What is rotoscoping?

2. What is 3D modeling?

3. What is tracking and match moving?

4. What is 3D animation?

5. What is shading and texturing?

6. What is lighting and rendering?
7. What is matte painting?