Download the completed project file and sample media for this tutorial.
Prerequisites:
Using Masks and Luma Keys to create a layered composition.
Read the tutorial on Layer Composition which covers opacity and blend modes.
Read the tutorial on Adding an FX to a layer.
Read the tutorial on Applying a Mask to a layer.
One of the most powerful techniques for combining multiple layers of video into an output is the use of masking, also known as luma keying. With this process, two video sources are combined to create a "cut out" layer that can be composited over others image like a collage instead of simply blending them together. This style is commonly found in music videos, graphic design, and by VJs for live performance. Masking is also an important technique used when projection mapping video onto surfaces.
The basic idea behind this process is easy to learn and can be repeated any number of times within your VDMX project to create complex layered visual scenes.
Typically for our Mask we'll want to use a video source that has clearly defined shapes or patterns, preferably black and white or high-contrast. A few examples of good masks, some that we'll use for this example:
When applying the mask to a layer using the Layer Mask FX, the regions that are white will become textured by the source video, and the regions that are black will show through to video playing on the background layers behind it. Areas that are gray will mix between the two.
Foreground Layer before mask is applied.
Masking Layer provides the "cut out" shape.
Foreground Layer after the mask is applied.
Composited over a background.
When creating layered compositions using luma keying, a useful tip is to organize media files into three types: overlays, masks and backgrounds. The masks will provide the "cut out" shapes that appear over the background clips and the overlays will be the texture video that fills our shapes. In this video tutorial we'll begin by loading our files onto separate pages for these purposes.
After the media files are loaded we'll move to the Workspace Inspector in the Layers tab where the layers from our scene can be managed. For the basic example we'll create three layers: a Foreground, Background, and Mask.
As detailed in the applying a mask to a layer tutorial, once this basic layout is prepared we need to adjust two layer settings and add an FX. First, in the Layer Composition Controls for the Mask Layer, set the Opacity to 0.0 (or use the hide/show button) so that the mask is not visible in the final output directly.
Next on the Foreground Layer set the Composition Mode to "OpenGL-Over" (or "VVSource-Atop") and add a "Layer Mask.fs" FX with its Mask Image input set to receive from the "Mask Layer" that we've created.
Once this is complete to make our scene more visually interesting we'll then repeat this process to add a mid ground layer that appears in between the foreground and background layers.
Tips and Notes:
Use the OpenGL-Over composition mode on foreground and overlay layers.
Use the Contrast slider in the Layer Mask FX to smoothly invert the masking image.
Use the Chroma Mask FX to convert blue screen footage to a grayscale masks.
Text layers and other built in sources are also useful as mask generators.