Tracking faces, bodies, and hands with VDMX

The Video Tracking plugin provides an interface for detecting faces, bodies, and hands and using their locations as data-source values and masks that can be used to control virtually any part of VDMX.

In this tutorial we will look at using the face and hand tracking options in the Video Tracking to create a fun example of controlling the parameters of a simple ISF generator.

Download ISF Shader and Project Files here.

---

---

The main panel for the Video Tracking plugin contains a pop-up menu for selecting which video feed to analyze and a status indicator for the analysis.

For situations where multiple bodies and / or faces are detected, the provided 'prev', 'next', and 'rand' buttons can be used to switch between which is currently being tracked. When previewing the body and face tracking video streams clicking on a bounding box region can also be to change the active tracking.

The tracking options are as follows:

• Human Tracking: Detects human bodies and publishes the position as data-sources. This includes the center position, width, height, and the x/y coordinates of each corner of the box bounding detected people. A boolean 'Detected' data-source turns on / off when bodies are found.

• Face Tracking: Detects faces and publishes the position as data-sources. This includes the center position, width, height, and the x/y coordinates of each corner of the box bounding detected face. A boolean 'Detected' data-source turns on / off when faces are found.

• Hand Detect: Detects one or more hands and publishes the center position as data-sources. Includes options for specifying the maximum number of tracked hands and the chirality type (All, Left, Right, or Pairs). A boolean 'Detected' data-source turns on / off when hands are found.

• Human Mask: Generates a mask image that can be used e.g. with the Layer Mask effect to remove the background from images. Includes a quality setting (accurate, balanced, or fast). Also see the 'Remove Background' effect.

• Attention Saliency: Generates a mask image using an Attention Saliency algorithm.

• Object Saliency: Generates a mask image using an Object Saliency algorithm.

Where available the 'Publish Preview' option can be enabled for each tracking mode. This will create a video feed with overlays showing bounding boxes and other visualizations related to the analyzed images.

The "Minimum Latency" option in the inspector can be enabled to reduce latency (masks will be tighter) but takes longer to process (framerate may be lower, depending on the overall system load). This is option is off by default.

Hand tracking in VDMX6

If you have a LeapMotion or UltraLeap, and want to try the GECO application to share OSC data with VDMX. You can find it here: https://uwyn.com/geco/ We used the first generation LeapMotion controller in this tutorial.

For the iPad you can use Shoots Pro for sharing over NDI or Elgato Epoccam

Introduction to OCR & QR code capture in VDMX

VDMX 6.1 introduces a powerful new OCR (Optical Character Recognition) plugin that allows you to scan text and QR codes from live video input. These scanned results can be published as data-sources and used to trigger UI elements, update text layers, or automate actions in your VDMX setup.

Getting Started

To try it out, load the OCR Example template from the Templates menu. This preset is included with the latest version of VDMX and demonstrates how to use both OCR and QR scanning.

Before using the OCR plugin, make sure:

• You’re running VDMX version 6.1 or later

• Quartz Composer is enabled under VDMX > Preferences > Rendering

What You Can Do

• Live Text Scanning: Point your webcam at printed or handwritten text and see it appear in real time.

• QR Code Scanning: Detect and display QR content directly into your project.

• Data Routing: Use the UI Inspector to map scanned strings to text layers, pop-up menus, or other elements.

• Clock Syncing: Trigger OCR or QR scans automatically on every beat using the Clock plugin.

This makes it easy to create interactive visuals using real-world inputs—great for installations, performances, or creative automation.

---

Trigger Media Clips in Real Time Using OCR and QR Codes in VDMX

By why stop there!? These outputs can be used to control UI elements like pop-up menus and, in turn, trigger clips in the Media Bin.

In this other tutorial, we’ll walk through how to set up a system that lets you hold up color-coded QR labels or text to control clip playback — ideal for powering interactive installations, printed cue cards, or playful VJ sets.

Getting Started

1. Open the “OCR Example” template from the Templates menu.

2. In the Workspace Inspector (Cmd+1), add a Control Surface plugin.

3. Create a Pop-Up Button and label its items (e.g., Red, Green, Blue).

4. In the UI Inspector (Cmd+2), set the pop-up to be controlled by the OCR text string:

• Navigation > Select by string

• Data-Source > OCR Text

Link OCR to Media Playback

Once your pop-up button is receiving the OCR string:

1. Go to the Media Bin Controls tab.

2. Set Trigger by Index and choose the pop-up button as the source.

3. When the pop-up changes value (based on OCR input), the corresponding clip will be triggered.

Try It Live

Switch the OCR video source to a FaceTime or external camera, then hold up QR codes or printed text. As the plugin reads values like “red,” “green,” or “blue,” it updates the pop-up and triggers the matching clip.

You can also sync scanning with the Clock plugin to automatically scan at regular intervals, creating hands-free interaction loops.

Tips & Tricks

• Case matters – OCR text strings must match your pop-up labels exactly.

• You can also scan handwritten words, printed stickers, or even project QR codes from your VDMX interface using:

• QR Code Generator Source (creates QR codes as layer sources)

• QR Code Overlay FX (renders QR code overlays on top of any layer)

---

Try It Yourself

The OCR Example template is available in the latest build of VDMX. If you’re experimenting with it in your work, tag us—we’d love to see how you’re using this feature.

If you’re building an interactive installation or performance using OCR, we’d love to see it. Tag us or share your project with [@VIDVOX].

Mastering Projector Rigging: Elevate Your Visual Installations with Pro Techniques

So, you've got a projector? Now, let's take your visual installations to the next level! While there's no shortage of inexpensive projector mounts online, some fall short for custom setups and quick turnarounds. Enter GRIP hardware – a tried-and-true solution from the film world that's been revolutionized with 3D printing.

Mount Anywhere! Don’t forget the safety cable!

Image from ‘supercell by slowdanger’ taken at the Flea theather in New York City. January 2024.

Lots of options!
Avenger F810 Baby Plate with 5/8" Swivel Spigot + a Grip Head + 6” Baby Pin Wall Plate

In the film/video realm, GRIP hardware is the unsung hero of lighting and rigging, trusted on film sets, TV shows, and in theaters. This gear, designed to support hefty lighting rigs for extended periods, can be the perfect match for your projector mounting needs.

Lots of metal bits for any type of installation. It only gets strange when explaining to TSA!

Adaptable and Accessible: The ⅝" Baby Pin Connection

3” Baby Pin Wall plate attached with a custom made 3D printed mount for Optoma Projectors that works with Impact and Manfrotto plates.

At the core of this hardware is the ⅝" Baby Pin connector – a versatile link that works wonders for small to medium-sized projectors. If you're dealing with a large event projector, chances are it comes with its own secure cage or mounting structure for a safer installation.

The black coupler is a “Double Female Adapter” made by Kupo. The rest is a mix of Manfrotto Avenger and Impact GRIP hardware.

This type of connection opens the door to hundreds, if not thousands, of mounting possibilities. Trusted manufacturers include Impact Lighting (budget-friendly), Manfrotto Avenger Series, Matthews Lighting, and Kupo Lighting.

Left: Projector + wood mount with super clamp on a swivel head. Right: Projector with magic arm + wood mount and receiver plate.

The Rigging Essentials: Cardellini Clamps, Jaw Vice Clamps, and More

The Cardellini Clamp, affectionately known as a Mathelini in theatrical circles (or alternatively as a Jaw Vice Clamp online), offers an impressive 6"+ range, making it an indispensable tool for securing projectors in diverse scenarios. When paired with an Impact Baby Pin Swivel Head Mount, complete with a sleek black Kupo connector, and anchored by a Manfrotto Avenger ⅝" baby pin receiver, this setup guarantees both stability and flexibility.

While different brands may come with varying price tags, their performance is generally comparable. However, it's worth noting that complications can arise when mixing brands, as they may have slightly different pin "lock" heights (the indentation at the top of the baby pin). Although it's possible to mix and match, optimal connectivity is often achieved when sticking to the same brand for all components in your hardware ensemble. (Link to 2” vice clamp)

Jaw Vice Clamp

Cardellini Clamp! (or Mathelini) Assuming you bought from Matthews Studio Equipment.

Using a Jaw Vice Clamp

Mounting projectors where ever you want!

This hardware is strong!

Seriously, this Manfrotto Super Clamp survived years outside and even made it through a hurricane! (video) (It was only holding a camera, but still).

Once tightened in place, these components stand firm, even under the weight of heavy projectors. However, for added security, never forget the importance of a safety cable – a 1/16" aircraft cable that ensures your setup stays put, and it is relatively inexpensive to make your own safety cables after purchasing steel cable cutters and a swaging tool.

Expanding Your Toolkit: Accessories for Seamless Installations

Beyond the essentials, assembling a well-rounded toolkit is paramount for a flawless installation. Consider expanding your arsenal with beam clamps, spring clamps, additional IEC cables, HDMI over Ethernet adapters, and HDMI cables under 50'. It's crucial to remain mindful of HDMI cable limitations – once you exceed 50 feet, exploring signal boosters or HDMI over Ethernet solutions becomes imperative.

Here's a pro-tip: I highly recommend using an IEC cable tap for added convenience. This allows you to power a media player ( this one offers seemless loop with .mkv files, must used a hidden file cleaner like BlueHarvest to remove hidden “.trash” files from USB drive or SD card before looping a folder of files). , Raspberry Pi, or HDMI over RJ45 adapter with a single cord, streamlining your setup.

Speaking of HDMI over RJ45 adapters (not HDMI over Ethernet!), I found mine for less than $20 USD, featuring both HDMI input and output on the transmitter (TX). While it seems they're currently sold out, there's no need to break the bank; spending $50 or more on this type of adapter is unnecessary. Instead, consider investing in a quality shielded CAT6 cable or making one yourself. A shielded cable helps minimize noise, making it especially beneficial for longer runs, particularly when running cables alongside power cables. This cost-effective approach ensures optimal performance without compromising your budget. (Note: HDMI over Ethernet means you could send the signal over a network, switch, router, etc. HDMI over RJ45 or HDMI over Cat5e requires a “homerun” cable that runs direct from the transmitter (TX) to receiver (RX). The protocol that is used by the manufacturers may be different then others, so you can’t mix and match these RX and TX receivers with different brands.)

Beam Clamps, pipe clamp, super clamps, baby pin adapters, yoke mount, grip head, adjustable magic arms, and a swivel head mount baby pin plate enhance adaptability, offering creative solutions for various mounting scenarios.

Beam Clamp

Handy for attaching things to “I Beams.” Especially helpful for securing safety cables in place.

Spring Clamps

Inexpensive, and versatile, and good to have in your bag!

Pipe clamp

Good for when you’re working in a lighting grid environment.

GRIP Head

These are commonly found on C-Stands, and can hold a bunch of things at various angles.

Magic Arm

Can position it almost any which way. These tend to sag over time with heavier projectors and in long installations, but great for quick configurations!

Gaffers Tape

Great to have in your kit. A good quality Gaffers tape goes a long way and avoids tape residue after removed.

Conclusion: Your Projector Rigging Journey

As you embark on your projector rigging journey, the right accessories make all the difference. This comprehensive guide ensures you're well-equipped for any installation, whether it's for escape rooms, VJing, projection mapping, or visual effects. Elevate your visual installations with the perfect blend of industry-proven hardware and cutting-edge solutions – because your projector deserves nothing less!

This article was written by ProjectileObjects.  You can learn more about them at http://projectileobjects.com/ or follow them on Instagram @ProjectileObjects 

Akai APC40 MK II 2-Channel VJ Mixer template for VDMX

Templates are a great way to get started with VDMX and with this template you can take an out of the box APC40 MKII and jump right in!

A few things to note about the APC40 MK II before we get started.

The APC40 MK II has three internal MIDI mapping modes.

• Generic Mode (Default)

• Ableton Live Mode

• Alternate Ableton Live Mode

To use this template correctly, you’ll need your APC40 MK II to be set to the default “stock” Generic Mode. More information about these modes can be found here (PDF) Bottom of Page 10.

When you first turn on the controller, it will default to the correct button mapping. To reset the template to all defaults, it is recommended that you hit this button when you start the template to eject all clips and set everything to its default.

This button ejects all media, clears all the FX and syncs the LFO view to the LFO slider. (Warning: You’ll lose FX in Layer A and B if you don’t save them as a new FX chain).

Not all buttons are RGB. When clips are ready to be triggered in your media bin, the 40 RGB button grid will light up blue, then yellow when the clip is selected. You can customize these colors yourself in the media bin options:

Image found on page 10, Akai communication protocol manual.

There are two versions of this template. A blank version without FX and a starter version with one layer of FX presets.

• Download BLANK template here.

• Download VJ Mixer FX Starter here.

This template is structured to be a 2-channel video mixer. Both video layer A and B flow to a Master output (Projector, TV, etc.) The cross fader blends between both layers and each layer has its own FX chain presets.

The Master output FX are turned on and off by the top 8 rotary knobs. The first vertical slider on the right side of the controller labeled “MASTER” controls the master opacity. If it is all the way down, your screen output will be black. You can change this later to preference or disable it entirely.

Selecting clips for both layers A and B:

Both layers use the same 40 RGB button grid to trigger clips. To switch between Layer A and B, when selecting clips — use the first two buttons on the top right side of the grid under the label “SCENE LAUNCH” They will light up when they are selected. Top goes sets destination for Layer A, bottom for Layer B. The two buttons beneath that (Green) are page up / page down buttons for moving through your media bin. They are also linked to your Audio Analysis Filter 3 and will flicker based on your computers mic peaking. Beneath that (Yellow) is a random clip trigger.

To trigger to the next clip in the media or move up and down the media bin, redirect your eyes to the “BANK SELECT” 4 button arrow keys.

The rest of the buttons should be self explanatory based on the image above, or you can read through the “User Notes” built into the template which explains all of this and more.

---

Template Tip!

If you’re adding new FX to your A and B layer FX chains, make sure to save them as a preset by clicking the + in the top of the FX window. This will save your FX chain and you can assign it to a new FX preset button. You can always disable the FX layers MIDI triggers in your project until you build out the template more to your liking!

---

Here’s a brief overview video of this template:

Advanced v002 Rutt Etra Techniques in VDMX

Download the completed project file and media files for this tutorial.

One of the most fun visual effects used in live visuals and VJing dates back to the analog Rutt Etra video synthesizer which had many capabilities that have over the years been translated into digital form. Most notable among the digital versions is the v002 Rutt Etra Quartz Composer plugin which was developed by vade along with Bill Etra, one of the original creators of the Rutt Etra system. We’ve included this plugin as an FX, along with several other v002 creations with VDMX for a while now and, over the years we’ve seen a lot of projects that have made use of them… and in this video tutorial we’re hoping to teach you a few new tricks…

Before beginning this tutorial, make sure to use the v002 installer on the VDMX disk image in the Extra folder.

---

For this demonstration, instead of using the Rutt Etra as an FX, we have a Quartz Composer composition that generates video based on two input video streams. In this case we will create sets of hidden layers that will be used for these inputs: the Video Input will provide the colors for each particle and the Displacement Mask will provide RGB color gradients.

The main technique used here will be using an RGB to XYZ displacement map instead of a luma based displacement map. This means that we both have more flexibility for changing the positions of individual pixels, but we are now also responsible for creating the basic gradients that are needed to draw the basic image in a non-distorted way.

To accomplish this, we included special ISF compositions that are RGB color gradient generators. Each of these sketches includes three different gradients that are combined into a single output. One special feature of the gradient generators is the ability to pass in another video feed as a ‘look up table’ for one of the curves, making it possible to use waveforms and FFTs from audio analysis plugins, or any other image, as data that can modulate the output.

Once the basic setup is complete we can group the layers and add a Two Channel Mixer plugin to mix between different gradient outputs to fade the Rutt Etra generator between shapes and apply FX to the gradient images can create all kinds of different variations in the output. Here we use the VVMotionBlur effect to apply temporal smoothing to images and the Bad TV FX to create ripples.

---

Tips and Notes:

View fullsize

Audio analysis, camera inputs and other images can also be used as lookup tables for the gradients.

The Video Input will provide the colors for each particle and the Displacement Mask will provide RGB color gradients.

View fullsize

Use basic gradients to create outputs without distortion.