Using the v002 Analog Glitch.fs FX on a live camera in VDMX6.
As VDMX6 and other tools were forced to sunset Quartz Composer, we've gotten quite a few requests to help revive some beloved FX patches. For those who are curious about how some of these shaders were adapted from GLSL to ISF, in this tutorial we will look at the ‘v002 Glitch Analog’ plugin as an example.
Read MoreThe Video Tracking plugin provides an interface for detecting faces, bodies, and hands and using their locations as data-source values and masks that can be used to control virtually any part of VDMX.
In this tutorial we will look at using the face and hand tracking options in the Video Tracking to create a fun example of controlling the parameters of a simple ISF generator.
The main panel for the Video Tracking plugin contains a pop-up menu for selecting which video feed to analyze and a status indicator for the analysis.
For situations where multiple bodies and / or faces are detected, the provided 'prev', 'next', and 'rand' buttons can be used to switch between which is currently being tracked. When previewing the body and face tracking video streams clicking on a bounding box region can also be to change the active tracking.
The tracking options are as follows:
Human Tracking: Detects human bodies and publishes the position as data-sources. This includes the center position, width, height, and the x/y coordinates of each corner of the box bounding detected people. A boolean 'Detected' data-source turns on / off when bodies are found.
Face Tracking: Detects faces and publishes the position as data-sources. This includes the center position, width, height, and the x/y coordinates of each corner of the box bounding detected face. A boolean 'Detected' data-source turns on / off when faces are found.
Hand Detect: Detects one or more hands and publishes the center position as data-sources. Includes options for specifying the maximum number of tracked hands and the chirality type (All, Left, Right, or Pairs). A boolean 'Detected' data-source turns on / off when hands are found.
Human Mask: Generates a mask image that can be used e.g. with the Layer Mask effect to remove the background from images. Includes a quality setting (accurate, balanced, or fast). Also see the 'Remove Background' effect.
Attention Saliency: Generates a mask image using an Attention Saliency algorithm.
Object Saliency: Generates a mask image using an Object Saliency algorithm.
Where available the 'Publish Preview' option can be enabled for each tracking mode. This will create a video feed with overlays showing bounding boxes and other visualizations related to the analyzed images.
The "Minimum Latency" option in the inspector can be enabled to reduce latency (masks will be tighter) but takes longer to process (framerate may be lower, depending on the overall system load). This is option is off by default.
Hand tracking in VDMX6
If you have a LeapMotion or UltraLeap, and want to try the GECO application to share OSC data with VDMX. You can find it here: https://uwyn.com/geco/ We used the first generation LeapMotion controller in this tutorial.
For the iPad you can use Shoots Pro for sharing over NDI or Elgato Epoccam
VDMX 6.1 introduces a powerful new OCR (Optical Character Recognition) plugin that allows you to scan text and QR codes from live video input. These scanned results can be published as data-sources and used to trigger UI elements, update text layers, or automate actions in your VDMX setup.
To try it out, load the OCR Example template from the Templates menu. This preset is included with the latest version of VDMX and demonstrates how to use both OCR and QR scanning.
Before using the OCR plugin, make sure:
• You’re running VDMX version 6.1 or later
• Quartz Composer is enabled under VDMX > Preferences > Rendering
What You Can Do
• Live Text Scanning: Point your webcam at printed or handwritten text and see it appear in real time.
• QR Code Scanning: Detect and display QR content directly into your project.
• Data Routing: Use the UI Inspector to map scanned strings to text layers, pop-up menus, or other elements.
• Clock Syncing: Trigger OCR or QR scans automatically on every beat using the Clock plugin.
This makes it easy to create interactive visuals using real-world inputs—great for installations, performances, or creative automation.
By why stop there!? These outputs can be used to control UI elements like pop-up menus and, in turn, trigger clips in the Media Bin.
In this other tutorial, we’ll walk through how to set up a system that lets you hold up color-coded QR labels or text to control clip playback — ideal for powering interactive installations, printed cue cards, or playful VJ sets.
1. Open the “OCR Example” template from the Templates menu.
2. In the Workspace Inspector (Cmd+1), add a Control Surface plugin.
3. Create a Pop-Up Button and label its items (e.g., Red, Green, Blue).
4. In the UI Inspector (Cmd+2), set the pop-up to be controlled by the OCR text string:
• Navigation > Select by string
• Data-Source > OCR Text
Link OCR to Media Playback
Once your pop-up button is receiving the OCR string:
1. Go to the Media Bin Controls tab.
2. Set Trigger by Index and choose the pop-up button as the source.
3. When the pop-up changes value (based on OCR input), the corresponding clip will be triggered.
Try It Live
Switch the OCR video source to a FaceTime or external camera, then hold up QR codes or printed text. As the plugin reads values like “red,” “green,” or “blue,” it updates the pop-up and triggers the matching clip.
You can also sync scanning with the Clock plugin to automatically scan at regular intervals, creating hands-free interaction loops.
Tips & Tricks
• Case matters – OCR text strings must match your pop-up labels exactly.
• You can also scan handwritten words, printed stickers, or even project QR codes from your VDMX interface using:
• QR Code Generator Source (creates QR codes as layer sources)
• QR Code Overlay FX (renders QR code overlays on top of any layer)
The OCR Example template is available in the latest build of VDMX. If you’re experimenting with it in your work, tag us—we’d love to see how you’re using this feature.
If you’re building an interactive installation or performance using OCR, we’d love to see it. Tag us or share your project with [@VIDVOX].
Using the New Color Transfer FX in VDMX6
The Color Transfer FX is one of the new additions in VDMX6, providing a powerful way to match the color and brightness levels of a video stream to a reference image or layer. Whether you’re blending visuals from different sources or aiming for a cohesive color palette across layers, this effect makes it easy to achieve polished results without LUTs or external tools.
Similar to using LUTs to stylize an image, the new Color Transfer FX in VDMX6 can be used to alter the colors of a layer by using any available video stream as a reference.
This video will walk through the basics of using both the Color Transfer and Segmented Color Transfer FX.
Quick Start
• Open the Simple Mixer template to get a two-layer setup.
• Load image or video assets into the media bin.
• Apply the Color Transfer FX to one layer, and select another as the reference source.
• Use the chroma and luma sliders to control how much color and brightness are shifted.
For macOS 14+ users:
• Try the Segmented Color Transfer FX to independently adjust foreground and background tones using separate references.
Color Transfer FX
Tips
• For a uniform look, you can apply this FX to your Main Canvas FX and use hidden layers to regulate your project’s colors.
• Apply other FX (like Color Controls) to the reference layer to influence the result.
• You can try stacking layers and blend modes to get a unique look.
Share Your Work
If you’re using the new Color Transfer or Segmented Color Transfer FX in your own projects or live performances, we’d love to see what you’re making. Tag us (@VIDVOX) or drop us a message to share your work.
Use the new Remove Background FX in VDMX6 to automatically key any video stream without having to use a blue / green screen.
Read MoreOne of the most fun aspect of using Eurorack setups is the ability to quickly reroute control data and sound between different modules. Conversely one of the most limiting parts of using Eurorack setups is the ability to quickly swap out different modules from your rack to get different kinds of control data and sound coming and going from your system. In this tutorial we will look at how the Step Sequencer and LFO plugins in VDMX can be used alongside Eurorack setups to provide a versatile approach to generating CV values.
As Eurorack modules are also often a significant investment of money, it can also sometimes be useful to use software tools like VDMX to simulate their abilities to determine if they are a good fit for your needs before purchasing.
This tutorial is broken into three main parts:
Setting up our Eurorack to convert MIDI to CV.
Setting up VDMX to send MIDI to the Eurorack.
Configuring step sequencer and LFOs in VDMX to control parameters on our Eurorack.
Univer Iter MIDI to CV and Tiptop Audio Buchla 258t Eurorack modules.
For this initial demonstration of doing MIDI to CV we are using the Noise Engineering Univer Inter along with a Buchla & Tiptop Audio 258t Dual Oscillator module to generate tones.
The Univer Iter has 8 CV out ports along with a USB port which can be directly connected to a computer for receiving incoming MIDI. Within applications like AudioMIDI Setup and VDMX it appears as a standard MIDI output device option. It also can be configured to use a custom MIDI mapping as needed and can be daisy chained with a second module for another 8x outputs.
A variety of different modules are available for taking MIDI data in one form or another and converting it to CV. As always with Eurorack setups it is prudent to spend some time looking at all of the module options and picking the best for your specific needs.
Most user interface controls in VDMX such as sliders and buttons can be configured to directly send their current value as MIDI output using the “Send” tab of the “UI Inspector” window. When configuring VDMX to drive external devices such as a Eurorack it is often useful to add a “Control Surface” plugin with customized set of UI elements that represent each of our individual CV outputs.
Steps:
Use the “Plugins” tab of the “Workspace Inspector” to add a “Control Surface“ plugin to the project.
Use the sub-inspector to add one or more UI elements (sliders, buttons, pop-up menus, etc). to the control surface interface.
Click on each UI element in the Control Surface main window to inspect it. Use the “Send“ tab of the “UI Inspector” to configure the MIDI mapping and output device.
Now that our Eurorack is receiving MIDI from VDMX and converting it to CV we can begin to set up our Step Sequencer and LFO plugins to drive individual parameters of our synthesizer.
A VDMX setup with a two track step sequencer, an LFO, a clock plugin, and a control surface configured to send MIDI output.
Steps:
Right-click on sliders and buttons to assign data sources.
Use the “Plugins” tab of the “Workspace Inspector” to add a “Step Sequencer“ plugin and an “LFO” plugin to the project.
Use the sub-inspector to customize Step Sequencer / LFO configurations as needed.
Right click on output UI elements in the Control Surface or use the UI Inspector to route generated control data to our MIDI outputs.
Patch the MIDI module CV output to synthesizer input parameters.
Use the “Clock” plugin to adjust the overall BPM.
Once we’ve created our parameter routings on the Eurorack we can also optionally further customize our Control Surface with appropriate labels and display ranges, or continue to leave them as generic 0-1 values that are commonly re-patched on the fly.
So, you've got a projector? Now, let's take your visual installations to the next level! While there's no shortage of inexpensive projector mounts online, some fall short for custom setups and quick turnarounds. Enter GRIP hardware – a tried-and-true solution from the film world that's been revolutionized with 3D printing.
Mount Anywhere! Don’t forget the safety cable!
Image from ‘supercell by slowdanger’ taken at the Flea theather in New York City. January 2024.
Lots of options!
Avenger F810 Baby Plate with 5/8" Swivel Spigot + a Grip Head + 6” Baby Pin Wall Plate
In the film/video realm, GRIP hardware is the unsung hero of lighting and rigging, trusted on film sets, TV shows, and in theaters. This gear, designed to support hefty lighting rigs for extended periods, can be the perfect match for your projector mounting needs.
Lots of metal bits for any type of installation. It only gets strange when explaining to TSA!
3” Baby Pin Wall plate attached with a custom made 3D printed mount for Optoma Projectors that works with Impact and Manfrotto plates.
At the core of this hardware is the ⅝" Baby Pin connector – a versatile link that works wonders for small to medium-sized projectors. If you're dealing with a large event projector, chances are it comes with its own secure cage or mounting structure for a safer installation.
The black coupler is a “Double Female Adapter” made by Kupo. The rest is a mix of Manfrotto Avenger and Impact GRIP hardware.
This type of connection opens the door to hundreds, if not thousands, of mounting possibilities. Trusted manufacturers include Impact Lighting (budget-friendly), Manfrotto Avenger Series, Matthews Lighting, and Kupo Lighting.
Left: Projector + wood mount with super clamp on a swivel head. Right: Projector with magic arm + wood mount and receiver plate.
The Cardellini Clamp, affectionately known as a Mathelini in theatrical circles (or alternatively as a Jaw Vice Clamp online), offers an impressive 6"+ range, making it an indispensable tool for securing projectors in diverse scenarios. When paired with an Impact Baby Pin Swivel Head Mount, complete with a sleek black Kupo connector, and anchored by a Manfrotto Avenger ⅝" baby pin receiver, this setup guarantees both stability and flexibility.
While different brands may come with varying price tags, their performance is generally comparable. However, it's worth noting that complications can arise when mixing brands, as they may have slightly different pin "lock" heights (the indentation at the top of the baby pin). Although it's possible to mix and match, optimal connectivity is often achieved when sticking to the same brand for all components in your hardware ensemble. (Link to 2” vice clamp)
Cardellini Clamp! (or Mathelini) Assuming you bought from Matthews Studio Equipment.
Mounting projectors where ever you want!
This hardware is strong!
Seriously, this Manfrotto Super Clamp survived years outside and even made it through a hurricane! (video) (It was only holding a camera, but still).
Once tightened in place, these components stand firm, even under the weight of heavy projectors. However, for added security, never forget the importance of a safety cable – a 1/16" aircraft cable that ensures your setup stays put, and it is relatively inexpensive to make your own safety cables after purchasing steel cable cutters and a swaging tool.
Beyond the essentials, assembling a well-rounded toolkit is paramount for a flawless installation. Consider expanding your arsenal with beam clamps, spring clamps, additional IEC cables, HDMI over Ethernet adapters, and HDMI cables under 50'. It's crucial to remain mindful of HDMI cable limitations – once you exceed 50 feet, exploring signal boosters or HDMI over Ethernet solutions becomes imperative.
Here's a pro-tip: I highly recommend using an IEC cable tap for added convenience. This allows you to power a media player ( this one offers seemless loop with .mkv files, must used a hidden file cleaner like BlueHarvest to remove hidden “.trash” files from USB drive or SD card before looping a folder of files). , Raspberry Pi, or HDMI over RJ45 adapter with a single cord, streamlining your setup.
Speaking of HDMI over RJ45 adapters (not HDMI over Ethernet!), I found mine for less than $20 USD, featuring both HDMI input and output on the transmitter (TX). While it seems they're currently sold out, there's no need to break the bank; spending $50 or more on this type of adapter is unnecessary. Instead, consider investing in a quality shielded CAT6 cable or making one yourself. A shielded cable helps minimize noise, making it especially beneficial for longer runs, particularly when running cables alongside power cables. This cost-effective approach ensures optimal performance without compromising your budget. (Note: HDMI over Ethernet means you could send the signal over a network, switch, router, etc. HDMI over RJ45 or HDMI over Cat5e requires a “homerun” cable that runs direct from the transmitter (TX) to receiver (RX). The protocol that is used by the manufacturers may be different then others, so you can’t mix and match these RX and TX receivers with different brands.)
Beam Clamps, pipe clamp, super clamps, baby pin adapters, yoke mount, grip head, adjustable magic arms, and a swivel head mount baby pin plate enhance adaptability, offering creative solutions for various mounting scenarios.
Handy for attaching things to “I Beams.” Especially helpful for securing safety cables in place.
Inexpensive, and versatile, and good to have in your bag!
Good for when you’re working in a lighting grid environment.
These are commonly found on C-Stands, and can hold a bunch of things at various angles.
Can position it almost any which way. These tend to sag over time with heavier projectors and in long installations, but great for quick configurations!
Great to have in your kit. A good quality Gaffers tape goes a long way and avoids tape residue after removed.
As you embark on your projector rigging journey, the right accessories make all the difference. This comprehensive guide ensures you're well-equipped for any installation, whether it's for escape rooms, VJing, projection mapping, or visual effects. Elevate your visual installations with the perfect blend of industry-proven hardware and cutting-edge solutions – because your projector deserves nothing less!
This article was written by ProjectileObjects. You can learn more about them at http://projectileobjects.com/ or follow them on Instagram @ProjectileObjects 
Mine S running VDMX at the Guggenheim Theater, NYC (photo @ProjectileObjects)
Normally when we write about a new controller, we offer up a template to get you started, but with the Special Waves Mine S controller series, we'll have to take a different approach.
For years now, users have chosen VDMX for it's modular workflow. Build out a project the way you'd like it, so when a modular MIDI controller comes along, we had to try it.
The Special Waves Mine S controller is the most budget friendly if you're out there looking for an inexpensive controller, but it does pack a range of features, quality buttons and faders, and is compact enough to fit inside any bag. Oh, and it's modular! (had to say it again).
Yes, you can build out your VDMX project, map the midi sliders, knobs, and buttons to your VDMX workspace, and then pop them out and move them around as desired without having to reconfigure the MIDI settings inside of VDMX. How cool!
Maybe you've built out a horizontal layout at your desk, but when you get to the show there's only enough space for your laptop and a vertical layout of your controller -- well, just move the buttons around!
Top down: Livid Ohm RGB slip, APC 40 MKII, Mine S, DJ2GO Touch.
The original Livid OHM 64 went for $699 (from from a mass produced APC40MKII at $299), and a Mine S will cost you anywhere from $399-? (There are 32 slots on the controller, a slider takes up 4, and a button or knob takes up one. The Mine S Pro Bundle sitting in front of me with an extra slider will set you back around $516+ ~$30 shipping (I wish I would have purchased an additional 2 Button Module ($14 more) to balance out this setup).
The build design, and packaging are exceptional, but one thing was missing. A case! Luckily, we found this padded case on Amazon for $21.99 USD. Designed for the Donner DMK 25 MIDI Keyboard, this fits the Mine S with some room to spare -- all while fitting in a backpack.
With all that being said, we wanted this tutorial to be informational. Since every setup with the Mine S is different, we can’t give you a preconfigured template with MIDI mappings, but we can show you how to make your own custom UI with the TouchOSC editor and import into VDMX
I’ve made a template for the Mine S controller the way I have it configured, running their default MIDI profile (in the Mine App). You can download that here and install in your VDMX template folder. The Mine supports MIDI or OSC, either can be used for this example.
To install templates in VDMX go to: ‘Your Drive’ > Users > ‘username’ > Library > Application Support > VDMX > templates
The VDMX custom control surface plugin has a great import feature that allows you to import a TouchOSC template. The trick is you have to use the TouchOSC MK1 editor to create a compatible file.
That editor can be downloaded from Hexler’s website here.
From here you can build out your Mine S controller layout using simple vertical and horizontal sliders and faders. Simply right click on the blank interface, select the button, slider, or fader you want, and piece your layout together. Don’t worry about the alignment right now. We’ll reconfigure that in VDMX after import.
Add a new Control Surface using the + icon in the Workspace Inspector
Next, click Import TouchOSC Layout and select the TouchOSC template you saved.
Click IMPORT, the default settings are fine.
When I import my template, the layout has the wrong orientation.
No worries, we’ll fix that in a second.
Return to the Workspace Inspector, select the control surface, and at the bottom of the User Interface Inspector, select the Edit UI box.
Now you can see options to change the sizing and Rotate the UI. Click Rotate, to get the correct orientation for your controller layout.
That’s much better. From here, I can resize, adjust, the buttons, sliders, and layout. I made my UI more compact, but you don’t have to.
Before finishing up. I’ll click Minimize under the control surface User Interface Inspector.
This will cleanup my interface.
When finished make sure you uncheck the Edit UI checkbox to return the control surface back to a locked setup. The UI grid will disappear, and now you’re interface is ready for mapping.
Mapping a custom UI is completely optional. You could skip this step by mapping other aspects of VDMX directly, but from time to time, it’s helpful to map out control surfaces such as this, so you can have a visual representation of what settings your current controller is outputting.
Pros
Modular, but remembers midi settings. Buy more buttons, configure to your hearts desire.
Kensington Lockable
MIDI IN and MIDI OUT
Optional 5V DC power supply (for USB ports that can't handle the setup). I haven't had to use this.
Power on/off button.
Quality buttons, knobs, and sliders.
MIDI Notes, CC, or OSC -- all custom configurable.
Custom colors, and great range of dimmable LEDs
Cons
Only one type of medium resistance slider available, no low resistence option available for a cross-fader. (You'll have to develop stronger fingers)
No USB C port.
Not fully standalone (currently requires the Mine S software to be installed on your machine). Their website says they will release and update to fix this, but right now, you'll have to run another app.
MineApp application required for Mine S run.
If you do end up purchasing a Special Waves Mine S. We recommend buying a case, and a USB B to USB C cable to directly connect.
Using SnapChat lenses for video playback and live performance tutorial by ProjectileObjects.
Read MoreTemplates are a great way to get started with VDMX and with this template you can take an out of the box APC40 MKII and jump right in!
A few things to note about the APC40 MK II before we get started.
The APC40 MK II has three internal MIDI mapping modes.
Generic Mode (Default)
Ableton Live Mode
Alternate Ableton Live Mode
To use this template correctly, you’ll need your APC40 MK II to be set to the default “stock” Generic Mode. More information about these modes can be found here (PDF) Bottom of Page 10.
When you first turn on the controller, it will default to the correct button mapping. To reset the template to all defaults, it is recommended that you hit this button when you start the template to eject all clips and set everything to its default.
This button ejects all media, clears all the FX and syncs the LFO view to the LFO slider. (Warning: You’ll lose FX in Layer A and B if you don’t save them as a new FX chain).
Not all buttons are RGB. When clips are ready to be triggered in your media bin, the 40 RGB button grid will light up blue, then yellow when the clip is selected. You can customize these colors yourself in the media bin options:
Image found on page 10, Akai communication protocol manual.
There are two versions of this template. A blank version without FX and a starter version with one layer of FX presets.
This template is structured to be a 2-channel video mixer. Both video layer A and B flow to a Master output (Projector, TV, etc.) The cross fader blends between both layers and each layer has its own FX chain presets.
The Master output FX are turned on and off by the top 8 rotary knobs. The first vertical slider on the right side of the controller labeled “MASTER” controls the master opacity. If it is all the way down, your screen output will be black. You can change this later to preference or disable it entirely.
Selecting clips for both layers A and B:
Both layers use the same 40 RGB button grid to trigger clips. To switch between Layer A and B, when selecting clips — use the first two buttons on the top right side of the grid under the label “SCENE LAUNCH” They will light up when they are selected. Top goes sets destination for Layer A, bottom for Layer B. The two buttons beneath that (Green) are page up / page down buttons for moving through your media bin. They are also linked to your Audio Analysis Filter 3 and will flicker based on your computers mic peaking. Beneath that (Yellow) is a random clip trigger.
To trigger to the next clip in the media or move up and down the media bin, redirect your eyes to the “BANK SELECT” 4 button arrow keys.
The rest of the buttons should be self explanatory based on the image above, or you can read through the “User Notes” built into the template which explains all of this and more.
If you’re adding new FX to your A and B layer FX chains, make sure to save them as a preset by clicking the + in the top of the FX window. This will save your FX chain and you can assign it to a new FX preset button. You can always disable the FX layers MIDI triggers in your project until you build out the template more to your liking!
Here’s a brief overview video of this template:
How to use VDMX to add custom effects, visuals, and videos to a Zoom Meeting.
Read MoreProject in Apple Motion using ISF generators and effects from VDMX
For VJs and other visual performers, one of the biggest parts of the job is creating your own original materials for your shows. If you like the workflow of compositing layers, applying effects, and automating values found in VDMX, the interface of Apple Motion offers a similar approach for creating visuals with the intention of exporting as movie files, at an affordable price.
Last month we introduced ISF for Motion, a new plug-in that makes it possible to use the same GLSL shaders supported in VDMX as generators and effects inside of Motion and FCP X. This includes our 200+ standard set of assets, as well as all of the creations shared on https://www.interactiveshaderformat.com/
In this tutorial we will take a look at using these ISF generators and effects in Motion to create short video loops to use in VDMX or other VJ software.
Before beginning, make sure to install both Apple Motion and ISF for Motion from the Mac App Store.
Read More
The technique of using a blurry / zoomed version of an image in the background was popularized by the LiveLeak app and is now commonly used as a solution for working with mixed aspect ratio footage. Photo credit: fish2000
A question that comes up for modern live visualists is how to best work with footage that doesn’t match the aspect ratio that is being used for the final display. Two particularly common situations are when you may be working with older format video that is 4:3 and in other cases it could be camera phone footage captured in vertical mode. In some cases it is okay to use the basic “fill” option within the composition mode settings so that the imagery is scaled to fill the entire output area when needed, but another useful technique is the “LiveLeak” style effect where a blurry version of the image appears blown up in the background.
Read MoreCompleted project file for mixing between RGB->XYZ displacement images for the v002 Rutt Etra generator
Download the completed project file and media files for this tutorial.
One of the most fun visual effects used in live visuals and VJing dates back to the analog Rutt Etra video synthesizer which had many capabilities that have over the years been translated into digital form. Most notable among the digital versions is the v002 Rutt Etra Quartz Composer plugin which was developed by vade along with Bill Etra, one of the original creators of the Rutt Etra system. We’ve included this plugin as an FX, along with several other v002 creations with VDMX for a while now and, over the years we’ve seen a lot of projects that have made use of them… and in this video tutorial we’re hoping to teach you a few new tricks…
Before beginning this tutorial, make sure to use the v002 installer on the VDMX disk image in the Extra folder.
For this demonstration, instead of using the Rutt Etra as an FX, we have a Quartz Composer composition that generates video based on two input video streams. In this case we will create sets of hidden layers that will be used for these inputs: the Video Input will provide the colors for each particle and the Displacement Mask will provide RGB color gradients.
The main technique used here will be using an RGB to XYZ displacement map instead of a luma based displacement map. This means that we both have more flexibility for changing the positions of individual pixels, but we are now also responsible for creating the basic gradients that are needed to draw the basic image in a non-distorted way.
To accomplish this, we included special ISF compositions that are RGB color gradient generators. Each of these sketches includes three different gradients that are combined into a single output. One special feature of the gradient generators is the ability to pass in another video feed as a ‘look up table’ for one of the curves, making it possible to use waveforms and FFTs from audio analysis plugins, or any other image, as data that can modulate the output.
Once the basic setup is complete we can group the layers and add a Two Channel Mixer plugin to mix between different gradient outputs to fade the Rutt Etra generator between shapes and apply FX to the gradient images can create all kinds of different variations in the output. Here we use the VVMotionBlur effect to apply temporal smoothing to images and the Bad TV FX to create ripples.
Audio analysis, camera inputs and other images can also be used as lookup tables for the gradients.
The Video Input will provide the colors for each particle and the Displacement Mask will provide RGB color gradients.
Use basic gradients to create outputs without distortion.
In our introduction tutorial for working with Processing and VDMX we examined how to manually configure sending OSC messages from VDMX interface items to control properties of a Processing sketch running in the background. For this follow up lesson we will look at how the OSCQuery Helper tool can be used to publish the OSC address space for a sketch so that they can be remotely accessed by other software like VDMX or the OSCQuery Browser tool.
Read MoreCreating a GLSL based “Random Squares Mask” FX using the ISF Editor
Download the completed ISF FX from this tutorial
Update [Sept 28, 2019]: The custom masking effects created in this tutorial can also be used in Motion and Final Cut Pro X using ISF for Motion.
Image masking is one of the most powerful tools for digital visual artists when compositing multiple layers together. Masking makes it possible to partially or completely “remove” parts of an image so that it shows through to layers behind it. Though some media files contain embedded alpha channels for each pixel, this process is often accomplished by applying an FX that adjusts the alpha channel based on some parameters. Some of the more common masks that are used in VDMX for live visuals are:
Luma Key (Layer Mask): Adjusts the alpha channel based on the brightness level of pixels. Can use an optional external stream to provide values and includes controls for adjusting the brightness / contrast of the masking image before it is applied.
Chroma Key (Chroma Mask): Adjusts the alpha channel based on how similar the pixel color is to a specified masking color.
Shape Mask: Masks the image using patterns made from basic shapes such as triangles, rectangles, circles, diamonds.
Completed Checker Mask.fs used as an FX in VDMX
The Layer Mask FX in particular is extremely powerful because it can use a secondary layer as its input. This makes it possible to change the mask between source material, such as still images, movies, live cameras and video generators, as well as apply custom FX to the mask before it is applied.
As all of these FX themselves are written in GLSL in the ISF specification, they can also be easily remixed or used as starting points for creating new masking FX that adjust alpha channels based on different parameters. In this tutorial we’ll look at creating two custom masks: Checker Mask and Random Squares Mask.
Download the completed ISF FX from this tutorial
We’ll start by using the ISF Editor to create a new shader and saving it. Next, since this is very similar to the Checkerboard.fs generator, we can use that as a starting point – the code portion from this can be copied and pasted into the Checker Mask.fs, along with some of the input variables.
Only a few basic modifications to the code are needed. Here instead of using colors for the different sections of the pattern we’ll include sliders that map to the amount of alpha adjustment. The color information will come from an input of type ‘Image’ which is called ‘inputImage’ to match the FX protocol. Finally, make sure to set the category of the composition to Masking so that it is included with the other similar FX.
Once the FX is completed we can use it in VDMX; it shows up in the list under the Masking category and each of the parameters appears as the appropriate UI element that can be automated with control plugins or sync’d to MIDI / OSC / DMX inputs.
Download the completed ISF FX from this tutorial
For the next masking example we will begin is very much the same way. Our new FX is very similar to the Checker Mask we just created and it can be used as the starting point here. The main difference is that we will adjust the logic behind deciding which sections are masked and which are not by using some randomization.
Random value generator functions are not built into GLSL, but we can find an example of a fast pseudo-random function in the Noise.fs example that can be copied and pasted into our new FX. Other types of noise functions can be found in other shaders if you’d prefer to use those instead. Pseudo-random functions are useful here because we can get a unique output by providing the same seed value to the function. This makes it possible for a grouping of pixels that have different coordinates to access the same randomized number.
When including randomized values in shaders there are often two useful parameters to include for them that can be adjusted during live performance: (1) Random Seed (for changing the randomize pattern) and (2) Threshold (for controlling how much of the image is masked).
Here instead of using the checker pattern we will determine the index coordinates of square that contains each pixel, and use that as to generate a pseudo-random value; this value will be the same for every pixel in the same region. If this number is above our threshold, we apply our mask, otherwise, we leave the pixel alone.
One of the usual options you can include when making custom masking FX is a setting for how the alpha channel adjustment is applied. Consider these possibilities:
Additive: Adds the new alpha to the existing alpha value of the incoming pixel.
Multiply: Multiplies the new alpha with the existing alpha of the incoming pixel.
Replace: Overwrites the existing alpha of the incoming pixel.
Show Alpha: Useful for creating images that can be used as luma keys, this option sets the RGB of the pixel to the alpha level.
Examples of how these three different modes can be set up can be found in the Layer Mask, Chroma Mask and Basic Shape Mask FX mentioned above.
Another useful input parameter here can be an option to invert the behavior of the mask before it is applied.
Install custom FX to the “~/Library/Graphics/ISF” directory
Installed FX appear in their specified categories in VDMX
Visit the ISF sharing site for more starting points for custom masking FX
Minimalist design setup in VDMX
In this technique tutorial we’ll be looking at minimalism, one of the most popular styles used by live visual performers and VJs. Like the minimalist movement in painting and photography that was developed in the 1960s, this form is marked by its usage of geometric abstractions, negative space and mostly monochromatic color spaces. For live visuals, particularly when accompanying music, these ideas are often further extended into the dimension of time.
Read MoreOne of the most powerful tools for working with MIDI and OSC control data is Max, which is widely known for its easy to use interface for “patching” and working with data streams. While Max does not yet support OSCQuery natively, it is a great example of how the free OSCQuery Helper and MIDI OSCQuery Helper tools can be used to publish OSC and MIDI parameters from Max patches so that they can be remotely accessed by other software like VDMX and the OSCQuery Browser.
In this set of tutorials we’ll look at the process for adding basic MIDI and OSC inputs in a simple Max patch and then creating a JSON file that describes the routings. Once those are prepared we can see how to access these parameters using other software in the OSCQuery ecosystem.
Read MoreThe OSCQuery Protocol is a new specification that allows live performance tools to automatically communicate its parameters for rapid setup and improvisation between performers. Along with native support within VDMX here at VIDVOX we have developed several useful utilities that make it possible for people to take advantage of these new capabilities with software that support MIDI and OSC.
In this introduction tutorial we’ll be looking at how to use the free (and open source!) MIDI OSCQuery Helper utility to publish parameters from an Ableton Live project so that they can be accessed as browsable OSC parameters from other software such as VDMX. The MIDI OSCQuery Helper also includes its own built in Interactive Web Interface which can be loaded in web browsers on desktops, laptops, smart phones and tablets to remotely control any published controls.
Read MoreAlong with the basic controls of inverting values and applying basic math equations, number FX chains can used to adjust the values of data-sources before they are applied to sliders. In this example the 'Fall' FX will be applied to an audio analysis level to create a falling style before being applied to a VU meter generator.
Read More
