The new “Auto Colors Histogram” effect uses Synopsis histograms to automatically normalize colors.
The latest release of VDMX adds support for Synopsis, an exciting new open source project that puts the power of machine learning for video into the hands of new media artists and VJs.
Synopsis works by first analyzing each frame of your movie files and adding a metadata track containing information such as dominant colors and feature vectors. After the analysis is complete, a set of top level metadata characteristics are added to the file header, where they can be detected by Spotlight searches and applications that support Synopsis. You can read more about Synopsis on the Synopsis.video website and in our blog post announcing support in VDMX.
One of the particularly interesting things about Synopsis is that it is not designed as a solution to a single problem – it can be used in a variety of different ways within existing workflows, and opens doors to new possibilities that are yet to be discovered. Once your media files have been analyzed, the Synopsis Inspector app can be used as a starting point to browse, filter, sort and preview the metadata from movies. Within VDMX there are currently two main places where you can use Synopsis metadata for searching and filtering files: the File Browser window and the Media Bin plugin.
In this tutorial we will take an introductory look at how to use Synopsis analyzed movie files in VDMX, starting with an overview of the Synopsis Analyzer app.
Before beginning this tutorial…
Download the free Synopsis Analyzer and Synopsis Inspector apps: Synopsis Apps (requires 10.13 or later)
Download a sets of pre-analyzed movie files from our free sample media collections:
Note: The Synopsis Analyzer and Synopsis Inspector apps both require 10.13 or later. Pre-analyzed movie files will work in VDMX in 10.11 or later.
Analyze movies with the Synopsis Analyzer app
The Synopsis Analyzer is a free tool for batch analyzing and transcoding movie files. The main interface for the app contains three sections: the toolbar, the session list, and the session inspector.
The Synopsis Analyzer can also transcode to various media formats while it performs analysis.
Select clips to preview and get basic file information.
Files and folders can be added to the session list by clicking the + Add button, or by dragging from the Finder. The “Add Watch Folder…” option from the File menu can be used to select a directory in which any added media files are automatically converted. Once a session has been created, select it to display its settings in the inspector panel. Here you can adjust the following options for each session:
Encoding Preset: Specify if the audio and video tracks from each movie should passthrough, be removed, or converted into a different codec. Presets can be managed from the Preferences window.
File Locations: Specify where the new movie files will be created. This can be set to the same location as the source files, or a specific directory. If desired an additional directory can be set for the location where temporary media files are stored during the analysis process (useful when working with network drives).
Individual files within a session can also be inspected to reveal information about the movie file, such as the current codecs and duration.
When ready, click the ‘Start’ button in the toolbar to begin analysis on all sessions. To begin analysis on a single session, click the start button next to its name in the session list. If you are running the Synopsis Inspector app, as each session finishes analyzing a movie, you should see it automatically show up in its browser view.
The Synopsis Inspector
After your movie files have been analyzed you can begin to use the Synopsis Inspector app to browse, sort, search and preview the produced metadata for each clip.
The inspector interface is divided into three sections:
Toolbar: Provides buttons for re-sorting the browser and searching for tags.
Browser: Shows thumbnails for movie files. Right-click to access sorting options.
Inspector: Displays the global and per frame metadata for the selected file.
Items in the browser area can be dragged into other apps that support receiving standard file drops, making it useful as a general purpose tool for media management.
Using Synopsis Movies in VDMX
The final section of our video tutorial demonstrates some of the possibilities of using the metadata produced by Synopsis within VDMX, such as:
The VDMX File Browser window has been improved to make it easy to find Synopsis media based on their tags and by search.
Within the Media Bin plugin clips can be sorted by histogram, features, or using a hybrid mode, using the same algorithms as the Synopsis Inspector app. Click the Synopsis icon to immediately automatically trigger instead of sorting. Can be used with a specific selected media file, or relative to the currently playing movie on a layer.
Synopsis provides precomputed per frame histograms that can now be accessed by effects / generators and visualized by Preview Window plugins. See the included Auto Colors Histogram, Duotone from Histogram and Histogram Display ISF shaders for examples.
Searching for Synopsis tags works with compound searches using “AND”, “OR” and “NOT” terminology. For example, try searching for “gauzy AND pastel”.
This is of course just the beginning of what will be possible with Synopsis – check out the development roadmap to see what is coming next.