Some techniques for drawing text in GLSL

Rendering TG Character Test.fs in the ISF Editor utility.

While GLSL is not a particularly great language for rendering text strings, many shader developers have found it a fun challenge to do so anyway. Over the years we’ve run into several interesting blog posts and examples from people showing off different techniques that they’ve come up with. In this write up we are going to look at three ‘common’ approaches to drawing text in GLSL along with how the concepts can be adapted when making ISF shaders:

  • Using a bitmap image of characters as a lookup.

  • Encoding the bytes from a bitmap font in an array as part of the shader.

  • Individually drawing each character in GLSL.

As with most coding problems even within these general techniques there are several ways the objective can be solved, and no doubt other clever developers out there will continue to come up with other approaches that will leave our mouths agape.


Rendering text in GLSL using fonts encoded in bitmap images

The first technique we will look at is very well covered in the blog post Text Rendering by Jon Baker. describes how a monospaced font can be encoded into a bitmap image which is read from as part of the shader: “we're using a bitmapped monospaced font, where every font glyph is 6 pixels wide and 8 pixels tall.”

Within the shader itself each “cell” can determine its own local pixel coordinates and then look up the corresponding pixel from the provided bitmap image for the desired ASCII character to display.

ISF allows us to include images and reference them in the JSON blob, which you can find in the top portion of the JB Monospace Character Test.fs example.

/*{
    "CATEGORIES": [
        "Text"
    ],
    "CREDIT": "Jon Baker, adapted by VIDVOX",
    "IMPORTED": {
        "fontImage": {
            "PATH": "jbakermonospaced.png"
        }
    },
     "DESCRIPTION": "Test for drawing characters from a bitmap font stored as an image.",
    "INPUTS": [
 ...
    ],
    "ISFVSN": "2"
}
*/

The bitmap image containing the font.

This basic test shader provides us with a slider (float) with a range of 0 to 255 for selecting which character we want to repeat in each cell. It also includes a toggle button (boolean) for enabling the debug mode for visualizing the local pixel coordinates for each cell. When loaded into a host app like VDMX, it looks like this:

Rendering JB Monospace Character Test.fs in VDMX6 with the debugOverlay enabled to show the local pixel coordinates for each cell.

Building on the test shader we can create something more complex, such as the JB Monospace Random Characters.fs example which renders an image full of random characters each with a random color. For this ISF the debug option has been extended to show either the local coordinates for each cell or the coordinates of each cell within the whole grid.

Rendering JB Monospace Random Characters.fs in VDMX. Includes sliders for adjusting the random seeds of the characters independently from the hues.


Encoding the bytes from a bitmap font in an array as part of the shader

The character A encoded in 16 bytes, stored as an uvec4, that’s 4 uints with each 4 bytes, from Texture-less Text Rendering.

Another approach that is well documented involves taking the data from a bitmap font file and encoding the byte values directly into the shader as an array. Here we are going to mainly look at two blog posts:

  • Tim Gfrerer’s wonderful post Texture-less Text Rendering uses the PSF1 and an in depth explanation of their implementation as a GLSL shader.

  • Jon Baker has made a beautiful adaptation of Code page 437, an extended ASCII table which shipped with the IBM PC, written as a GLSL shader. In the blog post Siren: Masks Planes he details the process of how he came up with and implemented the idea.

The Texture-less Text Rendering approach described by Tim has four steps:

  1. Get the bitmap data from the font file.

  2. Embed the byte data as an array in the shader file.

  3. Use another array to hold the character code values for a word.

  4. Looking up the character data for each index in the word array.

Using the ISF Editor to translate syntax between variations of GLSL.

While ISF allows for a custom vertex shader, to keep things simple for future remixing, for the TG Character Test.fs adaptation we put everything in the fragment shader. Starting from the debug_text.frag in the GitHub repository referenced in the blog post, there were a number of minor syntax changes that were needed to adapt this for ISF. Fortunately they were all fairly easy to handle by hand using the error messages in the ISF Editor utility.

Like with the basic test shader using the bitmap lookup, before getting into Tim’s approach for displaying full words and phrases, it is useful to start with an example that just generates a display for every possible character, along with the ability to test displaying a specific character. This makes it easy to verify each part of the shader is working as expected and validate the drawing.

With this working, creating an extra constant to hold the full phrase for display and swapping it out for our debug code, we now have a full implementation of Tim’s technique in TG Message Test.fs. This trick for holding a phrase as character codes in an array can also be used along with the bitmap image approach above.

We won’t go into converting it to ISF, but a third example of this approach can be found in the debugtext.comp.glsl shader from the niagara project.

Code page 437 font.

Jon Baker uses a similar approach for adapting the Code page 437 font to use a single fragment shader. The blog post diving into how it was developed includes example code shared on ShaderToy. Converting this to ISF is a fun exercise. The big detail is that when using this in VDMX6, the ‘char’ variable is a restricted name by Metal, so we have to change that to something else. Fortunately the error log points this out for us when we try to load it. The adapted shader, JB Code Page 437 Character Test.fs, is set up to just repeat the same character in each cell.

JB Code Page 437 Character Test.fs loaded in VDMX.


Drawing individual characters with GLSL code

Perhaps the most tedious version approach that we will look at in this discussion is using code to render out specific characters that are needed. Two examples of this are included in the standard set of ISF shaders:

  • Digital Clock.fs

  • ASCII Art.fs

In Digital Clock.fs, only the digits 0-9 and a colon are needed for display, so while this can be time consuming to write, there are only a handful of cases to deal with. The functions found in this example can be very useful in situations where only numbers are appropriate to be rendered.

Function for drawing a digit in Digital Clock.fs.

Rendering Digital Clock.fs in VDMX with the Bad TV effect added.

In ASCII Art.fs, the code for each character rendered was created by movAX13h using a custom tool to generate the code snippets for the supported characters: “: * o & 8 @ #” – while this is not useful for general purpose text writing, it works great in this use case where only a few are needed to match up with different brightness levels.

Another example of this approach in action can be found in the Text with Truchets Demo on ShaderToy.


Why convert these to ISF?

ISF (the Interactive Shader Format) is a useful standard for creating ‘write once’ shaders that can be used across different host applications as generators and filters. It allows for playing with and remixing GLSL based compositions without the extra effort of having to build an environment for rendering shaders and all the related code to get a basic pipeline running.


Other approaches…

These are of course not the only ways to go about drawing text in GLSL, for example Jazz Mickle has a great post describing their bitmap font renderer for Unity that is worth reading. If you have seen any others that we should check out, please send us an email with a link!

Tracking faces, bodies, and hands with VDMX

The Video Tracking plugin provides an interface for detecting faces, bodies, and hands and using their locations as data-source values and masks that can be used to control virtually any part of VDMX.

In this tutorial we will look at using the face and hand tracking options in the Video Tracking to create a fun example of controlling the parameters of a simple ISF generator.

Download ISF Shader and Project Files here.



The main panel for the Video Tracking plugin contains a pop-up menu for selecting which video feed to analyze and a status indicator for the analysis.

For situations where multiple bodies and / or faces are detected, the provided 'prev', 'next', and 'rand' buttons can be used to switch between which is currently being tracked. When previewing the body and face tracking video streams clicking on a bounding box region can also be to change the active tracking.

The tracking options are as follows:

  • Human Tracking: Detects human bodies and publishes the position as data-sources. This includes the center position, width, height, and the x/y coordinates of each corner of the box bounding detected people. A boolean 'Detected' data-source turns on / off when bodies are found.

  • Face Tracking: Detects faces and publishes the position as data-sources. This includes the center position, width, height, and the x/y coordinates of each corner of the box bounding detected face. A boolean 'Detected' data-source turns on / off when faces are found.

  • Hand Detect: Detects one or more hands and publishes the center position as data-sources. Includes options for specifying the maximum number of tracked hands and the chirality type (All, Left, Right, or Pairs). A boolean 'Detected' data-source turns on / off when hands are found.

  • Human Mask: Generates a mask image that can be used e.g. with the Layer Mask effect to remove the background from images. Includes a quality setting (accurate, balanced, or fast). Also see the 'Remove Background' effect.

  • Attention Saliency: Generates a mask image using an Attention Saliency algorithm.

  • Object Saliency: Generates a mask image using an Object Saliency algorithm.

Where available the 'Publish Preview' option can be enabled for each tracking mode. This will create a video feed with overlays showing bounding boxes and other visualizations related to the analyzed images.

The "Minimum Latency" option in the inspector can be enabled to reduce latency (masks will be tighter) but takes longer to process (framerate may be lower, depending on the overall system load). This is option is off by default.

Hand tracking in VDMX6

If you have a LeapMotion or UltraLeap, and want to try the GECO application to share OSC data with VDMX. You can find it here: https://uwyn.com/geco/ We used the first generation LeapMotion controller in this tutorial.

For the iPad you can use Shoots Pro for sharing over NDI or Elgato Epoccam

Animating Properties of GLSL Shaders in VDMX

When writing GLSL shaders that run as generators or are used as image filters, one of the most fun parts of the process is playing with different control functions to animate all of the various variables that you've created in the composition. Using the ISF specification, GLSL shaders can publish their uniform variables so that host applications can provide user interface controls that can be connected to MIDI, OSC, DMX or other data-sources for automation.

In this tutorial we will look at adapting an existing GLSL shader into ISF, publishing some of its variables as uniforms, and loading the composition into VDMX where we will animate its properties using a variety of different plugins and MIDI input.

Read More

How to create a retro Halloween visual style in VDMX

Creating the right look for Halloween and other spooky themed events is one of those tricks that every visual artists needs to have in their go to bag of tricks. There are lots of different techniques that can be used and in this guest tutorial we are joined once again by Colin Evoy Sebestyen for a demonstration of how to use a combination of LUT based FX, real-time video generators and logo images to create a retro horror film graphic scene in VDMX. In particular this look is inspired by intro sequences like the one from The Gate and more recently Stranger Things.

Read More

How to do a NYE Countdown

One of the biggest nights for live visual performers is New Years Eve and in particular there is responsibility in particular that can be unexpectedly tricky to get right. That is handling the countdown to midnight. Often it can mean coordinating with other performers or workers at the venue. Depending on what is required of you during the show there are a few different approaches you may want to take when setting up your projects. It also helps to have an idea in advance of what your options are for running a visual countdown.

Read More

Creating and Installing ISF FX

An ISF, or “Interactive Shader Format” file is a GLSL fragment shader (.fs) that includes a small blob of information that describes any input controls that the host application (such as slider, button, and color picker controls in VDMX) should provide for the user when the FX is loaded for use, as well as other meta-data including the authorship, category and a description.

In this two part tutorial we'll cover the basics of applying ISF based FX to layers in VDMX and how to install new example ISF files you may download from the Internet, followed by a quick introduction to creating your own image processing GLSL fragment shaders.

Read More

Creating and Installing ISF Generators

An ISF, or “Interactive Shader Format” file is a GLSL fragment shader that includes a small blob of information that describes any input controls that the host application (such as slider, button, and color picker controls in VDMX) should provide for the user when the generator is loaded for use, as well as other meta-data including the authorship, category and description.

In this two part tutorial we'll cover the basics of using ISF generators within VDMX as sources for layers and how to install new example ISF files you may download from the Internet, followed by a quick introduction to creating your own GLSL fragment shaders.

Read More