Sending Triggers with Pixel Mode

Questions can be sent to: [email protected]

Overview

This guide is an introduction to sending digital TTL triggers using VPixx Technologies’ “Pixel Mode.” While in Pixel Mode, your VPixx device continuously generates digital outputs based on the colour of the top left pixel of the display. By changing the colour of this specific pixel on a given frame in an experiment, researchers can send frame-accurate triggers to their recording system of choice. Pixel Mode can replace and enhance a photodiode-based triggering system—no programming required!

The guide includes an introduction to TTL triggers and their use, an explanation of digital outputs and how to set and interpret them, and finally, instructions for how to use Pixel Mode with your VIEWPixx /EEG. The guide ends with some useful tips, and instructions for how to use Pixel Mode with your other VPixx devices. 

Introduction to TTL triggers

A common requirement in research is to be able to relate neurological, physiological and behavioural responses to the onset of a visual event.

Suppose we are conducting an electroencephalography (EEG) experiment examining neural responses to pairs of objects varying on one or more feature dimensions. Specifically, we are interested in event-related potentials that occur in the first 250 ms following the appearance of these objects on our display.

In our EEG data, we need a way to identify, or mark, when the objects appear onscreen, so that the EEG data can be time-locked to it. The more consistent and reliable this segmentation is across trials, the more confidence we have in aligning multiple trials’ worth of data and averaging our results. This event marker is what supports trial-averaging, and the extraction of the event-related activity sequence. 

If the effect we are looking for is small, better time-locking may give us the statistical power we need to actually detect it. So ideally, our marker of stimulus onset should have very little variable error across trials; in other words, it must be precise.  

It would also be helpful if the marker is sensitive to the kind of objects being displayed. For example, the marker could signal what trial type is being presented, what iteration of this trial type is being presented, and details of the feature dimensions which vary. This information becomes useful when sorting and categorizing EEG data during analysis.

How do we communicate temporally precise, informative markers from our display to the EEG? We use transistor-transistor logic (TTL) triggers. These triggers are simple, rapid digital signals that can be sent between systems via inexpensive cables. TTL triggers are recorded by our EEG system, to help characterize our EEG data.

In general, we need to be careful of timing issues introduced by the computer. The experiment software, the processing done by the graphics processing unit (GPU), and the computer operating system can add delays that create a variable discrepancy between when the computer reports that a visual event has occurred, and when the visual information actually appeared on the screen.

In some experiments, those discrepancies are small enough to be negligible, on the order of a few milliseconds. However, since time-averaging benefits from precise time-locking, we want the most precise event marker for our stimulus onset that we can get. 

 One way to improve precision is to bypass our computer altogether and create a trigger based on when the pixels on our monitor physically change. We can mount a light-sensitive device called a photodiode on one corner of our monitor.  A photodiode amplifier can send out a TTL signal when it detects a change in the luminance of that patch of screen. If we change the patch luminance on the same video frame in which the stimuli appear, we have a fast and reliable measure of when the stimulus actually showed up on the display.

While economical and effective, photodiode setups take up space and require a bit of legwork to get running. They can interfere with full-screen stimuli like natural scenes, or search tasks. Perhaps most critically, changing luminance patches on your display can be a source of distraction for your participant.

TTL triggers are a way to insert markers of visual event onsets into our recording data. Ideally, we want these triggers to be precise, informative and minimally obtrusive. VPixx has designed a special triggering mode called “Pixel Mode,” which serves a similar purpose as a photodiode, but offers customizable triggers with no extra hardware, minimal onscreen distractions, and no programming required. In the next section, we will cover the basics of how Pixel Mode operates.

Principles of Pixel Mode

In a standard experiment setup with an LCD display, the video signal is passed from your computer’s graphics processing unit (GPU) to the display via a video cable. When the signal reaches the display monitor, multiple frames may be stored in a memory buffer before being shown on screen. 

When an image is ready to be shown, a current is applied to the liquid crystal layer of the display, changing the polarization of molecules inside the layer. The extent of this polarization determines how much light passes through each pixel.

 This polarization can take time, and so there is a delay before the desired pixel luminance is reached. We refer to this time as the “stabilization” of the pixel. Exact stabilization time depends on the total change of luminance from the previous frame to the current frame, i.e., the total change in polarization required. Here is an example of pixel illumination as it transitions from black to white on a 120Hz display (VIEWPixx in standard display mode), as measured by a photodiode:

Our VIEWPixx LCD displays have a special scanning backlight mode so that the pixel does not receive full illumination until it has already stabilized. This leads to better control over the luminance of the display. In scanning backlight mode, pixel illumination profiles look like this:

VPixx displays also come with an onboard field-programmable gate array (FPGA) chip which intercepts the video signal after it is sent from the computer GPU. In Pixel Mode, the FPGA generates a trigger based on information in the video signal. This trigger is available on the Digital Output DB-25 connector, and the video signal continues to the display. Here is a simplified schematic illustrating the process:

Because we know exactly when that TTL trigger is sent, and we know the illumination profile of our display, we know the exact temporal offset between the trigger and stabilization of the pixel in the top left corner of our display. On the VIEWPixx displays, this is exactly 6 milliseconds.

Using the FPGA we can send a highly precise frame-based trigger to our recording system. We define the content of this trigger so it is informative as well, by changing the colour of that top left pixel and reading the [R, G, B] values as digital outputs. We will cover how this works in the next section.

Pixel Mode was developed for the VIEWPixx /EEG; however, it can also be implemented on other devices like the DATAPixx data acquisition system and the PROPixx controller. For more information about using Pixel Mode with these other systems, please see the section on using Pixel Mode with other devices.

Defining triggers using Pixel Mode

The Digital Output DB-25 connector on your VPixx device has 25 physical pins, numbered 1-25. With the exception of ground pin 13, each pin is capable of transmitting a low or a high voltage output. More simply, we can say that each pin can send a value of 0 (low) or 1 (high). Therefore, the Digital Output connector can theoretically send 2²⁴ unique outputs. TTL pulses or “triggers” refer to a change in this output state, signalling a new event.

The assignment of pin numbers 1-25 on the Digital Output DB-25 connector is based on an industry standard, and doesn’t necessarily reflect the way any device uses the pins. VPixx relabels these pins as “Digital Outputs” for programming purposes. In Pixel Mode, since we will use the colour of a pixel to select which pins we want to trigger, we break the pins into three sets of 8 for red, green and blue. For simplicity, in this guide we have labelled them 7-0.

The digital outputs correspond to the pin shown in the same position. For example, if I set Red 1 to high, I am raising the voltage on physical pin 14. It is important to keep in mind that other manufacturers may label pins differently in their own applications.

Each of these pins can be in a state of 0 or 1. We can write the pattern of low and high states out from position 7 to 0, like so:

These states can be read as binary numbers (base 2, 0-1). If we convert these binary numbers into decimal (base 10, 0-9) we get Red 160, Green 1, and Blue 170. 

To set a specific trigger for a given frame of your display, you simply need to set the colour of the top left pixel of that frame to the [R, G, B] values that correspond to your desired digital output. You can use whatever program you like to do this—MATLAB, E-Prime, or even PowerPoint. As long as you can control the colour of that top left pixel, you can use Pixel Mode. Here’s what our example looks like, with the pixel size exaggerated:

Keep in mind that your recording system is completely agnostic to colour. In order to decode the trigger as it is read by your recording system, you will need to know how your system treats the 24 TTL pins.

The important thing to remember is that your [R, G, B] values, when converted to binary, correspond to the state of the digital outputs on the DB-25—where 0 is low and 1 is high.

If you are unfamiliar with counting in binary, you can use online converters (e.g., https://codebeautify.org/decimal-binary-converter) to convert between binary and decimal. Some software has functions to convert decimal to binary and back. MATLAB and Excel use bin2dec()and dec2bin(). Similarly, Python uses int()and bin().

Pixel value lookup table

You can work out what [R, G, B] values you need to send to raise the voltage on specific pins by using binary and the parallel port/digital out assignments, as we did above. You can also use our handy lookup table to figure out what values you will need to trigger specific pins:

Pin	DOut (VPixx)	Colour Channel #	Value to add to colour channel
1	0	Red 0	1
14	1	Red 1	2
2	2	Red 2	4
15	3	Red 3	8
3	4	Red 4	16
16	5	Red 5	32
4	6	Red 6	64
17	7	Red 7	128
5	8	Green 0	1
18	9	Green 1	2
6	10	Green 2	4
19	11	Green 3	8
7	12	Green 4	16
20	13	Green 5	32
8	14	Green 6	64
21	15	Green 7	128
9	16	Blue 0	1
22	17	Blue 1	2
10	18	Blue 2	4
23	19	Blue 3	8
11	20	Blue 4	16
24	21	Blue 5	32
12	22	Blue 6	64
25	23	Blue 7	128

For example, if you wanted to trigger DOut 5, 17, and 21, you would add 32 to the R channel, 0 to the G channel and 34 (2+32) to the B channel. Therefore the pixel colour would be [32, 0, 34].

Choosing good trigger values

Some graphics cards have a feature called dithering which can alter R, G and B values by +/- 1 after the value has been set by the user. This can inadvertently disable your triggers, and may have serious consequences for your experiment! For this reason, it is good to think carefully about what triggers you should use to signal events.

It is sometimes possible to disable dithering, but a better approach is to choose your triggers defensively. First, and easiest, is to define events by a range of codes, i.e., your desired code +/- 1.

Alternatively, you can ignore certain pins, and take care to avoid event codes which have a major impact on digital output. Consider Red “126” vs. “127” vs. “128”:

[0111 1110] = 126
[0111 1111] = 127
[1000 0000] = 128

If you are only concerned about the states of red 4, 5, and 6, then dithering will not be an issue for you if you use a value like 126 or 125; however, you will still need to avoid values on the cusp of a major change in the binary representation of the number, like 127 and 128.

Always double check your trigger output before collecting experiment data! In addition to dithering, some software can add overlays which may alter pixel values, particularly around the edges of the screen. Screen recording software in particular often does this. Other forms of pixel manipulation (e.g. blending, antialiasing, filters or gamma correction) can also affect the final pixel identity used to generate your TTL output. 

Enabling Pixel Mode

Pixel Mode is always enabled on the VIEWPixx /EEG. Simply connect the monitor to your computer, and the monitor’s digital output to a recording device, and triggers will be sent automatically.  For details on how to enable this mode with other VPixx devices, see the section on using Pixel Mode with other devices.

A final note on timing

Pixel Mode provides microsecond-precise triggering based on when the video signal reaches your display. As we have seen, the trigger begins 6 ms before the top left pixel is illuminated. 

On our CRT replacement monitors, which use a scanning backlight, exact timing of pixel illumination is location-specific. The backlight scans down the screen, creating a temporal offset between when the top left pixel and the bottom right pixel of the display are illuminated. This is the same behaviour you would observe in a true CRT monitor. Of course, this delay is always less than the screen refresh rate (8.33 ms/frame at 120Hz).

If you want to characterize the exact offset between your trigger (based on the start of the video frame), and the illumination of the specific patch of pixels where your stimulus appears, you will need to perform a one-time measurement using a photodiode placed over that stimulus’ position on the screen. This will give you the offset between the trigger and the spatially-dependent onset of your stimulus. 

If you choose to perform this measurement, ensure you are using the same refresh rate that is used in your experiment.

Using Pixel Mode with other VPixx devices

VIEWPixx and VIEWPixx /3D

Functionality is almost identical to the VIEWPixx /EEG; however, it is important to note that the 6 ms offset between trigger and pixel stabilization only applies to VIEWPixx/EEG and VIEWPixx/3D displays with scanning backlight enabled. If the backlight is disabled, the profile of the LCD illumination changes and you will need a photodiode to calibrate the new temporal offset.

PROPixx DLP Projector

Pixel Mode can be used with the PROPixx and the Digital Output DB-25 on the PROPixx controller. The temporal relationship between the trigger and the display is different due to the display technology. The projector uses a digital micromirror display to show the entire image simultaneously, exactly one video frame after the Pixel Mode trigger is activated. At 120 Hz this offset between the trigger and illumination of all pixels on the display is precisely 8.33 ms. 

 DATAPixx2 and DATAPixx3 I/O hubs and a third-party display

It is also possible to use Pixel Mode with a commercial third-party display, provided the video is passed through a DATAPixx2 or DATAPixx3 system. In this case, you will have to determine the temporal characteristics of your display by comparing the timing of the digital output to photodiode measurements of your screen illumination. Commercial displays typically do not have consistent timing across different pixel colour changes, so its best to measure the onset of your specific stimuli, presented in the same manner as in your experiment. Pixel Mode is not supported on the original DATAPixx I/O hub. 

Enabling Pixel Mode with PyPixx program 

Pixel Mode is enabled automatically on the VIEWPixx /EEG. It can be enabled and disabled on other devices using the PyPixx program.  Simply connect your computer and open PyPixx. PyPixx will automatically detect your VPixx device. In General > Configuration > Digital Output Configuration, there is a dropdown list with Pixel Mode as an item.

Whenever Pixel Mode is enabled, it will automatically send triggers based on the current video output. It will remain enabled until you revert to normal mode, or power off the device.

You will also notice that there is a “Pixel Mode GB.” This is simply Pixel Mode with only the green and blue channels activated, allowing for advanced applications. The 8 digital outputs associated with the red channel are made available for triggers coming from non-visual sources (like button presses) or when the DB-25 cable is split up into several outputs.

The method for determining the [R, G, B] values in Pixel Mode GB is the same as Pixel Mode, but the red colour channel will not impact the video-based triggering.

Enabling Pixel Mode via VPixx software tools

For the VIEWPixx, VIEWPixx /3D, PROPixx and DATAPixx devices, it is possible to enable Pixel Mode from within MATLAB or Python. Example code is shown below. For more information, please see the code documentation.

MATLAB/Psychtoolbox
Python (libdpx wrapper)
Python (object-oriented)
MATLAB/Psychtoolbox

Datapixx('Open');
Datapixx('EnablePixelMode');
Datapixx('RegWr');
%Experiment code
Datapixx('DisablePixelMode');
Datapixx('RegWr');
Datapixx('Close');

Python (libdpx wrapper)

from pypixxlib._libdpx import DPxOpen, DPxClose, DPxDPxEnableDoutPixelMode 
from pypixxlib._libdpx import DPxDisableDoutPixelMode, DPxWriteRegCache
DPxOpen()
DPxDPxEnableDoutPixelMode()
DPxWriteRegCache()
#Experiment code
DPxDisableDoutPixelMode() 
DPxWriteRegCache()
DPxClose()

Python (object-oriented)

from pypixxlib.datapixx import DATAPixx3
#substitute DPx3 with relevant device
myDevice = DATAPixx3()
myDevice.dout.enablePixelMode()
myDevice.writeRegisterCache()
#Experiment code
myDevice.dout.disablePixelMode()
myDevice.writeRegisterCache()

Remember that changing the device output configuration via code will not change the settings on the device until a Register Write command has been called.

Notes

Stimuli shown in the animation in the introduction are “Quaddles.” See:

Watson, M. R., Voloh, B., Naghizadeh, M., & Womelsdorf, T. (2019). Quaddles: A multidimensional 3-D object set with parametrically controlled and customizable features. Behavior research methods, 51(6), 2522-2532.

Cite this guide

Fraser, L., & Kenny, S. (2020, May 6). Sending triggers with Pixel Mode. Retrieved [Month, Day, Year], from https://vpixx.com/vocal/pixelmode