Questions can be sent to: [email protected]
Overview
This guide is part of a series on VPixx Technologies’ specialty high bit depth video modes. In this guide, we will cover M16 mode, which offers high-bit-depth greyscale and up to 256 custom colour overlays. M16 mode also allows the user to display content selectively on either the stimulus display or the console display using transparency colours.
M16 mode can be used with:
- the first generation DATAPixx video I/O hub and a CRT monitor
- our VIEWPixx and VIEWPixx /3D CRT replacement monitors, and
- our PROPixx projector
This guide starts with a brief introduction to the concept of bit depth and why it is important for vision scientists. After that we will discuss M16 mode in detail and show how this mode can be implemented in MATLAB/Psychtoolbox.
Some of our specialty modes have already been showcased in our virtual talks and MATLAB demo library. This VOCAL guide series is intended to complement those resources, explaining the general principles of each mode, along with helpful illustrations and tips. Throughout this guide we will post links to our other resources in case you would like to dive deeper into specific modes and how they are implemented. Of course, if you have any questions you can always reach out to our team of trained vision scientists at [email protected].
A brief introduction to bit depth
Digital images are comprised of millions of picture elements, or pixels, which form the smallest addressable component of an image. On a standard display, a pixel is defined by a triplet of red, green and blue (RGB) colour values. While the exact method by which these RGB components are physically expressed varies by screen technology, at a broad level we can say that the RGB values of a pixel determine the final colour value for that pixel.
Each of the R, G and B components of a pixel represent a separate colour channel. Colour channels have a minimum and a maximum saturation, determined by the screen’s physical properties. The range of colour saturation a screen can show is known as the screen’s gamut.
The full gamut of a given colour channel, from minimum to maximum, can be expressed as a continuous variable ranging from 0 to 1. For hardware reasons, there is a maximum precision with which we can select a colour value from this range. Displays that use an analog video signal, like CRT monitors, can achieve very high precision, which makes them valuable tools in colour research. By contrast, modern digital displays like LCD panels and LED projectors are constrained by the number of bits assigned to each colour channel by the digital video signal and the hardware itself.
Imagine that we have only one bit of data in our video signal for each colour channel. A bit can have two states, either on or off. We will denote these in binary as 0b1 (‘on’), and 0b0 (‘off’). The prefix ‘0b’ indicates a binary number. In a 1-bit colour channel, we have 2^1 states available that we can use to segment the colour gamut.
If we add a second bit of data, or bit plane, to our signal, we can describe the colour with greater precision. Now, we have 2^2 unique states available (0b00, 0b01, 0b10, and 0b11), and we can divide our spectrum of possible channel outputs into 4 discrete levels.
The number of bit planes available to define a colour channel is known as bit depth or bits per colour (bpc). Most modern displays are 8 bpc. This means the display can divide the colour space into 2^8 or 256 distinct levels of R, G and B. Some programs represent these steps as a proportion of maximum (e.g., 0-1) or as a raw value (0-255); regardless of the way the value is represented, the level of precision is the same.
It is important to remember that R, G and B colour channels are independent of one another, and combine to form the final colour output. A display with 8 bpc has a total of 24 bits per pixel, and can therefore show over 16 million unique pixel colours.
When we represent values in binary, the position of the bit is important. Bits further to the left in a binary number are considered more “significant” as they have a larger impact on the total value. For example, if we have 4-bit value like 0b1000 and we flip the bit in the least significant position to create 0b1001, this is equivalent to an increase from 8 to 9 in base 10 counting. If we flip a more significant bit, like 0b1100, we jump from 8 to 12.
The more bit planes we add to our video signal, the more precisely we can define a colour within our gamut. For example, a 10-bit display divides each colour channel into 2^10 or 1024 levels. A 12-bit display has 2^12 or 4096 levels. As the number of bit planes increase, flipping the least significant bit specifies smaller and smaller steps in colour output.
To underscore this point, high-bit-depth displays do not show richer or different colours than a standard 8-bit display. A higher bit depth simply allows you to divide your colour space into finer increments.
So why is this important for vision scientists? Under certain experiment situations, the human visual system is sensitive enough to differentiate between single-step increments in 8-bit displays. Detectable changes are emphasized by the visual system as edges, and this results in a perceptual artefact known as Mach Banding [1]. Mach Banding can be a nuisance for researchers studying colour, luminance and contrast perception. High bit depth displays are a valuable tool for avoiding such artefacts, as they divide the display gamut into extremely fine-grained increments that cannot be detected by the viewer.
Unfortunately, adding additional bits to the video signal requires bandwidth, and certain video protocols cannot exceed 8 bpc at full resolution. This is fine for most commercial applications, as modern screens are not capable of showing more than 8 bpc due to hardware limitations. Graphics processors may also employ strategies like temporal or spatial dithering to try and mitigate effects like banding, making them less noticeable.
On the other hand, specialty displays like the VIEWPixx /3D are capable of 10 bpc, and the VIEWPixx and PROPixx can achieve 12 bpc. An analog CRT monitor driven by a DATPixx video I/O hub converting digital video output to an analog signal can reach even higher bit depths. Therefore, we have the challenge of taking a video protocol restricted to 8 bpc and converting this video into a high bit depth image these displays can use.
M16 mode does this by re-assigning the default 8 bits of the red and green colour channels of a given pixel into a single, 16-bit grayscale value that is passed to the display hardware. The 8 remaining bits of the blue channel can then be used to index a colour lookup table which permits up to 256 custom colour overlays.
Before we describe this mode in detail, there is one principle that must be highlighted:
The final output of any high-bit-depth video mode will depend on the physical capabilities of the screen.
While some of our special video modes allow for up to 16 bpc, this refers to the capacity of the digital video signal. If this signal is passed to a VIEWPixx /3D, the final bit depth of the display will be 10 bpc, because that is the maximum depth the VIEWPixx /3D can physically show. The 6 least significant bits will be ignored because the screen hardware is incapable of showing increments that fine.
When we talk about high-bit-depth modes, we typically describe their bit depth based on the bits available in the digital signal. This value remains constant across display types. It is good to keep in mind the physical limitations of the screen you are using when assessing what the output of these modes will be.
M16: 16-bit grayscale, full resolution, optional colour overlay
M16 allows the user to display in high-bit-depth grayscale, at any supported resolution (e.g., full HD 1920 x 1080) and 120 Hz refresh. It works by concatenating the 8 bits of the red and green colour channels of a given pixel into a single 16-bit grayscale output.
The remaining 8 bits of the blue colour channel may be optionally used to index a user-defined colour lookup table. This table contains up to 256 custom 16-bit RGB pixel values. Images assigned a row in this table will be drawn in the designated colour, as an overlay on the display.
To use M16 mode you will need to ensure your image drawing software is capable of specifying 16-bit values. In MATLAB, the PsychImaging tools available through Psychtoolbox include a function to instruct drawing commands to use 32-bit floating point values. Enabling this setting allows users to exceed standard 8 bpc colour value assignment.
Example 1: Drawing a gray rectangle in M16 mode
MATLAB/Psychtoolbox
%Establish the correct color and video settings
AssertOpenGL;
PsychImaging('PrepareConfiguration');
PsychImaging('AddTask', 'General', 'FloatingPoint32Bit');
PsychImaging('AddTask', 'General', 'EnableDataPixxM16OutputWithOverlay');
%Open a full screen window with a black background
screenNumber = max(Screen('Screens'));
[win, winRect] = Screen('OpenWindow', screenNumber, [0,0,0]);
%Determine our target gray value as a proportion of the maximum possible value
max16BitGray = 65535;
myGrayLevel = 1026/max16BitGray;
%Draw our rectangle in the top left corner of the window and flip
Screen('FillRect', win, myGrayLevel, [0, 0, 400, 400]);
Screen('Flip', win);
Example 2: Adding an overlay window
The following example builds off of example 1. Here we are adding code to generate some red text on the stimulus display.
To add colour as an overlay, we must first define a colour lookup table, and open a special overlay window. Then we will draw our text to the overlay window with the appropriate index to our lookup table. Note that this lookup table is expecting RGB values with 16 bpc (i.e., 0-65535 levels, expressed as a proportion from 0-1).
MATLAB/Psychtoolbox
%Generate our colour look-up table (CLUT) and load it into our hardware
%The clut is first populated with black and then the first two rows are changed to blue and red
myTable=repmat([0,0,0], [256,1]);
myTable(1,:) = [0,0,1];
myTable(2,:) = [1,0,0];
Datapixx('SetVideoClut', myTable);
Datapixx('RegWr'); %straight calls to our Datapixx API require an explicit register write or update
%Get our overlay window
overlay = PsychImaging('GetOverlayWindow', win);
%Let's put some blue text in the overlay. The last argument here is the index of our lookup table
DrawFormattedText(overlay, 'Here is some text', 'center', 40, 1);
Screen('Flip', win);
Example 3: Adding a transparency and a separate lookup table for the experimenter console
It is possible to designate a specific RGB triplet in your lookup table as a “transparency” colour. Drawing text or shapes in this colour will cause the display to ignore the overlay, and draw the image in the 16-bit grayscale value specified by the red and green channels.
This is particularly useful if you have a second monitor connected to the Video Out 2 or Console Out port on your VPixx hardware. This second monitor shows a copy of the stimulus display, and is generally used as a “console” or experimenter view. Because this view is a copy of the video signal to the stimulus display, it does not put any additional pressure on the control PC’s graphics card and is therefore ideal as an experimenter view during data collection.
Using overlay transparencies is a way to show unique content on the console without taxing your graphics card. For example, a researcher might want to keep track of participant performance in their view, but hide this information from the participant. The PLDAPs Toolbox (pronounced “platypus”) by Eastman and Huk [2] uses this strategy to show real time eye tracking data and trial metadata during experiments.
To add unique content to the console, a second 256 x 3 16-bit RGB colour lookup table must be appended to first table. This second table is populated with a different set of RGB triplets that will show on the console monitor.
Next, when drawing content to the overlay, we use a colour index that points to a row in the stimulus display lookup table that contains the transparency colour. The same row in our second table contains a non-transparency colour. In this case, so the overlay text is drawn in grayscale on the stimulus display but in the overlay colour on the console.
MATLAB/Psychtoolbox
%Let's set an arbitrary transparency colour of full green
myTransparency = [0,1,0];
Datapixx('SetVideoClutTransparencyColor', myTransparency);
Datapixx('EnableVideoClutTransparencyColorMode');
Datapixx('RegWr');
%Create a copy of the first CLUT, and populate row three with our transparency (stimulus display)
%and full blue (console)
myConsoleTable = myTable;
myTable(3,:) = myTransparency;
myConsoleTable(3,:) = [0,0,1];
Datapixx('SetVideoClut', [myTable; myConsoleTable]);
Datapixx('RegWr');
%Draw our text with index 3
DrawFormattedText(overlay, 'Trial 2 begins', 'center', 40, 3);
Screen('Flip', win);
Transparency colours can also be used in the console lookup table, if you want to do the reverse of this example and show something only on the stimulus display instead.
For more examples of M16 mode, please see our MATLAB demo on Understanding CLUTs, M16 Mode, and Color Transparency.
M16 mode allows you to set a theoretical 65, 535 levels of gray. The final output will depend on your display’s physical limitations. For example, the PROPixx DLP projector will ignore the four least significant bits and show 12-bit grayscale. For more details, please see the final note at the bottom of the introduction to bit depth.
References
[1] Ratliff, F. (1965). Mach bands: quantitative studies on neural networks. Retina. San Francisco, CA: Holden-Day.
[2] Eastman KM and Huk AC (2012) PLDAPS: a hardware architecture and software toolbox for neurophysiology requiring complex visual stimuli and online behavioral control. Front. Neuroinform. 6:1. doi: 10.3389/fninf.2012.00001
Cite this guide
Fraser, L., (2021, March 29). High Bit Depth Grayscale with M16 Video Mode. Retrieved [Month, Day, Year], from https://vpixx.com/vocal/M169