Joe Maller: FXScript Reference: RGB and YUV ColorInformation about RGB and YUV (Y′CbCr) images and working with different colorspaces in Final Cut Pro. |
|||
Visit the New FXScript Reference and Joe's Filters sites. These pages will be phased out soon and may already be out of date. Up until I wrote Joe's Color Glow all my filters seemed to work fine in whatever color space they were given. But while trying to figure out the source of shimmering highlights, not-matching results and general wackiness, I noticed that all of Final Cut Pro's built-in keying filters were YUV aware. Since Color Glow is built around a keying operation, I started dissecting the built-in filters, looking for clues about when and how they used and switched between color spaces. Unfortunately there is virtually nothing documented about scripting YUV colors, so most everything on this page is based on my experiments and observations.
Diff'rent Spaces, Diff'rent NumbersMost computer graphics people are probably familiar with RGB, and most people with a video background are familiar with YUV. Hopefully this page can help to bridge any gaps. Final Cut Pro has three predefined color spaces constants available to scripts:
RGB can be thought of as three grayscale images (usually referred to
as channels) representing the light values of Red, Green and Blue. Combining
these three channels of light produces a wide range of visible colors.
Final Cut Pro contains two different RGB color spaces, History of YUV (Y′CbCr)YUV is the product of some amazing hacks. YUV uses RGB information, but it creates a black and white image (luma) from the full color image and then subtracts the three primary colors resulting in two additional signals to describe color. Combining the three signals back together results in a full color image. The engineers who invented this needed to find a way to make color television broadcasts backwards-compatible with black and white TVs. The color signal they came up with also needed to conserve bandwidth because three channels of RGB data would not fit into the limited broadcast signal space. By combining color information, YUV uses far less bandwidth than RGB and maintained compatibility with black and white TVs. YUV uses a matrixed combination of Red, Green and Blue to reduce the amount of information in the signal. The Y channel describes Luma (slightly different than Luminance), the range of value between light and dark. Luma is the signal seen by black and white televisions. The U (Cb) and V (Cr)channels subtract the Luminance values from Red (U) and Blue (V) to reduce the color information. These values can then be reassembled to determine the mix of Red, Green and Blue. Some deeper research into YUV reveals two reasons why Blue always looks so crummy when extracted from video images. The U channel ranges from Red to Yellow, the V channel ranges from Blue to Yellow. Because Yellow is Red and Green, Red is essentially sent three times, Green twice and Blue only once. Reconstructing the Luminance component reveals another reason Blue suffers, the Blue channel is only 11% of Luminance. The following formula shows the weighting of each channel in the Luminance mix:
The large percentage of Green and the small percentage of Blue (along with Green being sent twice) help to explain why chroma-keying for video is done against greenscreens and not bluescreens like film. Here are a few links to more information:
It's not often fussing around with things on a computer lets me directly play with anything created before I was born. (except for transistors, binary data, QWERTY, Basic... well, I still think YUV is cool.) Open SpacesAny FXScript can work in any of the three color spaces by adding the
following line to the definition section of the script, before the
This tells FXScript to look for and convert between RGB and YUV pixel formats where specified. Personal SpaceFXScript uses the
Final Cut Pro's built-in filters usually define a variable to contain
the format for comparison and formatting throughout the script. An example
which stores the Pixel Format of the image buffer
Because the value returned is really a number, the script uses a float to define a floating point variable. While this can save a few keystrokes and clean up the code a bit, I see no reason not to use Trading SpacesFXScript offers two ways of changing color spaces. Image buffers can be declared to be a certain color space or they can be converted to a specific color space.
Leaving Well Enough AloneAdding the info flag introduces another interesting problem. Final Cut Pro doesn't check the PixelFormat before applying a conversion operation, so it's possible for an image buffer to be double-converted. Double or mis-converting an image buffer causes really strange problems and artifacting. Because of this, it's important to trap format conversions inside of conditional statements, to make sure the script won't be applying a double conversion. The following conditional mode change is being used in Joe's Levels:
The goal of that snippet is to convert RGB images back to YUV. If the
images are not YUV already, the script first sets the format of One of the built-in YUV filters seemed to constantly check to make sure it's image buffers didn't mysteriously revert to another pixel format. It's a good idea to be extra cautious when working with YUV in FXScript, finding mistakes can take a long time. Assume NothingThe area I've had the most trouble with is forgetting the format of an existing image buffer. Consider the following code which will work sometimes but not others:
The reason that failed, resulting in a flashing green/pink mess, was that dest was never reassigned to match the colorspace of xbuffer. Since The fix for this problem is fairly simple, just add a setpixelformat command to reset the colorspace of
Manual Color ConversionFinal Cut Pro handles most of the conversions between YUV and RGB colors spaces, but these conversions are based on image buffers and not single color values. I couldn't figure out a more efficient way to change a color from RGB to YUV then converting every pixel in an image buffer. But after some searching around online, I found this page with the following formula for changing RGB color values into YUV color values:
Out of several formulas I found, this one seemed to work the best. Converting three numbers has to be faster than converting every pixel in an image buffer, so there is a definite advantage to those three seemingly arbitrary lines of numbers. Space ToolsJoe's PixelFormat Tester is an exploration filter that shows and reports on each of the pixel formats. I wrote it to help better understand the changes between YUV and RGB color spaces. The filter is a part of my free set of Exploration and Debugging filters, available with Joe's Filters. Links to more YUV informationThe following sites helped me understand what is happening with YUV color:
TerminologySince writing this document, I've learned the terms "YUV" and "luminance" are largely inaccurate. Video luminary Charles Poynton makes an elegant case for proper terminology in his article YUV and luminance considered harmful. My uses are based on what I found online to correspond with the information in Final Cut Pro's documentation and example scripts. Based on Mr Poynton's article, these terms seems inaccurate. I've thought about the propagation of misinformation before this, and it's a serious problem. For example, there is no "sweet-zone" on your tongue, every part can taste every flavor, but that bogus theory is represented in classroom textbooks today even though it's been proven wrong for 50 years. While this document concerns terminology associated with Final Cut Pro, these concepts extend far beyond this one program and using the correct terminology will help myself and everyone else outside of the Final Cut Pro universe. I will continue using YUV to refer to Final Cut Pro's image buffers since the FXScript format name uses this term. Luminance refers to true CIE Luminance, while Luma describes Y′. In most cases these should be interchangeable, but they are slightly different. Before reading Mr. Poynton's article, I assumed Luma was just shorthand for Luminance. While my grasp on these terms is not 100%, I'm making an effort to use the correct terminology. Other notesI've noticed that most all of the color modes seem to work regardless
of the color space they are given, but of course, there are exceptions.
Add, Subtract and Difference freak out when applied to YUV color. The
reason for this is that YUV's color channels work outward from the middle,
instead of straight 0 -255 like RGB. When values are subtracted outside
of the allowable YUV range, dark colors turn dark green. When pushed above the
maximum allowable limit, bright colors turn pink. I've also seen RGB images
exceed their color allowance and their bright pixels seem to vibrate on the video
output. This RGB vibration is often caused by conversion through the ConclusionSwitching between RGB and YUV isn't the nightmare I was afraid it would be. Now that I have something of an understanding of what is happening, and can recognize the wacky color symptoms of incorrect pixel formats, it's gotten easier add YUV support to my filters. Adding the Y only checkbox to Joe's Levels took less than an hour to implement, figuring out what was happening on Joe's Color Glow the first time took several days. |
The FXScript ReferenceFXScripting Joe's FiltersJoe's Filters for Final Cut Pro Building Joe's Minimum Maximum Joe's FXScript ExplorersJoe's Debug and Explore Filters Joe's 3x3 Convolve Matrix Tester Joe's 3x3 Matrix Values Tester Other FCP Stuff
Film & VideoRecording the sound of PhotographyProjectsFinal Cut ProDesignWRITINGSite Notes Archive (weblog) Web ResourcesAbout Joe MallerPast Home Pages Etc.Search joemaller.comContact Me
|
||