Joe Maller: FXScript Reference: Joe's Default Composite Controls

This page describes how I built the set of composite controls which appear in almost all of my filters. There are two versions of the composite controls set, the simple one, and one for YUVaware filters.

Composite Foundation

I've been working with Photoshop since 1992, maybe even earlier. When I started building my filters for Final Cut Pro and the time came to add compositing modes, the basic Photoshop set was a no-brainer.

Add and Subtract are not standard Photoshop composite modes, but back before the dawn of layers, we had to composite using Photoshop's Calculate command where Add and Subtract were options. They are also available in After Effects and common in most other image manipulation applications.

Most of these modes were already supported in FXScript and work as expected. However a few modes required extra tweaking to preserve the source clip's alpha channel. Missing from the list are Soft Light and Hard Light. These modes are not supported by FXScript, but I'm looking into other ways of implementing them.

Basic Set

Below is the code for the basic set of compositing controls. My filters place each if statement on it's own line, the statements are expanded here for clarity.

In the Definition section of the script, add the following two inputs:

input ApplyAs, "Mode", Popup, 1, "Normal", "Multiply", "Screen", "Overlay", "Lighten", "Darken", "Add", "Subtract", "Difference";

input Opacity, "Opacity", slider, 100, 0, 100;

The ApplyAs pulldown menu returns a number corresponding to the selected composite mode. Opacity is a slider from 0 - 100, dividing this value by 100 results in a percentage which will determine the mix opacity.

In the following statements, the image buffer xbuffer contains the results of previous filtering, src1 is the original image and dest is the target image buffer which will display the result of the script.

if ApplyAs ==1;
   Matte(xbuffer, src1, dest, opacity/100, kalpha);
end if

if ApplyAs ==2;
   Multiply(src1, xbuffer, dest, opacity/100, kalpha);
end if

if ApplyAs ==3;
   Screen(src1, xbuffer, dest, opacity/100, kalpha);
end if

if ApplyAs ==4;
   Overlay(src1, xbuffer, dest, opacity/100, kalpha);
end if

if ApplyAs ==5;
   Lighten(src1, xbuffer, dest, opacity/100, kalpha);
end if

if ApplyAs ==6;
   Darken(src1, xbuffer, dest, opacity/100, kalpha);
end if

if ApplyAs ==7;
   add(src1, xbuffer, dest, opacity/100, kalpha);
end if;

if ApplyAs ==8;
   subtract(src1, xbuffer, dest, opacity/100, kalpha);
   ChannelCopy(src1, dest, kalpha, knone, knone, knone);
end if;

if ApplyAs ==9;
   Difference(src1, xbuffer, dest, kalpha);
   ChannelCopy(src1, dest, kalpha, knone, knone, knone);
   Matte(dest, src1, dest, opacity/100, knone);
end if;

There are two important things to note. First, Matte() works backwards. The order of the input image buffers (xbuffer and src1) is reversed from the other composite controls. This drove me crazy until I noticed how it worked.

Second is the extra code in ApplyAs==9. The built in Difference mode in FXScript does not offer and opacity setting. A little dissecting of Photoshop's Difference mode convinced me that the result is essentially the inverted image applied onto the original at various percentages. 50% resulted in a completely gray image.

My Difference hack implements opacity by first placing the straight 100% difference composite into dest, copying the alpha channel of the original image back into dest, then using Matte() to composite the differenced dest image buffer onto the original contents of src1.

I had to take the additional step of copying the alpha channel back into dest because Difference() has the nasty (and logical) habit of reversing the target alpha channel when it is used. Copying the orignal alpha channel back into the differenced result preserves the original opacity of the image for the Matte() composite which finishes up the mix.

Dealing with YUV

Several of the built in composite modes do not cooperate with YUV information. Sending YUV data into Add, Subtract or Difference results in a disturbing mess of dark greens or bright pinks. These modes seem to operate on all the channels even though the result might push the image data beyond normal YUV amounts.

The solution is simpler than it looks. The images need to be converted to RGB before processing, as discussed on the RGB and YUV page and the FXScript behind Joe's Color Glow, each conversion is wrapped in an if statement, which makes this look a lot worse than it really is.

The first six composite modes are not affected by YUV image data, so only the last three need changing. In the following examples there are two additional image buffers, xbuffer and xbuffer2. The original image, src1, was transferred into xbuffer2 earlier in the script.

The code for the YUVaware Add composite mode works like this:

if ApplyAs ==7;

Add is the seventh item in the ApplyAs menu.

if GetPixelFormat(xbuffer2) != kFormatRGB219;
   dest = xbuffer2;
   ConvertImage(dest, xbuffer2, kFormatRGB219);
end if

The above statement checks the format of the source image buffer. If that format is not RGB219, xbuffer2 is temporarily transferred to dest. Then the temporary contents of dest are converted back into xbuffer2 as RGB219. The following statement performs the same operation on the contents of xbuffer.

if GetPixelFormat(xbuffer) != kFormatRGB219;
   dest = xbuffer;
   ConvertImage(dest, xbuffer, kFormatRGB219);
end if

Now that both source image buffers are set to RGB219 and can be safely composited with Add, Subtract or Difference, the target image buffer needs to be formatted correctly. Because the target has nothing in it, it's format can be declared in one simple statement:

  setpixelformat(dest, kFormatRGB219)

Some of Final Cut Pro's built in FXScripts implied that the above declaration might not be necessary, but many left that sort of statement in place "just in case" (check the source code of Brightness and Contrast in 1.2.5 or 2.0, a comment actually says that. These are not the sorts of things that build confidence...)

Finishing up the conversion requires a bit more buffer juggling. The filter will screw up if it ends with anything other than YUV219 (this is a DV bias). Since the data is now RGB219, the composite can end with a conversion where the result ends up in dest. To prepare for this, the script first transfers xbuffer2 back into dest.

dest = xbuffer2

Next, dest and xbuffer are added and the result is placed into xbuffer2.

add(dest, xbuffer, xbuffer2, HowMuch, kalpha)

Finally, the Add() composite is converted back to YUV219 and placed into the dest image buffer concluding the composite operation. All that's left is to close the original if statement.

ConvertImage(xbuffer2, dest, kFormatYUV219);
end if;

Here is the code for Subtract in one code block. It is exactly the same except for the final compositing function.

if ApplyAs ==8;
   if GetPixelFormat(xbuffer2) != kFormatRGB219;
      dest = xbuffer2;
     ConvertImage(dest, xbuffer2, kFormatRGB219);
   end
   if GetPixelFormat(xbuffer) != kFormatRGB219;
      dest = xbuffer;
      ConvertImage(dest, xbuffer, kFormatRGB219);
   end if
   setpixelformat(dest, kFormatRGB219)
   dest = xbuffer2
   subtract(dest, xbuffer, xbuffer2, HowMuch, kalpha)
   ConvertImage(xbuffer2, dest, kFormatYUV219);
end if;

Difference mode is also the same, except that it incorporates the alpha channel juggling mentioned at the end of the basic set of compositing controls.

if ApplyAs ==9;
   if GetPixelFormat(xbuffer2) != kFormatRGB219;
      dest = xbuffer2;
      ConvertImage(dest, xbuffer2, kFormatRGB219);
   end if
   if GetPixelFormat(xbuffer) != kFormatRGB219;
      dest = xbuffer;
      ConvertImage(dest, xbuffer, kFormatRGB219);
   end if
   setpixelformat(dest, kFormatRGB219)
   dest = xbuffer2
   Difference(dest, xbuffer, xbuffer2, kalpha);
   ConvertImage(xbuffer2, dest, kFormatYUV219);
   ChannelCopy(src1, dest, kalpha, knone, knone, knone);
   Matte(dest, src1, dest, HowMuch, knone);
end if;

Conclusion

These compositing controls appear in the majority of Joe's Filters. It might seem like a lot to think about, but in most instances everything on this page fits in a dozen lines of code. Repetition is not complexity, they only look alike from a distance.

The unlocked set of Joe's Filters include the complete FXScript source code so you can see the composite controls in use.