Inspired by Steve Wrights’ Book “Digital Compositing for Film and Video” I want to show you how to create your custom despill Operation inside Nuke.
This method is actually handy, if it comes to a situation, where you need to blend in your edges even harder than usual, and normal HueCorrect or ColorExpressions are not sufficient. With this method you have the advantage to change the color of your background to any color you want, so you can choose a color that blends into the background which will make your edges smoother.
Now as Steve Wright calls it, I will stick to the term of the Unspill method, and here is how it is set up: First of all we need to create a spillmatte, which is an image that contains all the spill we want to remove. There are a lot of options to create this image. For our setup what we do is first of all separate the channels out into R,G and B using 3 Shuffle Nodes.
In our case we have a Bluescreen (first image) so we will subtract the blue (image2) from the green channel like in the image below (result is seen in image 3)
This new image is now subtracted from the original blue. To control the amount we add a Grade node in between. You can also use a ColorCurve node or any other Color Correction tool. Now usually the spillmatte is subtracted from the channel we want to despill and the other channels remain untouched. To create a custom background color we need to slightly change the other channels. So we will add the spillmap to Red and Green and by inserting a Grade Node again, we have the option to scale the amount the matte is added.
Now finally we just need to recombine our three separated channels to one RGB image. I use a Shuffle Copy for that. Now to get different colors in the background, simply adjust the 3 Grade Nodes to get the desired result. Be aware that this can help you in some situations, but by adding values into the normally untouched channels, you can end up with problematic areas in your image (color shifts in the foreground). Carefully used it can help to make your composite even better.