This is one of the tricks that comes in handy, whenever you have to deal with an unevenly lit green- or bluescreens, but it has to be said, that this is not a one click solution ...it will just bring you closer to the desired result.
First of all check out your material for the uneven area and make sure to not lose any detail around the object you want to keep. In our case the uneven region is in the lower right corner, which can easily matted, but imagine the character will move in front of it, then there would be no other way than doing some nice roto or use this trick.
So now let's check out how it's done
First of all we need to create a clean plate, which means, that we want to get rid of the actor in our case. How do we do that ?? In Nuke, you will have several options, so I describe one, use whatever you want- really.
I usually start by pulling a hard key and for that I most of the time use Keylight.
It does not need to be perfect, you can refine it later on if this makes any troubles. Next we INVERT this mask and add an ERODE filter to the flow. Set the ERODE to a high value, like around 50 and then add a BLUR node to soften the result.
This is how everything should look so far....so now move on.Copy this alpha into your Plate and add a PREMULT operation. After that add another BLUR node, set to a high value like around 300 and add an UNPREMULT right after that. What you will end up with is a CLEAN greenscreen Plate.
We now keymix the clean plate over the original using the soft mask we genereated before...which will end up looking like the image below. Why are we doing this? We want to prevent our cleaning task to change anything on the foreground region , so we need to make sure that this is a solid color and will key out nicely.
Well and where is the trick now...hang on, compositing takes time, so patience is what you need to have: We continue by pulling a new Key of the plate and apply the exact same node to what we created right now, invert both Results and Divide them. Check the Alphas to understand what's going on. The Plate Alpha has the uneven patch in there and of course the character, the CleanPlate Alpha only has the patch, so by dividing them, we end up with an Alpha that holds out character from the patch...nice ha.
So now what we finally need to do is generate a Constant and sample a green color from the plate, and subtract the Keymix node. This will look like below.
Now copy our recently generated Alpha into that and add a PREMULT.
After the PREMULT add a Merge node and PLUS this on top of your plate ! You will have a nice evenly light greenscreen, that is now ready to be comped ! You can play around with the amount of blur and the 3 Keyers we used to get slightly better results in the edges.
It looks like a lot of nodes but this can safe you from painting out folds or shadows on a bad baking. Remember that this is far from the perfect solution, but it will help you here and there, I am sure.
As this seems to be harder to explain than i thought, I added a sample script here (around 100MB) for you to play around with...have fun !