PI TGV Denoise

[lensman57]

Hi,

Does anyone use this tool or know of any meaningful tutorial as to its correct usage?  Some of the tutorials that I have looked at just do not make sense to me as the function of the parameters are vague to say the least. I have spent hours trying to experiment with this tool with no success.

Regards,

A.G

[IanL]

Here is the original announcement and 'documentation':

http://pixinsight.com/forum/index.php?topic=5603.0

Try this tutorial from Warren Keller:

http://www.ip4ap.com/IP$AP_Flash/Pixinsight_2/PI-13_Nonlinear-7/PI-13_Nonlinear-7_player.html

Here is one official tutorial:

http://pixinsight.com/forum/index.php?topic=5606.0

Here is another official one:

http://pixinsight.com/forum/index.php?topic=5612.msg38878#msg38878

I have to say the tool is easier to use on non-linear images at first (i.e. after you have done a histogram stretch) so if I was you I'd definitely start by testing on a few stretched images.  The default parameters should be about right as a starting place for a typical image.

But the problem is that NR is better done at the linear stage, since you don't end up boosting the appearance of noise with the histogram stretch only to have to work 10x harder to remove it again. The problem is that the TGVDenoise defaults are way off what is required for use on linear images.  Here is my post on the topic and how I got it working (reproduced below):

http://pixinsight.com/forum/index.php?topic=5606.msg38418#msg38418

Once you get it right for linear images, the effects are magical since you can clean up small-scale noise right to the edges of pretty faint structures without having to choose between leaving residual noise in a band around the edges of structures or killing that noise and eating in the edges themselves.

----- Original Post -----

I am starting to have some success with linear images.  The key is to work on the edge protection parameter first, and to use much smaller values than the default.  For example I am using 1e-5 for Chrominance and 9e-6 for Lightness.

I found the right values by creating a local support image first:

- Extract the Lightness and then use the histogram transformation process (with preview) to find the appropriate Highlight, Midtones and Shadow values.

- Cut and paste these in to the Local Support settings (quicker than repeatedly executing and undoing previews in the TGV tool to get the right stretch)

- Preview the noise reduction setting until you find a value that kills off any background noise in the protection image.

Next work on Lightness only (disable preview, disable chrominance):

- Apply an STF to the target image (tricky to find a good stretch for lightness only display when the image is linear but managed it in the end).

- Work on a small preview of the main image first, set iterations to 500 or 1000 and don't use convergence to start with.

- Start with default values and run TGV.

- You will get terrible ringing artefacts on a linear image so reduce the edge protection exponent by -1.

- Undo the TGV on the preview and reapply with the new exponent.

- Repeat the process of reducing the exponent, undoing and re-applying until you find a value for edge protection where you get no smoothing effect at all.

- Now perform a binary search between the lowest value where you got ringing and the value where there was no smoothing.  You should just need to work on the coefficient (or maybe go back up one exponent and then change the coefficient).  

- Start with a coefficient halfway between the lowest value you where you still got ringing and the no ringing values.

- If there is still ringing try the value halfway between the current one and the lowest value, conversely if there is no ringing go up halfway to the higher value.

- Repeat the binary search until you home in on the value where ringing just stops, which should give you maximum smoothing.

The range between ringing and no ringing is tiny in my linear images, the difference between 0.00000900 and 0.00000800 for example.

Then turn off lightness and follow the same process for chrominance.  It will be in a different range but probably very small again.  Once you have nailed the edge protection you can play with strength and smoothness to see what looks best.

Just be aware that once you switch to the full image using convergence you will get ringing again.  You just need to tweak the edge protection for lightness and then chrominance in the same way as before.  It takes longer on a full image but you should be a lot nearer to start with now.

In the sample below all the normal images are linear, starting with the stack which has had Canon Banding script, 2 x DBE iterations and initial ColourCalibration.  Next right is the result of TGV using values as shown in the dialog (plus slightly more agressive values for chrominance).  Next right is the extracted Lightness to the right as support and a clone of it showing the stretch applied by TGV.

Bottom left is an Atrous denoise on layers 3, 4 and 5 with a luminance mask as shown to the right of it.  The final result is a lot smoother than any other method I have tried on my noisy DSLR images and could be improved with a bit more tweaking on the Atrous settings I think.  Blinking between the denoised and original images, the features seem like they have been layered on top perfectly with no damage to the interesting stuff but good smoothing of the background, whereas most of the other techniques you can see a clear erosion of signal as well as noise to the outer regions of galaxies, etc.

Ready for the next steps in processing.

I achieved this in about 2.5 hours from a cold start after from reading the tutorial today, so this is an excellent new tool I'd say!

[lensman57]

strec

Here is the original announcement and 'documentation':

http://pixinsight.com/forum/index.php?topic=5603.0

Try this tutorial from Warren Keller:

http://www.ip4ap.com/IP$AP_Flash/Pixinsight_2/PI-13_Nonlinear-7/PI-13_Nonlinear-7_player.html

Here is one official tutorial:

http://pixinsight.com/forum/index.php?topic=5606.0

Here is another official one:

http://pixinsight.com/forum/index.php?topic=5612.msg38878#msg38878

I have to say the tool is easier to use on non-linear images at first (i.e. after you have done a histogram stretch) so if I was you I'd definitely start by testing on a few stretched images.  The default parameters should be about right as a starting place for a typical image.

But the problem is that NR is better done at the linear stage, since you don't end up boosting the appearance of noise with the histogram stretch only to have to work 10x harder to remove it again. The problem is that the TGVDenoise defaults are way off what is required for use on linear images.  Here is my post on the topic and how I got it working (reproduced below):

http://pixinsight.com/forum/index.php?topic=5606.msg38418#msg38418

Once you get it right for linear images, the effects are magical since you can clean up small-scale noise right to the edges of pretty faint structures without having to choose between leaving residual noise in a band around the edges of structures or killing that noise and eating in the edges themselves.

----- Original Post -----

I am starting to have some success with linear images.  The key is to work on the edge protection parameter first, and to use much smaller values than the default.  For example I am using 1e-5 for Chrominance and 9e-6 for Lightness.

I found the right values by creating a local support image first:

- Extract the Lightness and then use the histogram transformation process (with preview) to find the appropriate Highlight, Midtones and Shadow values.

- Cut and paste these in to the Local Support settings (quicker than repeatedly executing and undoing previews in the TGV tool to get the right stretch)

- Preview the noise reduction setting until you find a value that kills off any background noise in the protection image.

Next work on Lightness only (disable preview, disable chrominance):

- Apply an STF to the target image (tricky to find a good stretch for lightness only display when the image is linear but managed it in the end).

- Work on a small preview of the main image first, set iterations to 500 or 1000 and don't use convergence to start with.

- Start with default values and run TGV.

- You will get terrible ringing artefacts on a linear image so reduce the edge protection exponent by -1.

- Undo the TGV on the preview and reapply with the new exponent.

- Repeat the process of reducing the exponent, undoing and re-applying until you find a value for edge protection where you get no smoothing effect at all.

- Now perform a binary search between the lowest value where you got ringing and the value where there was no smoothing.  You should just need to work on the coefficient (or maybe go back up one exponent and then change the coefficient).  

- Start with a coefficient halfway between the lowest value you where you still got ringing and the no ringing values.

- If there is still ringing try the value halfway between the current one and the lowest value, conversely if there is no ringing go up halfway to the higher value.

- Repeat the binary search until you home in on the value where ringing just stops, which should give you maximum smoothing.

The range between ringing and no ringing is tiny in my linear images, the difference between 0.00000900 and 0.00000800 for example.

Then turn off lightness and follow the same process for chrominance.  It will be in a different range but probably very small again.  Once you have nailed the edge protection you can play with strength and smoothness to see what looks best.

Just be aware that once you switch to the full image using convergence you will get ringing again.  You just need to tweak the edge protection for lightness and then chrominance in the same way as before.  It takes longer on a full image but you should be a lot nearer to start with now.

In the sample below all the normal images are linear, starting with the stack which has had Canon Banding script, 2 x DBE iterations and initial ColourCalibration.  Next right is the result of TGV using values as shown in the dialog (plus slightly more agressive values for chrominance).  Next right is the extracted Lightness to the right as support and a clone of it showing the stretch applied by TGV.

Bottom left is an Atrous denoise on layers 3, 4 and 5 with a luminance mask as shown to the right of it.  The final result is a lot smoother than any other method I have tried on my noisy DSLR images and could be improved with a bit more tweaking on the Atrous settings I think.  Blinking between the denoised and original images, the features seem like they have been layered on top perfectly with no damage to the interesting stuff but good smoothing of the background, whereas most of the other techniques you can see a clear erosion of signal as well as noise to the outer regions of galaxies, etc.

TGV.jpg

Ready for the next steps in processing.

I achieved this in about 2.5 hours from a cold start after from reading the tutorial today, so this is an excellent new tool I'd say!

Hi Ian,

Thank you very much for such detailed reply, I will most certainly follow the instructions. At the moment I am only trying the transfer of the STF values to the local support settings and as you  mentioned I am finding the default values totally unacceptable at the linear stage, I have not as yet tried to use  it on an stretched image.

Many thanks and Regards,

A.G