Processing NB images - when to denoise?

[Adreneline]

So far when processing Nb images prior to combining as Bi-Colour or HST etc. I have followed the advice given in the LightVortex tutorials, namely "They have been fully pre-processed, have been registered with each other, have had background gradients removed with DBE and have been noise reduced with MLT...... in order to get the most out of colour combining these images, it is best they are stretched to non-linear".

I was nosing around on the PixInsight forum and came across a MureDenoise script for denoising linear monochrome images, where the advice given was  "Do not combine denoised images. Signal-to-noise ratio (SNR) will be enhanced by combining noisy images and denoising the result".

So now I'm confused - I think! What's the considered view on SGL? (a) pre-process, denoise, stretch and combine, or (b) pre-process, stretch, combine and denoise.

Adrian

[vlaiv]

I would say that it depends on denoising algorithm used.

Some denoising algorithms are designed to work best with linear data of known noise statistics (by known here it means estimated from data, but of known distribution type). Others are designed based on perception uniformity of the image, or general smoothness (like TV family of algorithms that assume smooth underlying data).

1 hour ago, Adreneline said:

 "Do not combine denoised images. Signal-to-noise ratio (SNR) will be enhanced by combining noisy images and denoising the result".

I don't think this refers to channel combination. It looks like it was written in response to question like: "Should I denoise subs prior to stacking or denoise resulting stack". Simple SHO palette combination of data does not influence particular noise distribution per channel. Some other methods of combining data will influence noise statistics (like linear combination per channel), but again - denoising algorithms work per channel, so it will depend on type of algorithm used (some even transform RGB image into Lab and employ different levels of denoising based on working with luminance or chrominance data).

[Adreneline]

Hi vlaiv

Many thanks for your reply.

27 minutes ago, vlaiv said:

I don't think this refers to channel combination.

The PI forum article was specifically about combining Nb images, e.g. Ha, OIII and SII, and using MureDenoise, making the point that individual masters should not be denoised prior to combining. The LV article mentions MLT but previous tutorials make it clear MLT is only one option, others include MureDenoise. Hence my confusion. The only thing in common is that images should be stretched to non-linear before combining.

Adrian

 

P.S. This is the link to the PI forum article: https://pixinsight.com/forum/index.php?topic=9206.0

Edited February 9, 2019 by Adreneline
Link included

[vlaiv]

From what I can see in first post of that topic you linked to, MURE denoise works as first type of algorithm that I described. It assumes image is combination of Poisson and Gaussian noise distributions and does some clever math to remove associated noise. In that context, sentence:

"Do not combine denoised images. Signal-to-noise ratio (SNR) will be enhanced by combining noisy images and denoising the result. Combined images must be equally exposed, have the same pixel resolution, and be registered by projective transformation with no distortion correction."

refers to exactly what I mentioned - stacking of denoised images vs denoising stack and not channel combine.

It also says that during registering and combining you need to be careful not to do any sort of projective corrections (like fixing lens distortion in wide field images and such).

Because this algorithm utilizes noise distribution statistics - it needs linear data, so you should apply it after stacking and before any sort of non linear transform (histogram stretch). Non linear transform skews relationship between signal strength and associated noise in Poisson distribution.

It is explained in detail in this post:

https://pixinsight.com/forum/index.php?topic=9206.msg59115#msg59115

 

[wimvb]

My more or less standard workflow consists of:

- Crop

- Dbe

(- Colour calibration of rgb data)

- Deconvolution

- TgvDenoise with a 50% mask or mlt with a linear mask

- Stretching

- Contrast enhancement and sharpening

- Noise reduction, either tgv or mlt.

But I only use noise reduction when it's absolutely necessary. Usually if the data is good, it doesn't need noise reduction.

The best noise reduction is to collect more data.

 

Edited February 9, 2019 by wimvb

[Adreneline]

4 minutes ago, wimvb said:

Usually if the data is good, it doesn't need noise reduction.

Hi Wim,

I agree entirely, but these two articles seem to be at odds - one says denoise before combining Ha, OIII, SII masters using PixelMath (or whatever) and the other says denoise after combining the individual masters.

I have a feeling it's all pretty marginal if the original data is good and you would be hard pressed to tell the difference as to when the noise reduction was performed. As you say, best to have good data and not to have to denoise and not to have to use deconvolution either!

Thanks for your response.

Adrian

[vlaiv]

11 minutes ago, Adreneline said:

Hi Wim,

I agree entirely, but these two articles seem to be at odds - one says denoise before combining Ha, OIII, SII masters using PixelMath (or whatever) and the other says denoise after combining the individual masters.

I have a feeling it's all pretty marginal if the original data is good and you would be hard pressed to tell the difference as to when the noise reduction was performed. As you say, best to have good data and not to have to denoise and not to have to use deconvolution either!

Thanks for your response.

Adrian

Can you give a link to second one - recommending denoising after channel combine?

[Adreneline]

14 minutes ago, vlaiv said:

Can you give a link to second one - recommending denoising after channel combine?

This is the link vlaiv:

https://pixinsight.com/forum/index.php?topic=9206.0

Thanks for looking.

Adrian

[vlaiv]

1 minute ago, Adreneline said:

This is the link vlaiv:

https://pixinsight.com/forum/index.php?topic=9206.0

Thanks for looking.

Adrian

I'm not following. This thread / first post explicitly says that denoising should be done at linear phase after stacking. MURE denoising does not work on multichannel data, like RGB / false color images - it even states that you can't do it on OSC data (you can, but you need to split your OSC subs into separate channels without debayering and treat them as individual mono/filter subs that you stack and then apply denoise as you would with regular mono subs).

30 minutes ago, Adreneline said:

I agree entirely, but these two articles seem to be at odds - one says denoise before combining Ha, OIII, SII masters using PixelMath (or whatever) and the other says denoise after combining the individual masters.

You say here that there is article that says you should apply some type of denoise algorithm after combining of individual masters into multichannel image? Right? Which article would that be?

[Adreneline]

2 minutes ago, vlaiv said:

apply some type of denoise algorithm after combining of individual masters

Hi vlaiv,

The PI forum article (the link I've sent) says in Warning (Note) 2 : "Do not combine denoised images. Signal-to-noise ratio (SNR) will be enhanced by combining noisy images and denoising the result. Combined images must be equally exposed, have the same pixel resolution, and be registered by projective transformation with no distortion correction."

The LV tutorial (  https://www.lightvortexastronomy.com/tutorial-narrowband-bicolour-palette-combinations.html ) advises the opposite - perform denoising before combining. That's why I'm confused.

Sorry if I am confusing you as well

Adrian

[vlaiv]

3 minutes ago, Adreneline said:

Hi vlaiv,

The PI forum article (the link I've sent) says in Warning (Note) 2 : "Do not combine denoised images. Signal-to-noise ratio (SNR) will be enhanced by combining noisy images and denoising the result. Combined images must be equally exposed, have the same pixel resolution, and be registered by projective transformation with no distortion correction."

The LV tutorial (  https://www.lightvortexastronomy.com/tutorial-narrowband-bicolour-palette-combinations.html ) advises the opposite - perform denoising before combining. That's why I'm confused.

Sorry if I am confusing you as well

Adrian

I think you are confused with ambiguous usage of term "combine".

Here is quote from LV article that you just linked (I just did quick search on term noise and this is pretty much where it appears in text):

"Though your images may differ, it is common to apply some noise reduction to images in their linear state. The choice of whether or not to do this, or how aggressively to do it, depends on the level of noise in your images. Please note that this is beyond the scope of this tutorial and is covered amply by another tutorial specially written on the subject of noise reduction. My personal noise reduction routine of choice for the images we are about to post-process was using MultiscaleLinearTransform (as described in the tutorial on noise reduction) with stretched clone copies of the images themselves acting as masks. "

Here it advises you to do denoise at linear stage.

Let's quickly "run thru" stages of data reduction and processing:

1. calibration

2. stacking (combining)

3. non linear transform

4. multichannel combining

In some workflows 3 and 4 can swap places. What is important is that both articles place denoise phase at the same place in the workflow. First article about MURE denoise from PI forums says that you should do it after phase 2 but prior to phase 3. What has confused you is that it uses term combine instead of stacking and that warning about doing it after "combine" (stacking) is referring to the fact that you should do it after phase 2 and not after phase 1 - after calibration. If you do it  "prior to combining" (meaning prior to stacking) - you will end up skewing statistics of signal and noise and stacking (or here called "combining") will not produce wanted results - down the thread on PI forum OP says that he tried that with result of loss of faint detail - this is why you should not denoise individual subs prior to stacking (or "combining"), but rather after stacking / "combining", but prior phase 3 while data is still linear.

To reiterate - both articles say the same thing - denoise while data is still linear - this has nothing to do with multichannel combining, or making color (false or otherwise) image out of individual mono masters. Denoising should be done at linear stage prior to this.

[Adreneline]

Hi vlaiv,

Thanks for clearing that up.

Adrian

[Starflyer]

MURE denoise only works on a linear image that is an integration of your subs, a mono image from one channel. 

From how I understand it, If you do anything to that image, crop it, DBE it etc, then MURE denoise won't work correctly. It needs to be the first process you perform after image integration. 

[Barry-Wilson]

On 10/02/2019 at 21:17, Starflyer said:

MURE denoise only works on a linear image that is an integration of your subs, a mono image from one channel. 

From how I understand it, If you do anything to that image, crop it, DBE it etc, then MURE denoise won't work correctly. It needs to be the first process you perform after image integration. 

? - that's correct.

[ollypenrice]

On 14/02/2019 at 16:27, Barry-Wilson said:

? - that's correct.

Cropping included? This seems odd to me. Mind you I don't know the routine at all but I'd have thought that stacked image borders had unique artefacts of their own.

Olly

[Rodd]

On ‎14‎/‎02‎/‎2019 at 10:27, Barry-Wilson said:

? - that's correct.

Actually, I read somewhere in the literature for MD, or a tutorial, can't remember where that cropping is fine, but nothing else.  The idea is you can't change the pixels, can't modify them from the raw state.  cropping just removes pixels--it does not alter the pixels that remain.  In fact, Mure Denoise has no way of knowing if you cropped or not.

Rodd

[jiberjaber]

On 15/02/2019 at 20:27, ollypenrice said:

Cropping included? This seems odd to me. Mind you I don't know the routine at all but I'd have thought that stacked image borders had unique artefacts of their own.

Olly

Yes, the process uses the flat data to form the model to detect noise from, so they all need to be the same, if you crop one, you would need to crop all.  

There is also a helper script to calculate the model data, this needs 2 of each dark, flat and light sub-frames to work so they all need to stay the same size (though not sure on alignment of the two sub-lights but I would expect that might have a slight impact if they are not)