Blotchy or mottled background nebulosity. Pixinsight.

[Ouroboros]

I have recently had another go at processing 3 hours of OSC data of the Cave Nebula acquired last year with my SW ED80 and ZWO ASI2600MC.

I’m reasonably pleased with the resulting colours and the look of the image overall. But I’m less happy with the blotchy colour in the fainter areas of nebulosity.  Maybe it’s real of course or maybe three hours just isn’t enough.

My workflow has been roughly as follows: 

Dynamic crop.    
SPCC.  
BlurXterminator.  
NoiseXterminator.  
StarXterminator to separate nebula from stars.  
Generalised hyperbolic stretch of starless and star images.  
Several selective applications of TGV Denoise applied using masks (to protect brighter areas).  
Curves applied to the starless image for contrast and emphasis of brighter regions. 
Colour saturation applied to brighter areas and to suppress some unsightly purple areas in the background.  
Some Local Histogram Equalisation to brighter areas and some Dark Structure Enhance.  
Recombination with stars.

Desaturating colour in the background just looked artificial and unrealistic.  So I ditched that approach.

Any suggestions of possible solutions? 

[Fegato]

I would guess 3 hours at F/7.5 (if that's right?) is a little bit on the low side, so more data would probably help

In terms of workflow, there are obviously different options which work for different people, but a couple of comments:

I always apply DBE as a first step, which is what is normally recommended. I've learnt to do this starless now (courtesy of a free Adam Block youtube video), which makes it so much easier on a busy frame like this. So run StarXTerminator with Unscreened option not ticked. Peform DBE on the starless image, making sure to tick the normalize option - much easier to place samples on the bits that have minimal dust or nebulosity. Then add the stars back with PixelMath - a simple $T + stars.

Did you run NoiseXTerminator before stretching? I thought the recommendation was on a non-linear image (but not after too much processing). I tend to do it after an initial couple of stretches, but before any other processing.

[Ouroboros]

~F/6.5 with flattener/reducer. But yes, @Fegato, not oodles of data I agree.

I did do DBE. I just missed it off my list.  Traditionally I’ve always done DBE immediately after dynamic crop. But I read recently that it didn’t matter when you did it during the linear stage. This time I did it on the starless image after doing dynamic crop, SPCC, BlurXterminator and NoiseXTerminator on the original starry image, if you see what I mean.

I’m not sure DBE would make much difference  because the mottled features are on a smaller scale than the DBE background which is usually slowly varying.

While we’re on the subject of applying DBE to the starless image ….. I too have followed Adam Block’s method on this. Have you noticed how applying DBE to a uncolour calibrated starless image does not get rid of the green colour cast you often see at the beginning of processing?  

Yes, I applied NoiseXTerminator on the unstretched linear image. I thought that was recommended. 

 

[Fegato]

OK yes, maybe just more data will have the most impact.

With regard to the green cast, I thought it was BackgroundNeutralization that really got rid of that, rather than DBE? Mind you, I find I still have issues with background colour after further processing, although I'm often stretching hard to pull dust out, which puts everything a bit on the edge.

NoiseX - I was sure it was recommended as best on non-linear, but I can't find any advice on this on the RCAstro site now, so can't really back that up.

an endless road of discovery and re-discovery this hobby...!

 

[ONIKKINEN]

I think you might have pushed the data a little bit too hard in terms of denoising or stretch (or both). Could use more data too of course, 3 hours is really not that much. Since you have noiseXterminator i would rather use that than any other method of denoising like the TGV denoise you applied here. Try using only noiseXT with the same masks you did with TGV denoise? Use a value lower than full power, like 50% or so. Too much will turn the background into a painting which will also look blotchy since some detail that should have stayed as noise will have been evened out by noiseXT and will look unnatural.

But I'd say its not too big of a deal here, the uncropped image is not at all blotchy looking if you dont go looking for it. Image is not too bad to my eyes, would probably not have noticed any blotchiness if you had not mentioned it.

[Ouroboros]

1 hour ago, Fegato said:

With regard to the green cast, I thought it was BackgroundNeutralization that really got rid of that, rather than DBE? Mind you, I find I still have issues with background colour after further processing, although I'm often stretching hard to pull dust out, which puts everything a bit on the edge.

You’re right, but I usually found DBE applied early went a long way to dealing with the green cast too. I normally only do background neutralisation within SPCC. Seems to work anyway. 

[Ouroboros]

1 hour ago, Fegato said:

NoiseX - I was sure it was recommended as best on non-linear, but I can't find any advice on this on the RCAstro site now, so can't really back that up.

You might be right when they first launched it. In fact the process included a box to check for linear. That’s gone now.

I apply it early because not only does it work extremely nicely in linear but also because I thought it was recommended to do some noise reduction before stretching.  

[Ouroboros]

24 minutes ago, ONIKKINEN said:

I think you might have pushed the data a little bit too hard in terms of denoising or stretch (or both). Could use more data too of course, 3 hours is really not that much. Since you have noiseXterminator i would rather use that than any other method of denoising like the TGV denoise you applied here. Try using only noiseXT with the same masks you did with TGV denoise? Use a value lower than full power, like 50% or so. Too much will turn the background into a painting which will also look blotchy since some detail that should have stayed as noise will have been evened out by noiseXT and will look unnatural.

But I'd say its not too big of a deal here, the uncropped image is not at all blotchy looking if you dont go looking for it. Image is not too bad to my eyes, would probably not have noticed any blotchiness if you had not mentioned it.

Thanks for your thoughts on this. I’ll revisit the data and have another go by trying those things.

I feel astro image processing is a constant learning experience. In a sense an image is never ‘done’.  I’ve picked up a lot in the last year or so, and feel it’s good practise to go back and redo old data.  The weather is lousy anyway so few opportunities to get new data.  

 

 

[ollypenrice]

On 29/10/2023 at 10:01, Ouroboros said:

Several selective applications of TGV Denoise applied using masks (to protect brighter areas).  

Personally, I'm suspicious of this stage. The effects you describe look like excessive NR to me.

Regarding NoiseXt, or any other NR, I don't know what the official recommendations are but I cannot imagine running NR before knowing where I wanted it and what its consequences were. Hence I run Noise Xt on the starless once it is sufficiently stretched to reveal noise. I apply it to a top copy layer in Ps so I can blink the NR version on and off to see where it's good, where it's bad and whether or not it's over-done. I can then erase it from where it's harmful, retain it where it's good and adjust its opacity. Using layers I can see what I'm doing as I do it.  I don't think anyone could ever persuade me that making a guess at what to mask and hitting 'Apply' was a better way of doing it.

Olly

Edit: regarding green, try SCNR green. I run it on 9 images out of 10. DBE doesn't deal with it for me.

Edited October 30, 2023 by ollypenrice
SCNR

[Ouroboros]

39 minutes ago, ollypenrice said:

Edit: regarding green, try SCNR green. I run it on 9 images out of 10. DBE doesn't deal with it for me.

Yes. I use SCNR a lot. Usually later and very successfully to remove residual green after background modelisation and colour calibration.  The green I’m referring to above is the bright green colour cast observed in many OSC images immediately after preprocessing and before you do anything else. I have read it arises as a consequence of OSC sensors having twice as many green sensors as red or blue.  

Just out of interest I just applied SCNR to a just processed green image. It goes sepia, and looks yuck! I think it’s better to apply SCNR when you can see what it’s doing in a controlled way, possibly masked too to protect certain areas. 

Edited October 30, 2023 by Ouroboros

[gorann]

SCNR usually works well for getting rid of green, but not always. On some images it creates ugly blue casts on dark patches of sky. In those cases I skip SCNR and try playing with the green curve to suppress it in parts of the histogram. But I do that in PS, where I do all my stretching, but I assume something similar can be done PI.

The mottled background you have could be real but NR may have made it look odd. Try processing it without NR to see what it looks like. Like @ollypenrice I would never even think about applying any NR before I know what the image looks like (so after a fair amount of stretching) and I can se what the NR does to it (and I have the possibility to use it selectively).

[ollypenrice]

1 hour ago, Ouroboros said:

Yes. I use SCNR a lot. Usually later and very successfully to remove residual green after background modelisation and colour calibration.  The green I’m referring to above is the bright green colour cast observed in many OSC images immediately after preprocessing and before you do anything else. I have read it arises as a consequence of OSC sensors having twice as many green sensors as red or blue.  

Just out of interest I just applied SCNR to a just processed green image. It goes sepia, and looks yuck! I think it’s better to apply SCNR when you can see what it’s doing in a controlled way, possibly masked too to protect certain areas. 

I regard SCNR as a linear-stage tool. Sometimes I'll run it and not run it, making two versions which I can then blend easily and selectively in Ps Layers.

Olly

[Ouroboros]

1 hour ago, gorann said:

SCNR usually works well for getting rid of green, but not always. On some images it creates ugly blue casts on dark patches of sky. In those cases I skip SCNR and try playing with the green curve to suppress it in parts of the histogram. But I do that in PS, where I do all my stretching, but I assume something similar can be done PI.

The mottled background you have could be real but NR may have made it look odd. Try processing it without NR to see what it looks like. Like @ollypenrice I would never even think about applying any NR before I know what the image looks like (so after a fair amount of stretching) and I can se what the NR does to it (and I have the possibility to use it selectively).

OK. I can see the amount of noise with STF. 

[Ouroboros]

2 hours ago, gorann said:

The mottled background you have could be real but NR may have made it look odd. Try processing it without NR to see what it looks like.

Yes, thanks for this suggestion. I’m going to revisit the data later when I’ve got some time to experiment with different amounts of noise reduction at different stages. Just got to experiment.  

[windjammer]

I'll go out on a limb and say, yes it is real and you haven't denoised enough.  Looking at my own image of the cave (a smaller FoV) in the area where they overlap, I have the same features as you in the murk.   I posted my pic here if you are interested - it needs to flip vertical to match your orientation.

https://stargazerslounge.com/topic/412672-cave-nebula-update-algorithm-beats-human/#comment-4403775

Simon

 

[windjammer]

.. but a great pic anyway !

[Ouroboros]

1 hour ago, windjammer said:

I'll go out on a limb and say, yes it is real and you haven't denoised enough.  Looking at my own image of the cave (a smaller FoV) in the area where they overlap, I have the same features as you in the murk.   I posted my pic here if you are interested - it needs to flip vertical to match your orientation.

https://stargazerslounge.com/topic/412672-cave-nebula-update-algorithm-beats-human/#comment-4403775

Simon

 

Very nice pic.   It’s interesting comparing the two. It took me a while to get my eye into spotting the similarities and differences. Interesting to see how the different techniques compare and what each bring out. I’m not sure my blotches appear in your image. But I’m going to reprocess mine and see what I get. 

1 hour ago, windjammer said:

.. but a great pic anyway !

Ta!  

[ollypenrice]

1 hour ago, windjammer said:

I'll go out on a limb and say, yes it is real and you haven't denoised enough.  Looking at my own image of the cave (a smaller FoV) in the area where they overlap, I have the same features as you in the murk.   I posted my pic here if you are interested - it needs to flip vertical to match your orientation.

https://stargazerslounge.com/topic/412672-cave-nebula-update-algorithm-beats-human/#comment-4403775

Simon

 

Can't agree on NR. It already looks like 'vaseline on the lens' and that's the root of the problem. But I do agree that it's a good image.

How about resampling it down to a lower resolution , accepting a smaller rendition, and getting a cleaner image?

Olly

 

[Ouroboros]

1 hour ago, ollypenrice said:

Can't agree on NR. It already looks like 'vaseline on the lens' and that's the root of the problem. But I do agree that it's a good image.

How about resampling it down to a lower resolution , accepting a smaller rendition, and getting a cleaner image?

Olly

 

well, just tried that by halving the resolution (ie reducing the image height and width by a factor of two, if that's what you meant) and it made no discernible difference. Thinking about it anyway, what you're looking at is reduced resolution because I reduced image size by a factor of about 5 before saving to jpg to post. 

.... and I don't think it is a particularly good image in that the framing could have been better ie the nebula could have been more central, but that's in the data acquisition stage.  

[windjammer]

mmmm, well, you could try posting your master lights here (just calibrated, coscor and aligned) and see what the ravening, opinionated, barbarian hordes here make of them.  Seeing as how its raining and we don't have anything else to do : )

[Ouroboros]

7 hours ago, windjammer said:

mmmm, well, you could try posting your master lights here (just calibrated, coscor and aligned) and see what the ravening, opinionated, barbarian hordes here make of them.  Seeing as how its raining and we don't have anything else to do : )

OK Let's see if this works ...

This is 90 x 120s pre-processed data.  All I've done is a slight dynamic crop of edge effects. Enjoy!  

NGC281_dc.xisf

[Fegato]

Ha ha - yes, the weather has been so bad - nice to have a go at processing something! My effort below...

Observations:

1. The integrated image was noisier than I'm used to dealing with, so I think as has already been said, the best improvement can be made with more data. I purely applied NoiseX with this, after initial stretches, and quite strongly. I think blotchiness is not really there, but I guess it's a bit "plastic" from the noise reduction

2. I got tiling artifacts with StarX, so had to apply the "Large Overlap" option to prevent this

3. I've probably not stretched as much as yours. This felt like a bit of a limit to me. I had to deal with some nice purple and green hues in the background dust, and used a range mask to push the saturation a little in the nebula and dampen it elsewhere

Having said all that, processing this reminded me that I did find my own effort on the Cave a tricky one to process - I think maybe it's an area with quite a lot of obscuring dust that makes it difficult to pull out detail?

Edit: looking at that, I've perhaps not got enough contrast - a bit too dusty?

Edited October 31, 2023 by Fegato
added comment on picture

[Ouroboros]

Thanks, @Fegato.  Your background looks much better than in my effort, which encourages me to have another go.I think we all agree there is insufficient data. But then I’m thinking of this as a learning exercise.

I am amazed how quickly you produced a result. I work very slowly, not least because I don’t quite know where I’m going so I spend ages on each process trying something, going back, trying something else.  But how else can you come to understand what is after all a very complicated processing tool which can be applied any number of possible ways to achieve different results?Watching videos only takes one so far. 

[Fegato]

7 minutes ago, Ouroboros said:

Thanks, @Fegato.  Your background looks much better than in my effort, which encourages me to have another go.I think we all agree there is insufficient data. But then I’m thinking of this as a learning exercise.

I am amazed how quickly you produced a result. I work very slowly, not least because I don’t quite know where I’m going so I spend ages on each process trying something, going back, trying something else.  But how else can you come to understand what is after all a very complicated processing tool which can be applied any number of possible ways to achieve different results?Watching videos only takes one so far. 

I did obviously spend many many hours learning PI... the Covid lockdown really helped me with this!  But tbh I'm sure I'm guilty now of having found a way that works for me and not exploring other options enough! The RCAstro tools obviously contribute to that, and as I've discovered how to GPU accelerate them, it can make the process very quick indeed!

[gorann]

Here is my version. All done in PS except BlurXT (default setting except I used 0.15 "Sharpen stars") and a dose of SCNRgreen at the end, both done in PI. In PS I stretched the image a bit, then run StarXT, and continued to stretch the starless image. I got rid of most of the green with Gradient XTerminator. Tweaked the colour curves a bit to suppress some blue casts. Gave it a dose of Noise XT near the end and increased contrast with the HiPass filter.

Yes, more integration time would help but I did not see much mottle.

Cheers, Göran