Image Integration results very noisy

[souls33k3r]

Hi All,

I've just finished completing 20hrs of data run on a new target.

The usual processes were followed up to image integration but the end final stacked sub was not smooth as i had hoped after 20hrs of data.

I took Darks, Bias (took them at the same temperature as lights), Flats and calibrated them using BPP but the noise is still there in the end in all channels.

I have handled the noise using noise reduction techniques but wondered what else can i do?

The single sub is of course noisy, i'm using Unity Gain and each sub is 5 min long, in total i've got around 6.5hrs of data on each channel.

I do dither but if memory serves me correct it's (within SGP) small dither, every 4 frames.

Maybe i'm using the correct setting, maybe it's something else. I don't know

Just to add, i'm using ASI1600MM-C with Astronomik 6nm filters

Any suggestions would be greatly appreciated.

Thanks in advance

 

[tooth_dr]

Can you post the image? Is it noisier than usual or your expectations? What were the capture conditions like?

[RolandKol]

Do you mean even Ha is very noisy? or just S/O?

 

P.S.

if it is in relation to the most recent nights... it was always hazy in my part of London

[souls33k3r]

9 minutes ago, tooth_dr said:

Can you post the image? Is it noisier than usual or your expectations? What were the capture conditions like?

Please see the image below. This is right after the stack.

It's just that i've never been able to get smoother (not necessarily completely noise free) image after the stacked ever. 

My camera is always cooled down to -20 degree. I see people with CCDs as an example take images and even their single sub is smooth as a silk (compared to mine they are). I wonder if i'm not approaching the complete process in the right now.

Capture conditions were good, not great but as good as it could get.

3 minutes ago, RolandKol said:

Do you mean even Ha is very noisy? or just S/O?

All three unfortunately. I do end up doing some sort of noise reduction to control it but i then have to subsequently do other bits to bring the sharpness in the image back.

[tooth_dr]

Just see you are using the 1600.  I cant help I'm afraid, except to say that my CCD sub frames are FAR from silky smooth, but do stack up nicely.

[souls33k3r]

3 minutes ago, tooth_dr said:

Just see you are using the 1600.  I cant help I'm afraid, except to say that my CCD sub frames are FAR from silky smooth, but do stack up nicely.

Well stack up better than what i get i'm sure. That's exactly what i mean. After 20hrs of data, i expect smoother image or am i just wishing and hoping too much?

[RolandKol]

Just to be clear, - the example is not 20h data, but only 6.5h (per channel).

And it looks just just a bit better than my usual 4h HA data  
London.... Nothing to add... 

We all need 3nm here!!!

However, 
I would expect quite a nice Final image from the example you have

 

P.S.

Taking into account, that "bloody" SNR formula, if you want to double SNR value, from 4h, you need at least 16h.

One more thing, the worst SNR Channel will be the one which will make the largest impact to the final image quality.....

I am not telling I do it correctly,
but I try to decrease HA total time, and increase S/O accordingly and stack only the best/close to the best from the session...

 

 

[souls33k3r]

4 minutes ago, RolandKol said:

Just to be clear, - the example is not 20h data, but only 6.5h (per channel).

And it looks just just a bit better than my usual 4h HA data  
London.... Nothing to add... 

We all need 3nm here!!!

However, 
I would expect quite a nice Final image from the example you have

 

P.S.

Taking into account, that "bloody" SNR formula, if you want to double SNR value, from 4h, you need at least 16h.

One more thing, the worst SNR Channel will be the one which will make the largest impact to the final image.....

I am not telling I do it correctly,
but I try to decrease HA total time, and increase S/O accordingly and stack only the best/close to the best from the session...

 

 

Correct, this is just Ha channel data (6.5hrs).

I've already corrected the noise but i was pulling whatever hair was left on my head as to why my final stacked image isn't as smooth for this amount of data.

I'm sure we all have found ways to better ourselves and i'm trying to get better at this as well. Joys of imaging from Bortle 8 skies eh?

[RolandKol]

2 minutes ago, souls33k3r said:

Correct, this is just Ha channel data (6.5hrs).

I've already corrected the noise but i was pulling whatever hair was left on my head as to why my final stacked image isn't as smooth for this amount of data.

I'm sure we all have found ways to better ourselves and i'm trying to get better at this as well. Joys of imaging from Bortle 8 skies eh?

You should join us in astrocamp next autumn  just 1.5h from Lnd, and the milkyway is clearly visible with a naked eye  
Do not remember, The Spot has  B3 or B4 skies  probably 4, 3 would be a bit too good to be true
 

[vlaiv]

1. Don't use bias.

2. What sort of offset are you using? If camera is not pro model and you can set offset - set it to somewhere about 60 or so. I use value of 64. (this will not help now)

3. How many calibration frames are you using? Take as many as you can (maybe you can add darks or even flats if you have permanent setup, but darks will be more important).

4. Dithering every frame is very good practice.

5. I've found that that particular region of the sky is faint. Any sort of LP, regardless of NB filters used will have an impact with faint signal. Was there moon present? Did you do multiple sessions and conditions were different between them (transparency, LP, Moon). If so, you need weighted average rather than regular average.

[souls33k3r]

2 minutes ago, RolandKol said:

You should join us in astrocamp next autumn  just 1.5h from Lnd, and the milkyway is clearly visible with a naked eye  
Do not remember, The Spot has  B3 or B4 skies  probably 4, 3 would be a bit too good to be true
 

I sure will mate. Hopefully by that time i'll have my own car as well (wife has been pushing me to get one before June this year). Can't wait. 

2 minutes ago, vlaiv said:

1. Don't use bias.

2. What sort of offset are you using? If camera is not pro model and you can set offset - set it to somewhere about 60 or so. I use value of 64. (this will not help now)

3. How many calibration frames are you using? Take as many as you can (maybe you can add darks or even flats if you have permanent setup, but darks will be more important).

4. Dithering every frame is very good practice.

5. I've found that that particular region of the sky is faint. Any sort of LP, regardless of NB filters used will have an impact with faint signal. Was there moon present? Did you do multiple sessions and conditions were different between them (transparency, LP, Moon). If so, you need weighted average rather than regular average.

Thank you for your input vlaiv.

1) I wasn't using Bias before but for this image i did try and it did simmer the noise down quite a bit. Before that it was just weird. It looked more like suns chormosphere type noise 

2) I'm using Unity Gain. It's the version 3 of the camera, one before the pro model.

3) I'm using 50 flats, 100 darks and 100 bias

4) I could indeed try that but that does add up to the image acquisition time. Is there no other way i can dither better rather than every frame?

5) Yes this area is faint but the moon was not present. In fact tell a lie it was not present on SII & OIII but it was on Ha and Ha was the better of the results. I did use weighted average.

[vlaiv]

100 darks should be enough.

What are you using bias for?

For dark subtraction - regular darks are enough (just make a master dark by average with sigma clip stacking). For flats - use flat darks rather than bias. These are like regular darks for lights, except done for flats - meaning same settings and exposure length.

Dithering every frame is the least costly when doing NB, as exposures take longer than LRGB - 5 minutes is good value for NB. This means that you loose about 5-8% of imaging time - that is not much (every dither will take max 15-20s, on 300s exposure). You can dither every N frames, but best results are when doing it each frame - this spreads residual calibration noise around and also helps with FPN.

On further inspection, I don't think image is that poor - careful stretching will hide much of background noise, and slight noise reduction can go a long way on such image.

[RolandKol]

 Our Results are similar as everything is very similar...

My usual cal.subs: 50flats, 100darks, 200 bias or 100 FlatDarks (FD - while been testing options, not using them regularly).

However, I have never managed to find Weighted Average on PI, - only the Average, I also target to stay on Linear Fit Clipping.
 

[tooth_dr]

1 minute ago, souls33k3r said:

Well stack up better than what i get i'm sure. That's exactly what i mean. After 20hrs of data, i expect smoother image or am i just wishing and hoping too much?

I think for 6.5 hours that is ok, certainly on a par with my level of processing and equipment.

[souls33k3r]

6 minutes ago, vlaiv said:

100 darks should be enough.

What are you using bias for?

For dark subtraction - regular darks are enough (just make a master dark by average with sigma clip stacking). For flats - use flat darks rather than bias. These are like regular darks for lights, except done for flats - meaning same settings and exposure length.

Dithering every frame is the least costly when doing NB, as exposures take longer than LRGB - 5 minutes is good value for NB. This means that you loose about 5-8% of imaging time - that is not much (every dither will take max 15-20s, on 300s exposure). You can dither every N frames, but best results are when doing it each frame - this spreads residual calibration noise around and also helps with FPN.

On further inspection, I don't think image is that poor - careful stretching will hide much of background noise, and slight noise reduction can go a long way on such image.

Like i said, i've never used Bias before this and that was our of sheer frustration to try and minimise the inherent noise. I know Bias are not required for this type of camera but i've seen some people's work and their image quality and results are superb so i thought it's not going to cost me any time in taking them so let's see. I'm not sure what i was expecting when calibrating them but it did help reduce the noise slightly and enough for me to progress. I now do have a master flat so will use that. Never looked in to flat darks so will need to read up on that.

Dithering will take 15 sec per sub? 

I agree the image which came out isn't that poor and i have taken care of the noise residue in post processing but those are what i think are a bit unnecessary steps that i've taken or even if i had to do that, i could've been subtle with it but i couldn't. 

That is the reason for my question as to what i could be doing or do to get a much smoother stacked sub? Maybe it's the initial process that i'm not following or doing correctly?

4 minutes ago, tooth_dr said:

I think for 6.5 hours that is ok, certainly on a par with my level of processing and equipment.

This is my second image with this much integration. I find that processing becomes easier and i don't have to massively stretch things to the extreme but on the flip side it takes some time weeks to capture that amount of data. Nothing is easy in this stupid hobby that we love

[souls33k3r]

13 minutes ago, RolandKol said:

 Our Results are similar as everything is very similar...

My usual cal.subs: 50flats, 100darks, 200 bias or 100 FlatDarks (FD - while been testing options, not using them regularly).

However, I have never managed to find Weighted Average on PI, - only the Average, I also target to stay on Linear Fit Clipping.
 

Thinking about this, i think what @vlaiv means by weighted average is by using SFS to get the weights and then using the keyword to average it.

If we're both using similar, then i guess i don't need to bring my kit to the dark site then, i can just happily use yours and then process the data

[RolandKol]

9 minutes ago, souls33k3r said:

Thinking about this, i think what @vlaiv means by weighted average is by using SFS to get the weights and then using the keyword to average it.

If we're both using similar, then i guess i don't need to bring my kit to the dark site then, i can just happily use yours and then process the data

hahahaha

In fact  I even started the topic here on the forum asking to group up on the same target to increase SNR... No replies during 2 months or something  no joy

In your case, -  I am on 130PDS with star spikes and etc, so unfortunately no match
Maybe, someday, I will go frac way... As PDS demands quite a lot of attention... but it looks like I have managed to calm it's Ego a bit down, so not sure yet.

36 minutes ago, souls33k3r said:

I sure will mate. Hopefully by that time i'll have my own car as well (wife has been pushing me to get one before June this year). Can't wait. 

So no more New Astro-toys for awhile!????  Heh

P.S.

I am not sure how you combine all 3 channels in PI, - I noticed a better results while combining stretched (usually it is a bit boring way, but the best one....) 

Later, after all Color calibrations and etc.

Extracting luminance>reduce noise a bit again > sharpen it

The main image: Reduce noise quite heavily (with or without mask, - depends on target).

Apply some curves on Luminance and  blend it back using LRGB tool with Chrominance Noise reduction option On.
I have never succeed it from the first time, as Curves have a lot of impact, so I test different amount of S curves on Luminance for LRGB combination.
 

And sometimes something quite nice comes out... but only sometimes unfortunately

[souls33k3r]

2 minutes ago, RolandKol said:

hahahaha

In fact  I even started the topic here on the forum asking to group up on the same target to increase SNR... No replies during 2 months or something  no joy

In your case, -  I am on 130PDS with star spikes and etc, so unfortunately no match
Maybe, someday, I will go frac way... As PDS demands quite a lot of attention... but it looks like I have managed to calm it's Ego a bit down, so not sure yet.

So no more Astro-toys for awhile!????  Heh

P.S.

I am not sure how you combine all 3 channels in PI, - I noticed a better results while combining stretched (usually it is a bit boring way, but the best one....) 

Later, after all Color calibrations and etc.

Extracting luminance>reduce noise a bit again > sharpen it

The main image: Reduce noise quite heavily (with or without mask, - depends on target).

Apply some curves on Luminance and  blend it back using LRGB tool with Chrominance Noise reduction option On.
I have never succeed it from the first time, as Curves have a lot of impact, so I test different amount of S curves on Luminance for LRGB combination.
 

And sometimes something quite nice comes out... but only sometimes unfortunately

Funnily enough, i was talking to a very good mate about how much i hate Newtonians but i'd still love to have one. But given the skies we have, i don't want to be spending time on adjusting and collimating a newtonian. Frac was a hard pill to swallow but i did in the end. 

No more astro toys? Never say never my friend, there's always room for just one more toy  new toys means new excuses to get permission from the Boss lady. 

I don't stretch the image before combination, i always stretch it after i've combined the channels. I'm not sure why but i've always done it this way. Again you've made me think why i do that  and then the post processing follows.

 

[david_taurus83]

Looks similar to what I get? Do you use cosmetic correction? I find the auto detect function with a setting around 2.2 on both hot and cold pixels works well. Subtle, but any lower and you start losing sharpness.

[souls33k3r]

25 minutes ago, david_taurus83 said:

Looks similar to what I get? Do you use cosmetic correction? I find the auto detect function with a setting around 2.2 on both hot and cold pixels works well. Subtle, but any lower and you start losing sharpness.

Cosmetic correction as in the process or is it part of another process?

[david_taurus83]

The actual process. I do it after calibration and before sub frame selector/star alignment.

[souls33k3r]

Ah i see. I must admit, i've always skipped that then.

[david_taurus83]

I find I have more cold black pixels than hot white ones. Cosmetic Correction removes some of them. It's like pre processing noise reduction. Like I've said, a subtle iteration is key. Also, I use flat darks to calibrate flats and don't use bias at all.

[souls33k3r]

3 minutes ago, david_taurus83 said:

I find I have more cold black pixels than hot white ones. Cosmetic Correction removes some of them. It's like pre processing noise reduction. Like I've said, a subtle iteration is key. Also, I use flat darks to calibrate flats and don't use bias at all.

Funnily enough, i have noticed these "cold black pixels" in the last few images but i've always taken care of them using noise reduction. I'd be sure to look in to CC from now on. I've only used Bias because that was the only normal calibration frame that was missing and seeing someone on Youtube take them and producing amazing results made me think "Well if he's doing it, he must be finding them useful". Hence the reason why i took them for this image.

My main concern is still there. Why the 6.5Hrs of image is still showing so much noise? Is there something that i'm missing while going BatchPreProcessing, Image Integration or Star Alignment routine?

[david_taurus83]

I've attached the 3 stacks ive done in Ha. These are are just the stacks, fresh out of preprocessing and just an STF stretch applied, no noise reduction. The Flaming Star one is a stack of around 4 hours of 5 minute subs. I feel this is also too noisy. It was because of this i decided to try longer Ha subs. So i stretched out to 900s! The Rosette is only 7 x 900s, just under 2 hours but he Horsehead is 16 x 900s, 4 hours. My favourite one as it needed nothing apart from a stretch! All used the same pre processing settings.