How many Dark frames for 30mins exposures?

[Droogie 2001]

Having recently increased my exposure times up to 30 minute subs I am now in need of taking the same length exposures for my Dark frames.

I understand the need to take multiple dark frames as part of the calibration process, usually more is better (up to a point). In fact I use a Master Dark when calibrating in PixInsight.  That is once I have taken my darks frames I go through the Image Integration steps in creating the master dark file so that I only have to reference this one file from then on.

Obviously longer exposures result in having less files when say taking the equivalent total exposure time, for example:

24 x 10min darks = 4 hours or
8 x 30min darks = 4 hours.

So I am wondering whether the fact I have less darks frames when taking 30min exposures, do I need to increase this so that the calibration process can benefit from having more frames?

Do the algorithms which help average out the dark frame calibration data benefit from more frames, I guess yes is the answer but to what extent?
Should I look at taking 8 hours of darks at 30mins so that I can increase the number of frames for calibration to 16 or perhaps even more?

Hopefully this makes sense?!

Thanks a lot.

[swag72]

I can't offer any scientific basis..... but I always use 25 darks.

[Horwig]

Same as Sara I'm afraid, I can't offer scientific reasoning, I just take the same number of darks as lights

Huw

[Droogie 2001]

Thanks for the speedy response Sara.

Phew that works out at 12.5 hours of Dark exposures. I guess I cant blame the weather when creating these but I am going to need to up my previous estimate.
Certainly 25 frames roughly matches up with my current 24 frames for 10mins darks.

 

 

[Droogie 2001]

5 minutes ago, Horwig said:

Same as Sara I'm afraid, I can't offer scientific reasoning, I just take the same number of darks as lights

Huw

Certainly makes sense as the greater number of files will help average out the dark noise I guess?

 

[ollypenrice]

A couple of thoughts.

- Have you considered not using darks at all? On our Kodak cameras we use a master bias as a dark along with a defect map and a hot pixel filter in the stacking. I find this generally works better than using darks, sometimes much better.

- I'd expect that a small number of darks, of whatever length, would risk doing more harm than good.

- Do you need to do 30 minute darks if stacking in PI? I know little about this but won't it optimize a different dark based on the master bias?

As ever, I think that experimentation triumphs over theory.

Olly

[Droogie 2001]

4 minutes ago, ollypenrice said:

A couple of thoughts.

- Have you considered not using darks at all? On our Kodak cameras we use a master bias as a dark along with a defect map and a hot pixel filter in the stacking. I find this generally works better than using darks, sometimes much better.

- I'd expect that a small number of darks, of whatever length, would risk doing more harm than good.

- Do you need to do 30 minute darks if stacking in PI? I know little about this but won't it optimize a different dark based on the master bias?

As ever, I think that experimentation triumphs over theory.

Olly

Thanks for the thoughts Olly.

- Have you considered not using darks at all? On our Kodak cameras we use a master bias as a dark along with a defect map and a hot pixel filter in the stacking. I find this generally works better than using darks, sometimes much better.

I have not really entered the realm of defect maps and hot pixel filtering so I have not looked at this avenue. Its a case of I have heard of it but shied away from it... I will have to carry out some reading and research on the topic.

- I'd expect that a small number of darks, of whatever length, would risk doing more harm than good.

Totally agree.

- Do you need to do 30 minute darks if stacking in PI? I know little about this but won't it optimize a different dark based on the master bias?

Not 100% on this. The master BIAS or in my case the Super BIAS I create in PI does not touch on any of the Dark frame creation? 

Thanks

 

[andrew s]

I think Olly makes good points. You can easily add more noise with darks and unless you are making scientific measurement adding some dark signal is just like increasing the sky background a bit. 

One other point.  The bias frame is measuring the offset in the CCD electronics and should be a constant for all pixels. In which case you can just subtract the average bias level from each pixel in the image frame! You would need to check your bias frame dose not have a gradient due to the readout process. This does not apply to CMOS cameras. 

Regards Andrew

[swag72]

10 minutes ago, andrew s said:

 You can easily add more noise with darks and unless you are making scientific measurement adding some dark signal is just like increasing the sky background a bit. 

Is there a general consensus as to how many dark frames are needed to NOT add noise? I ask as I need to take darks in one camera and I don't really want to add noise.

[andrew s]

The statistics are well known http://www.eso.org/~ohainaut/ccd/sn.html for example. I don't know if there is a consensus among imagers sorry. 

For scientific work I would do the sums based on the S/N I needed for a given use. 

For imaging with typical low dark current CCDs I think I would follow Olly.

Regards Andrew

PS invest in some liquid N2 cooling and you need never worry about darks again - frost bite maybe.

[swag72]

1 minute ago, andrew s said:

The statistics are well known http://www.eso.org/~ohainaut/ccd/sn.html for example. 

Ah of course...... that well known article ? If nothing else you made my head explode and eyes water with all those equations...... think I'll give that a miss and just hope all goes ok with my darks ?

Seriously I am sure someone may find that link helpful...... but it's not for me

[andrew s]

1 minute ago, swag72 said:

Ah of course...... that well known article ? If nothing else you made my head explode and eyes water with all those equations...... think I'll give that a miss and just hope all goes ok with my darks ?

Seriously I am sure someone may find that link helpful...... but it's not for me

I did say "well known" not well used or understood! Just remember you reduce the noise added by the square root of the number of exposures so 25 reduces the noise by a factor of 5 compared to a single dark. 

Regards Andrew

[Droogie 2001]

38 minutes ago, andrew s said:

I think Olly makes good points. You can easily add more noise with darks and unless you are making scientific measurement adding some dark signal is just like increasing the sky background a bit. 

One other point.  The bias frame is measuring the offset in the CCD electronics and should be a constant for all pixels. In which case you can just subtract the average bias level from each pixel in the image frame! You would need to check your bias frame dose not have a gradient due to the readout process. This does not apply to CMOS cameras. 

Regards Andrew

Not sure how to subract the average BIAS level from each pixel in the frame? Apologies if I missing something here? 

I think I will probably run a test stack without darks and see what results I get. I realise I would need to look at Olly's suggestion re defect maps as well.

I don't mind creating Dark frames but I definitely want to avoid adding noise. I am using the KAF-8300 chip in my CCD which is prone to noise? I think Sara uses one of these so I know from her images that she has good control over noise levels when Darks are used. 

[ollypenrice]

3 minutes ago, Droogie 2001 said:

Not sure how to subract the average BIAS level from each pixel in the frame? Apologies if I missing something here? 

I think I will probably run a test stack without darks and see what results I get. I realise I would need to look at Olly's suggestion re defect maps as well.

I don't mind creating Dark frames but I definitely want to avoid adding noise. I am using the KAF-8300 chip in my CCD which is prone to noise? I think Sara uses one of these so I know from her images that she has good control over noise levels when Darks are used. 

I've never owned an 8300 chip but our 11 megs push out more noise than a racing Norton. The darks looks like a snowstorm. For all that, it is commonplace for me stack and calibrate as I describe and to have literally no bad pixels to clean at the end. 

The funny thing is that, in processing, I have more trouble with noise from the ultra-clean Sony chip. This noise takes the form of inconsistent pixel values in the background sky, with some pixels being too dark relative to the others.

I followed AstroArt's instructions in making my bad pixel maps. Master dark with 2000 ADU clipped off the bottom. That's it. In the stacking page there's a place to put this BPM.

Olly

[swag72]

I'm not using darks with the KAF8300...... My calibration in APP uses bias, flats and a BPM.

[Droogie 2001]

7 minutes ago, ollypenrice said:

I've never owned an 8300 chip but our 11 megs push out more noise than a racing Norton. The darks looks like a snowstorm. For all that, it is commonplace for me stack and calibrate as I describe and to have literally no bad pixels to clean at the end. 

The funny thing is that, in processing, I have more trouble with noise from the ultra-clean Sony chip. This noise takes the form of inconsistent pixel values in the background sky, with some pixels being too dark relative to the others.

I followed AstroArt's instructions in making my bad pixel maps. Master dark with 2000 ADU clipped off the bottom. That's it. In the stacking page there's a place to put this BPM.

Olly

Liked the Norton reference that gave me a chuckle!

Hmm. Will take a look at BPM's, though I use PI for my calibration etc so will need to figure out where it goes.

Sara: Very intertested to hear that you do not use darks with your KAF8300, apologies for the assumption.

SGL is great for looking for an answer on one topic only to find that there maybe a half dozen better answers!

Thanks all.

[martin_h]

On 25/09/2017 at 12:29, swag72 said:

I'm not using darks with the KAF8300...... My calibration in APP uses bias, flats and a BPM.

+1 for that method, I've used a 8300 chip for quite a few years and never use darks.

[Starlight 1]

Not read any of the posts but in the land  of logarithms odd numbers works better than even , PS going by post title.

[Droogie 2001]

I am currently experimenting on a PixInsight rerun with a BPM rather than dark frames. The BPM list took a while but I think I have 95% of the troublesome blighters.

My data set is quite small (3.5 hours) so perhaps not an ideal test but compared to the image I created with the Dark Frames included then there is definitely no increase in noise, whether there is less is hard to say. The main issue I am having is that every time I rework an image I improve upon the previous version which whilst nice is not helpful when trying to make comparisons!

I originally assumed that darks were required as that seemed to be preferred method when processing data that wasn't say a Sony CCD chip, useful to know that this is not necessarily the case.  
At least this may help stop me having to keep building Dark Librarys.