When is it time for new calibration frames.....

[swag72]

I've seen quite a few threads on here and been asked on a number of occasions about when new calibration frames are needed. As I have a permanent setup I take my calibration frames (Bias, darks and flats) and then reuse them for a period of time until I start to see something amiss. The purpose of this thread really is that it may help someone reading it to diagnose when / if the problem is calibration frames.

I generally know when I need to replace my calibration frames when I start getting wiggly sort of diagonal lines on my integrated and calibrated data and usually my calibration frames will last me about 6-8 months before needing to be redone. Imagine my surprise when looking at the latest project when I saw vertical lines down the stacked data...... 

1) I stacked the data without ANY calibration frames and somewhere amongst the dust bunnies there was enough space to see that the vertical lines were not visible

It had to be calibration frames. So I redid all of them yesterday and re-stacked the resulting data to check that all was well. 

As you can see below I've added a comparison between both stacks, cropped to show the offending area. Each stack had curves and levels applied to each, they have been stretched so that the problem can be seen to be there and be resolved...... I was very surprised to see the difference in noise levels at the very least. But the new stack had MORE bias frames.

Stack 1) 75x bias, 30x darks, 45xflats
Stack 2) 250x bias, 30x darks, 40xflats

 

[Filroden]

That’s a really useful side-by-side comparison. Do you think it was all calibration files or just bias or darks causing the issue? I take flats every session but I rarely redo bias or darks. New bias is a quick job but rebuilding a dark library can take a day or two so not something I want to do often

[Allinthehead]

I found Astro Pixel Processor to be very useful recently for this exact purpose.

When i stacked my data i found a blue cast over the entire image (using a colour cam btw).

I was able to switch from linear to calibrated while a stretched image was in the view window telling me straight away that it was indeed calibration that caused the issue. Then it was just a matter of figuring out which of dark, bias or flat that caused the problem.

It was the bias frames which i had taken about a month ago. 

[alan4908]

I've also been surprised by how much noise can be generated by old calibration data. I generally find that my DARKS and BIAS frames last about 6 months, however, my flats last much longer since I have a sealed telescope (eg a refractor) and I never remove the camera.

I've also found that following Adam Block's advice for acquiring and stacking the calibration data helps.  This is basically:

1. Take long dark frames and then let your stacking software work out the required scaling factor for the lights. To work out the scaling factor, the advice is to minimize noise, rather than simply use time, assuming your stacking software supports this. I use 30mins DARKS since this is the longest duration exposure that I take.

2. When stacking DARKS and BIAS frames, the advice is to discard a few of the top and bottom pixel values for each pixel stack since this will give a better representation of the camera performance.  Since cosmic ray hits are more likely to add to signal rather that detract from it, an asymmetric MIN/MAX clipping approach is suggested - basically 2, 3 for bias frames and 2, 4 for darks. Then take the average of the remaining stack to work out your MASTER BIAS and DARK. 

Alan 

 

 

[vlaiv]

@swag72

Could you do some comparison between master calibration frames, just to let us know what has changed between them.

Probably easiest thing to do is post (new master - old master), it will show the differences nicely.

Also what sensor is that?

[Barry-Wilson]

1 hour ago, swag72 said:

Imagine my surprise when looking at the latest project when I saw vertical lines down the stacked data...... 

Startling coincidence Sara - I calibrated some subs from my home observatory yesterday after taking a set of flats and saw exactly the same vertical stripes on my stacked channels.  Could only conclude it was time for new calibration frames . . . .

[swag72]

3 hours ago, Filroden said:

That’s a really useful side-by-side comparison. Do you think it was all calibration files or just bias or darks causing the issue? I take flats every session but I rarely redo bias or darks. New bias is a quick job but rebuilding a dark library can take a day or two so not something I want to do often

I tried stacking all the subs in turn with or without each calibration frame and I still got the lines....... So I'm not sure which it was, but just redid them all to be on the safe side.

[swag72]

2 hours ago, Allinthehead said:

I found Astro Pixel Processor to be very useful recently for this exact purpose.

When i stacked my data i found a blue cast over the entire image (using a colour cam btw).

I was able to switch from linear to calibrated while a stretched image was in the view window telling me straight away that it was indeed calibration that caused the issue. Then it was just a matter of figuring out which of dark, bias or flat that caused the problem.

It was the bias frames which i had taken about a month ago. 

I used APP for this stacking - And interestingly there was about 2 weeks between when these old calibration frames last worked and then they stopped.

[swag72]

2 hours ago, alan4908 said:

I've also been surprised by how much noise can be generated by old calibration data. I generally find that my DARKS and BIAS frames last about 6 months, however, my flats last much longer since I have a sealed telescope (eg a refractor) and I never remove the camera.

I've also found that following Adam Block's advice for acquiring and stacking the calibration data helps.  This is basically:

1. Take long dark frames and then let your stacking software work out the required scaling factor for the lights. To work out the scaling factor, the advice is to minimize noise, rather than simply use time, assuming your stacking software supports this. I use 30mins DARKS since this is the longest duration exposure that I take.

2. When stacking DARKS and BIAS frames, the advice is to discard a few of the top and bottom pixel values for each pixel stack since this will give a better representation of the camera performance.  Since cosmic ray hits are more likely to add to signal rather that detract from it, an asymmetric MIN/MAX clipping approach is suggested - basically 2, 3 for bias frames and 2, 4 for darks. Then take the average of the remaining stack to work out your MASTER BIAS and DARK. 

Alan 

 

 

I'm quite happy with my stacking and acquiring calibration frames Alan..... I like to throw everything into the pot and see what comes out with the minimal intervention!!!

[swag72]

2 hours ago, vlaiv said:

@swag72

Could you do some comparison between master calibration frames, just to let us know what has changed between them.

Probably easiest thing to do is post (new master - old master), it will show the differences nicely.

Also what sensor is that?

The old calibration frames are long gone! They took up almost 10GB!!! The sensor is a KAF8300

[swag72]

2 hours ago, Barry-Wilson said:

Startling coincidence Sara - I calibrated some subs from my home observatory yesterday after taking a set of flats and saw exactly the same vertical stripes on my stacked channels.  Could only conclude it was time for new calibration frames . . . .

Timely post then Barry! Wonder no longer!!!

[ollypenrice]

I don't use darks. I have a defect map made from a 30 minute master dark clipped by 2000 ADU and I subtract a master bias as if it were a dark. These calibration files are so ancient that I'd be embarrassed to admit their age on a respectable family forum like this one. But they work. (I did try reshooting a set of normal darks about a year ago but the old files worked better.)

Flats tend to last a few months for me, then something crawls onto a glass surface somehow and I need new ones.

Olly

[wxsatuser]

Had no problems with dark's and bias 10 months old from the Atik 383L.
The only reason I did the second set of flats after 7 months is I had to remove the camera.
I could not get the desiccant screw out, normally the camera is never removed.

[swag72]

I have to do darks for the QSI690 due to amp glow...... So far the new ones are working fine  

[martin_h]

Ive been having a lot of problems with flats recently, and having seen this post decided to shoot some new Bias frames - nothing to loose seeing as its cloudy, so I shot 250 frames the other night, and have just run a test stack with and without calibration.................... problem sorted!

So was it the Bias frames? who knows? But the problem has gone away so who cares!

[swag72]

I have come to the conclusion Martin that whenever things start to go wrong with the data stacks, it is generally calibration frames...... I can't answer why sometimes mine have lasted literally 8 months and the last set didn't even last a month....... but when it starts to go wrong I at least know now where to start my search!

[martin_h]

I agree, at least with a cooled CCD its reasonably easy to shoot off a new set of frames one evening.