Whats a DARK FLAT FILE?

[vlaiv]

5 minutes ago, drivera said:

Thanks for that. that's quite interesting.  Sorry to sound so naive how would I learn and understand about stable bias for my camera and then look at dark optimisation with different darks? or would that be irrelevant on a DSLR?

 

I think that most DSLRs have stable bias. I noticed issues with bias only on non cooled dedicated astro cameras and even some cooled ones.

For example - my ASI1600 - has useless bias (although it is cooled camera). It has higher average ADU value than that of dark - which should not happen as dark is bias signal + dark current signal while bias is just bias signal and dark current signal can't be negative. This has something to do with different way exposure are timed - anything less than 1s is timed on sensor and above 1s is timed on computer. In other cases - sensors have auto calibrating bias - each time camera is powered on it adjusts its own bias level automatically - but problem is - it is different each time.

In order for algorithm to properly scale darks - it needs to have dark signal only with bias removed, and for that you need good bias files.

Here is a simple test that you can do to see if your darks are properly scalable.

Take camera and shoot:

- some bias subs  (you don't need a lot of then - something like 8 of each is fine).

- some dark subs of certain duration - say 30s darks

- some dark subs of double duration - say 60s darks

Stack each to their respective stack using simple average algorithm and then using pixel math create following image: 2*(30s_dark - bias) - (60s_dark - bias)

Resulting image should have average ADU value of 0 and no discernible patterns in the image - it should be pure noise.

If you don't know how to do all of that - I'll do it for you if you wish - just shoot the files and send them over.

Actual dark optimization is very simple to do - provided that it is implemented in software - like in DSS:

You just check appropriate checkbox and DSS will to that for you. You only need to have both bias and darks for your calibration.

 

[Seelive]

28 minutes ago, drivera said:

I'm jumping on this post.. I'm new to this. 

I'm using a DSLR (Sony A7iii). As an example I'm going to take some pictures tonight from say 11pm-1.30am.  Can I / should I take my dark frames prior to me kicking off my session? Instead of getting up at 1.30am to then put the lens cap on and run a session of say 10-20darks, could I do this at 10pm with my dark frames? Or will the temp be too far off my imaging and won't work?

Trying to think of preserving my sleep on these weeknights haha.

I wonder if you actually need to take Darks?  At this time of year, the sky doesn't get really dark from the UK so if the natural (or unnatural if you're imaging from a light polluted location) 'sky noise' exceeds the camera noise then I would have thought that using Darks wouldn't add much benefit if you're dithering (intentionally or not) and stacking with a sigma rejection algorithm.

[vlaiv]

1 minute ago, Seelive said:

I wonder if you actually need to take Darks?  At this time of year, the sky doesn't get really dark from the UK so if the natural (or unnatural if you're imaging from a light polluted location) 'sky noise' exceeds the camera noise then I would have thought that using Darks wouldn't add much benefit if you're dithering (intentionally or not) and stacking with a sigma rejection algorithm.

Darks don't serve to remove noise from image (which is random component of signal) - they serve to remove fixed signal from the image.

Dithering will turn part of fixed signal into random signal, but dark current level will not be removed with dithering. That makes flat calibration wrong. Flat calibration tries to correct attenuation of light that comes from objective lens. If you have some residual signal in image that is not light and you apply flats - you'll correct that signal - that does not need correction - so you'll "over correct it" - or you'll have issues with your flat calibration not working.

That is main reason why people should use darks - otherwise, bias is really enough to remove almost all that fixed signal / pattern present in the image.

If dark current of particular sensor is not strong enough then issue with flat calibration might be very slight and might not even show - in that case, well, don't bother with darks, just use bias (if stable of course).

[Seelive]

Agreed, I worded that poorly. My understanding is that darks are intended to remove the 'correlated' noise from the lights (but will still increase the uncorrelated random noise) and that in the absense of darks dithering with a sigma clipping algorithm will have a similar effect.  However, in the absence of dark flats (or flat darks), I though that only the bias is used to calibrate flats, not the darks?

[vlaiv]

1 hour ago, Seelive said:

Agreed, I worded that poorly. My understanding is that darks are intended to remove the 'correlated' noise from the lights (but will still increase the uncorrelated random noise) and that in the absense of darks dithering with a sigma clipping algorithm will have a similar effect.  However, in the absence of dark flats (or flat darks), I though that only the bias is used to calibrate flats, not the darks?

You are right in first part - it is enough to subtract bias to remove most of 'correlated' noise to use that term. Dithering is beneficial regardless if you use darks or not as it "shuffles" around that uncorrelated bit from calibration frames (be that darks, bias or flats).

When I say that not using darks will mess up flats - I'm not thinking about flat darks - but not using those does the same thing (to lesser extent).

Imagine following scenario - we observe two pixels - one vignetted with light at 70% and one in center of image with light at 100%.

Both receive same light signal of 100ADU.

There is dark current signal that is say 5ADU.

Bias is removed.

Now we have image that looks like 75ADU and 105ADU (that is with dark current and vignetting). We have two options - remove dark current signal (dark calibration) - or leave it.

Let's leave it and apply proper flat calibration. Flat calibration in this case will be to divide first with 0.7 and divide second pixel with 1.0 (opposite of 70% and 100% light throughput). After calibration we should get "flat result" - or both pixels should have same value.

75 / 0.7 = ~107.143, 105 / 1 = 105

We have slight over correction of our vignetting - we did not get proper values - equal values, because we did not remove dark current signal. Let's now try with dark calibration:

(75 - 5) / 0.7 = 70/0.7 = 100 and (105-5)/1.0 = 100/1.0 = 100

Now we get perfect correction.

That is why we need to do dark calibration - if dark current is significant in comparison to recorded signal and we are doing flat calibration - flat calibration will fail if we don't remove respective dark current signal. Same happens if we don't do flat darks, but flat darks are usually very short because flats are short and can be substituted with bias in most cases - unless you want to be 100% correct in the way you are calibrating your data.