Jump to content

sgl_imaging_challenge_2021_annual.thumb.jpg.3fc34f695a81b16210333189a3162ac7.jpg

Whats a DARK FLAT FILE?


Recommended Posts

Just putting some files into DSS and I keep meaning to ask...

I know what a FLAT file is.

I know what a BIAS file is.

I know what a DARK file is.

But what's a DARK FLAT file?

Thanks

Ant

Link to post
Share on other sites
It's an exposure the same length as the flat frame but with the lens/scope covered up. It helps to reduce the noise in the flat frames.

Regards

Kevin

So what is the difference between that and a dark then?

Link to post
Share on other sites

The flat frame is of an even field, with the camera decided the exposure length (AV mode).

So the Dark will match the Light in duration, but the Dark Flat will match the Flat in duration.

If I've got that right.

Ant

Link to post
Share on other sites

That's right Ant. One word of caution... If you are using Bias frames you don't need dark flats, and vice versa. If you use both then weird things start to happen (I think it's down to the removal of the bias current twice but I've no clue about that level of stuff)

I used to take a sequence of dark flats as well as the Bias frames, but you can save some time not taking the dark flats and use the stored Bias frames instead. Thanks to whoever, I think it was Martin, that pointed out the Bias frames.

  • Like 1
Link to post
Share on other sites

interesting, since light images experience fixed pattern noise, which is removed with flats, and since dark frames have dark fixed pattern noise, could one flatten a dark to remove dark fixed pattern noise.

Its worth an experiment......

paul

Link to post
Share on other sites
  • 11 years later...

The experts say to use darks, flats and bias frames with CCD sensors and darks, flats and flat darks  with CMOS sensors, from my experience this would appear to be sound advice.

  • Like 1
Link to post
Share on other sites
15 hours ago, tomato said:

The experts say to use darks, flats and bias frames with CCD sensors and darks, flats and flat darks  with CMOS sensors, from my experience this would appear to be sound advice.

Yes, the convenient shortcut with CCD is that a master bias can perfectly well double as a flat dark because the thermal noise built up in very short flat exposures is insignificantly different from that found in a bias. This isn't true for CMOS cameras.

Olly

Link to post
Share on other sites

This got me thinking about whether there is ever a context where a bias is actually necessary (as opposed to using a flat dark). The only one I can think of is when using 'thermal' frames, which I believe are used to scale a master dark collected at a long exposure so that it can be used at arbitrary exposures without having to collect darks at that exposure. In this situation the bias has to be subtracted from the dark itself to remove fixed pattern noise. Does anyone use thermal frames?

Martin

 

Link to post
Share on other sites
4 minutes ago, Martin Meredith said:

This got me thinking about whether there is ever a context where a bias is actually necessary (as opposed to using a flat dark). The only one I can think of is when using 'thermal' frames, which I believe are used to scale a master dark collected at a long exposure so that it can be used at arbitrary exposures without having to collect darks at that exposure. In this situation the bias has to be subtracted from the dark itself to remove fixed pattern noise. Does anyone use thermal frames?

Martin

 

That approach is viable with CCD sensors and it is good idea to shoot only the longest darks if you use multiple exposures and scale them for shorter ones as longest darks will have the least overall thermal noise (although it seems that the longest dark will gather the most thermal current and hence associated noise - it will be scaled and noise will be scaled as well - signal scales with time but noise with square root of that).

With cmos sensors it usually does not work as there are issues with bias files. I've tried 5 different CMOS cameras so far and not one had usable bias subs (for dark scaling).

Another place bias can be used is if you don't have set point cooling on your camera. In that case, it is good idea to try dark scaling (algorithm that tries to scale master dark to match that of sub it is calibrating) - again bias needs to be removed prior to dark scaling as bias signal does not depend on temperature.

Interestingly enough, DSLR CMOS sensors don't seem to be suffering from bias issues like dedicated astro CMOS cameras (or I could be wrong there).

Link to post
Share on other sites

A sanity check then guys for the CMOS world....  Do you have to take flat darks with a CMOS? With a CCD when PixInsight calibration routine scales a dark to the flats  it kicks off a warning about the dark having no correlation to the flats (because of the radically different exposure time).  This implies that darks with flats on a CCD do not matter that much.  Are we saying it *does* matter with a CMOS camera?  Why so?  PI does not know the light/flat was taken with a CCD or a CMOS does it?

Also, with a CCD the "normal" dark of say 20 mins can be scaled to a light of 10 minutes by using "half" of the dark.  Is this still the case with CMOS?

Link to post
Share on other sites
2 minutes ago, kirkster501 said:

A sanity check then guys for the CMOS world....  Do you have to take flat darks with a CMOS? With a CCD when PixInsight calibration routine scales a dark to the flats  it kicks off a warning about the dark having no correlation to the flats (because of the radically different exposure time).  This implies that darks with flats on a CCD do not matter that much.  Are we saying it *does* matter with a CMOS camera?  Why so?  PI does not know the light/flat was taken with a CCD or a CMOS does it?

I'll need clarification on this.

Flat darks (or dark flats) are in principle always needed - regardless of CMOS/CCD sensor type technology.

Flats work when they contain only light signal.

When we take regular sub, it contains numerous signal sources: 1. Bias signal, 2. Dark signal, 3. Light signal. Same happens with flats - except light signal being uniformly lit flat panel. In order to have only light signal remaining in our master flat file we need to remove both bias signal and dark signal (however small it is for short flat exposure - if we want to be completely correct).

We can do that in couple of ways (right and "wrong" ones):

1. Subtract Flat darks that contain both bias and dark signal - this is what we are saying here

2. Remove bias with master bias and then remove remaining dark signal with some means - like dark scaling or similar

3. Remove bias signal only - wrong in principle but in practice it will work 99% of the time since flat exposures are really short and cooled cameras have low dark current, so overall dark signal in such short time is minimal.

CMOS sernsor have issues with bias and it is best not to use bias if one can with CMOS sensors - for this reason, I always recommend number 1. option - for both CMOS and CCD - simply because it works properly.

9 minutes ago, kirkster501 said:

Also, with a CCD the "normal" dark of say 20 mins can be scaled to a light of 10 minutes by using "half" of the dark.  Is this still the case with CMOS?

No. Yes. Well it depends.

In principle - same thing. If you can remove bias than you are all OK - you can scale dark exposure as dark current depends on time (and temperature but we assume temperature is fixed in this discussion).

Problem is that one should not expect bias to be well behaved with CMOS sensors.

I've found at least 2 or 3 different scenarios where bias is not what one would expect with CMOS sensors.

1. Internal bias calibration

This is present on smaller sensors - usually planetary type cameras / earlier models. Each power cycle, sensor would perform internal calibration and some sort of automatic offset. This means that one can't expect bias to be on the same level between power cycles as offset can be (and often is) different. Solution was to shoot everything in same session without shutting down camera.

2. Multimode operation

For some reason offset in very short exposures was different than offset in long exposures. I think this was driver issue rather than sensor issue, but could be both - driver just trying to deal with sensor firmware operation.

It means that bias for very short exposure (like under one second - explained by ASI as internal camera clock being used to time exposure as opposed to "bulb" mode where application is timing exposure for subs longer than 1s) was different than bias for long exposure and bias subs are inherently shortest exposure possible.

I measured mean value of bias sub to be larger than dark sub on several occasions (and that can be true if bias is correct because dark sub = bias signal + dark current signal).

 

Link to post
Share on other sites
9 minutes ago, kirkster501 said:

Thankyou @vlaiv for your time in explaining this.

So, in practise in the real world, what would one do with a CMOS to calibrate lights and flats in terms of what calibration frames are required ?

You take a set of flat darks (dark flats - or any other word order :D ) and you create master from those (stack them). You take your flats and you stack them and subtract master flat dark - this creates master flat.

You create master dark like you normally would (no dark scaling so you need to match exposure of your lights exactly).

Or in formulas:

master_flat = stack_of_flats - stack_of_flat_darks

(flat darks match flats in everything)

calibrated_light = (light - master_dark) / master_flat

(darks match lights in everything).

You need flats, flat darks, darks to do proper calibration (with any camera, and CMOS in particular).

Link to post
Share on other sites
  • 2 weeks later...
  • 5 months later...

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. By using this site, you agree to our Terms of Use.