Jump to content

Enough with darks flats, they are useless


Recommended Posts

Hi Cyril,

thanks for this interesting read! When I take flats (SW Esprit 150ED with ZWO ASI1600 Pro Cool and ZWO filters) I always do it in one sequence. To avoid too short exposures (which will cause banding due to the flickering nature of the flat panel I use), the exposure for luminance starts around 1s, as a result of which Red is taken with an exposure time of 2.5s, followed by Green (4s), Blue (11.5s), H-alpha (34s), S-ii (59s) and ending with approximately 67s for O-iii. In your article you wrote "However, a flat is an image taken in full light, at times very short generally less than 5s." Clearly this is not the case in my workflow, while you imply that masterdarkflats are useful for longer exposures. So I would like to turn the question around in the hope you can elaborate on that:

Upwards from what exposure-time should we use masterdarkflats?

Nicolàs

PS: the reason for my workflow is that I have an observatory and thus can permit to 'fire-and-forget': when I come back after several hours all subs have been taken. Same for darks...

Edited by inFINNity Deck
typo
Link to comment
Share on other sites

47 minutes ago, tooth_dr said:

@lock042 Is a dark flat detrimental to the final image, or is it useless because the bias does the same thing and therefore it is redundant?

Hi @tooth_dr,

Yes and no: Yes, because they add noise (depending on the amount of darks taken), and no as they will remove amp-glow.

A few years ago I did a test on the amount of darks, compared images corrected with 25 and 500 of them, to find out that 25 sufficed. The write-up of that test is on a Dutch forum, but should translate well when opened in Chrome: https://www.starry-night.nl/forums/topic/ruis-in-de-cmos-cameras-metingen-en-berekeningenn/#post-21015

Nicolàs

Edited by inFINNity Deck
  • Like 1
Link to comment
Share on other sites

37 minutes ago, lock042 said:

From a pure noise point of view, dark flat adds more noise as we show in the link. However, it is probably negligible.

But why make unnecessary frames when you can avoid it with synthetic biases?

I dont use Siril, is this possible in other software?

Link to comment
Share on other sites

13 minutes ago, inFINNity Deck said:

Clearly this is not the case in my workflow, while you imply that masterdarkflats are useful for longer exposures. So I would like to turn the question around in the hope you can elaborate on that:

Upwards from what exposure-time should we use masterdarkflats?

In fact that depends more of your ampglow (do you see ampglow flash in your masterflat?). Take a look at our figure 3: even if you do have some thermal signal you don't care. Don't forget your flat is full of light and the thermal signal is not a problem like it could be with your light frames.

Link to comment
Share on other sites

Issue that you mention - one of injecting more noise with flat darks - is non issue.

We control amount of noise we inject back by choosing number of subs we use to create our masters. Want less noise? Use more subs for masters.

Problem with bias only approach is that you will have incorrect flat calibration.

Using bias instead of flat darks can cause flats to over correct. If you don't remove dark signal - it will also be subject to flat calibration - but it was not subject to attenuation by optical train in the first place - so you'll end up with uneven offset signal (if your dark is uniform -and worse if it contains amp glow).

There is a reason why flat darks are used - because it is proper calibration of data.

 

  • Like 3
Link to comment
Share on other sites

11 minutes ago, vlaiv said:

Want less noise? Use more subs for masters.

Or just a constant. You can't beat it.

11 minutes ago, vlaiv said:

Problem with bias only approach is that you will have incorrect flat calibration.

Wrong. We wrote a tutorial here: https://siril.org/tutorials/synthetic-biases/#appendix-understanding-how-the-flats-correct-the-lights if you want to take a look.

The appendix details computation.

11 minutes ago, vlaiv said:

There is a reason why flat darks are used - because it is proper calibration of data.

Absolutely not. The most important in the process is the level you subtract to the image to have a proper calibration of data (and offset has same level than dark, you can check by yourself).

Just test it and you will see. And please, post the results here.

Edited by lock042
Link to comment
Share on other sites

2 minutes ago, lock042 said:

Wrong. We wrote a tutorial here: https://siril.org/tutorials/synthetic-biases/#appendix-understanding-how-the-flats-correct-the-lights if you want to take a look.

The appendix details computation.

It is not wrong - it is basic math.

If you have flat without removed dark - you have light signal and dark signal. Only light signal determines percentage of light - dark signal must be removed.

Say you have 70% vignetting at one pixel and your flat at that pixel is 7010ADU - 7000ADU being light and 10ADU being dark that you did not remove. Peak flat is 10000ADU or 10010 ADU because of dark (let's assume dark current is uniform).

So your scaled flat in that pixel is 7010 / 10010 = ~0.7003

There you go - we just got wrong flat - for 70% vignetting our flat will correct with 70.03%

It might not look like much - but once you stretch your data - that will show as improper correction

7 minutes ago, lock042 said:

Absolutely not. The most important in the process is the level you subtract to the image to have a proper calibration of data (and offset has same level than dark, you can check by yourself).

I have - many times. All of my cameras have measurable amount of dark current - and yes it corresponds for published dark current for those sensors.

Offset level is not the same as level of dark in any of my cameras.

  • Like 2
Link to comment
Share on other sites

17 minutes ago, vlaiv said:

It is not wrong - it is basic math.

You're right, but in your example you take a flat underexposed and a very high dark current to best match what you are saying.

Anyway, I take the example of a basic non cooled camera I have: Canon EOS 100D.

level of bias: 2048 (2^11).

level of dark (60s) : 2044.

It is the same especially when you compare to the flat. 

17 minutes ago, vlaiv said:

If you have flat without removed dark - you have light signal and dark signal. Only light signal determines percentage of light - dark signal must be removed.

Say you have 70% vignetting at one pixel and your flat at that pixel is 7010ADU - 7000ADU being light and 10ADU being dark that you did not remove. Peak flat is 10000ADU or 10010 ADU because of dark (let's assume dark current is uniform).

So your scaled flat in that pixel is 7010 / 10010 = ~0.7003

Then in this case take a constant that correspond to the value you want. We encourage users to test it. I'm sorry but it works, and it works well. Saving process time and disk space. Because yes, this is not a issue of noise.

Did you read the second link I gave you?

Edited by lock042
Link to comment
Share on other sites

3 minutes ago, lock042 said:

Then in this case take a constant that correspond to the value you want. We encourage users to test it. I'm sorry but it works, and it works well. Saving process time and disk space. Because yes, this is not a issue of noise.

Why would I take a constant to replace actual dark current that might not be constant - or rather never is?

Why should I forcefully push what is essentially wrong method of calibration over right one?

7 minutes ago, lock042 said:

Anyway, I take the example of a basic non cooled camera I have: Canon EOS 100D.

Did you try this on any dedicated astronomy camera with set point cooling?

Link to comment
Share on other sites

1 minute ago, vlaiv said:

Did you try this on any dedicated astronomy camera with set point cooling?

Yes. Example in the post are done with 294MC.

And dark of 300s has same level than bias, except in the ampglow flash

 

image003.png

Link to comment
Share on other sites

7 minutes ago, vlaiv said:

Why should I forcefully push what is essentially wrong method of calibration over right one?

15 minutes ago, lock042 said:

This not because you will repeat it's wrong that is really wrong :).

Link to comment
Share on other sites

4 minutes ago, lock042 said:

Yes. Example in the post are done with 294MC.

And dark of 300s has same level than bias, except in the ampglow flash

 

image003.png

Is that from a 294mc pro as it looks nothing like I've ever seen in mine, just curious?

Link to comment
Share on other sites

Just now, lock042 said:

This not because you will repeat it's wrong that is really wrong :).

You are correct - it won't be wrong just because I say it is wrong.

I've shown you very simple math that shows that you can't have correct flat if you don't remove offset.

You are further claiming that bias offset is the same as dark current offset, and that contradicts my measurements on numerous occasions.

It also contradicts common knowledge about dark current - why do we even have a term if it does not exist?

  • Like 1
Link to comment
Share on other sites

1 minute ago, vlaiv said:

Can you post mean ADU of 300s, 150s and bias that you measured in 294MC camera?

Mean is a non robust estimator. Median is a better one.

Here you have what you want. The level of a master dark at 5sec you can compare with the previous image: comparo_flat_MDF.png

Then in the the table here you can see median evolution:

std_comparison.png

Link to comment
Share on other sites

2 minutes ago, lock042 said:

Then in the the table here you can see median evolution:

You are saying that standard deviation is rising as exposure time but mean / median value stays the same?

What capture software are you using?

Link to comment
Share on other sites

5 minutes ago, vlaiv said:

It also contradicts common knowledge about dark current - why do we even have a term if it does not exist?

It contradicts common beliefs. But it is normal and this is why I encourage people to make tests!!

 

2 minutes ago, vlaiv said:

What capture software are you using?

The dark at 300sec come from the 294MC and NINA:

23:58:12: Reading FITS: file DARK_300s_G120_O30_T-10C_bin1.fit, 1 layer(s), 4144x2822 pixels
23:58:17: Running command: stat
23:58:17: B&W layer: Mean: 1922.3, Median: 1919.0, Sigma: 125.5, AvgDev: 4.9, Min: 1862.0, Max: 65532.0

Others example come from 294MM. In the post we did not show dark with longer exposure of 5s because it was not the purpose of the post.

 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. By using this site, you agree to our Terms of Use.