Do I need darks and flats for planetary work?

[narrowbandpaul]

Hi Tim...yes I have lost none of my diplomacy at all

maybe we can estimate the approximate non linearity for a CMOS chip....

lets say a 3/4 full well exposure, full well of say 30ke-, read noise of 20e-, PRNU of 1%

a single image will have a noise of

sqrt(20^2+(3/4*30000)+(0.01*3/4*30000^2)

=271 e-, giving an SNR of... 83

after combining 100 images....

Noise= 226e- giving an SNR of... 100

if we flat field, then the PRNU is zero...and after combining 100 exposures...

Noise= 15e- giving an SNR of 1486...

so given somewhat typical values (CMOS sensors certainly show more FPN than CCD sensors) there is a very significant increase in SNR once you flat field.

the brighter the object and the higher the PRNU and the more exposures you choose to combine (imagine if 1000 frames were combined!) the more prominent any FPN is, and so flat fielding becomes crucial....

hi martin

Yes I hear you shouldn't expose in to the non linear signal levels...but as CCD's are typical linear to a few % over the entire DR non linearity shouldnt show itself until say 90%...

but the signal of each flat isnt the really important thing. If you go through the maths, then the quantity total signal is important...that is the product of no. of flats*signal in each flat....this number has to be big (say 500,000e- or more) depends on how bad the PRNU is.

so you can take flats at say 2/3rds full well, which might be around 20,000e-, and then take >25 flats, and this should work well. Or 1/2 signal flats at approx 15,000e- and take around 35 of them....and so on. The more signal you have altogether in your flats, the better they will work. Obviously the number you need depends on how many electrons each pixel can hold and how bad the fixed pattern noise is.

hope this helps

paul

[sixela]

The thing is, with planetary images you're stacking to beat the seeing (i.e. extract what is behind the convolution of the planet's signal with a varying speckle pattern), not noise caused by the sensor or even the low SNR of the planet vs. other sources of energy --the planet's pretty bright.

So getting that sensor-related SNR from 100 to very high levels is fixing a problem that really isn't your main problem at all.

Sure, it'll help. But whether the time spent on it isn't spent more wisely on doing other things isn't at all certain.

/begin{tongue_in_cheek}

If you have a hammer, everything in the world starts to look like a nail. But there's more than nails in the world.

/ed{tongue_in_cheek}

[Tim]

Thanks for the explanation Paul, I am sure some people will find it useful. I have number blindness though, are there any pictures that illustrate the differences? I'd be very keen to see them? Have you managed to take any yourself?

Cheers

Tim

[TheThing]

Yes, what a good idea Tim. A picture speaks a thousand words, as they say...

[narrowbandpaul]

as you know Im no planetary imager. However a search of FPN removal on google should show what a difference flat fielding can make. There are example images in James Janesicks bible 'Scientific CCD's'

with regards to sixelas comment....

I am very aware why one takes several hundred images...I know the theory very well behind lucky imaging. The idea being to gather enough frames to achieve a reasonable number of diffraction limited images. The number falling sharply as integration time increases and r_0 (the fried parameter) falls.

One acquires maybe several thousand images, and a reasonable number, maybe several hundred will be quite sharp and will be used in the final stack. So hundreds of frames still end up getting stacked. I know you dont stack all the images, but there is still a significant number which means the random noise gets driven down meaning the FPN becomes the SNR limiting noise source.

Since SNR is the measure of observation quality, or image quality, it is certainly desirable to have as high an SNR as possible

the fact that the planet is bright is exactly why you MUST flat field. The FPN scales linearly with signal...more signal=more FPN=more need to flat field. FACT.

Once you have gathered enough diffraction limited frames....you have done all that you can. The issue now is to bring out detail. This involves varying degrees of post processing...the higher the SNR, the more you are able to do.

you may as well flat field....you could increase your SNR by approx an order of magnitude!

[riklaunim]

I have a flat and a dark for Saturn I've got today in almost daylight and I'll see what will be the difference between no-dark/flat and with-dark/flat stacks

[narrowbandpaul]

will be good to see the results...

are the flats and darks of high quality...ie plenty stacked together

paul

[riklaunim]

About 600 frames for Dark and Flat, the Saturn Avis are bit over 2000 frames (the stack will be lower) - one set at 1/30 and one set at 1/20 with bit lower gain. Flat looks good as it was almost daylight. Even those very tiny dust particles were visible. Bigger ones got cleaned before imaging.

[narrowbandpaul]

excellent...sound like very high quality flats and darks....

look forward to the results

paul

[sixela]

the random noise gets driven down meaning the FPN becomes the SNR limiting noise source.

The point is that the FPN only becomes the limiting noise source once the "random" noise is low enough, and you haven't convincingly demonstrated that it has actually become "low enough" for this to happen.

I don't know if you've seen a Hubble Jupiter image processed and then convoluted with the diffraction pattern of, say, 12" of aperture (Aberrator can do that for you, even for obstructed apertures with a number of aberrations), but it's pretty clear the random sources of noise aren't gone from anyone's images just yet.

And that what still discriminates, say, Bian Comb's or Damian Peach's best images from Hubble images is still not the rather typical fixed noise pattern of the sensor and readout electronics. Of course, it's only "anecdotal" evidence, but don't diss the eye/brain combination as a rather good detector. Unless you can show me that the "random noise" is low enough (with a quantitative analysis)...

Since SNR is the measure of observation quality, or image quality, it is certainly desirable to have as high an SNR as possible

No argument from me. Shaving your legs also reduces drag when you're running with shorts. That doesn't mean everyone not doing it is an idiot .

Once you have gathered enough diffraction limited frames....

Uhm - you mean enough seeing limited frames. The problem is that "enough" to completely cancel the seeing is not a realistic goal, due to several factors (which undoubtedly you'd know if you actually did image). Some --like planet rotation which changes the shape of features far from the meridian and makes some even disappear, limiting the number of images you can stack-- are rather hard to cancel out.

you could increase your SNR by approx an order of magnitude!

If you have eliminated the other errors!

We are all awaiting your images which are as good as those of space based telescopes of equal aperture with bated breath.

I agree that on those images, darks and flats will indeed be really interesting; I'm sure the Hubble space telescope sensors and image processing software do use these (or something similar, like a very good model of the errors characterised when the scope was built) when they produce a Jupiter image.

[riklaunim]

So here are my results for Saturn at dawn, 1/30 sec exposures. 2400 frame stack of Saturn, and ~600 frame stack for flat and dark. Everything in Registax. The AVI is 8-bit so flat and dark are BMPs.

You can see that with flat + dark there is no circle "gradient" (which is visible on stretched flat). Raw images attached.

Processed dark+flat stack looks like so (others similar):

dark.bmp

flat.bmp

nothing.tif

dark-only.tif

flat-dark.tif

[toml42]

First of all, nice shot of saturn!

To my eyes in the raw frames there is a noticeable contrast improvement on saturn, particularly in the rings when dark and flat are applied, as well as the removal of that circular gradient.

since you have only included one processed image, did you find there to be little difference in the final result?

[riklaunim]

Except of fixed ilumination of ring corner it's hard to find anything else that did noticeably changed at first look.