Lights/Darks/Bias/Flats ratio?

[all-on-black]

Hey,

I was just wondering what sort of ratio people tend to image with? I usually do about 40% of my lights for the darks/bias/flats.

However i've noticed, even at lower ISO's i'm still getting a fair bit of noise from my DSLR at prime focus.

If I up my darks/flats/ bias to perhaps 70%, will that reduce noise and increase quality of the images?

Thanks

[davew]

I don't believe there is a ratio. I also don't know where the idea came from.

Do as many calibration frames as you can to lower the noise. The more the better.

Dave.

[all-on-black]

I don't believe there is a ratio. I also don't know where the idea came from.

Do as many calibration frames as you can to lower the noise. The more the better.

Dave.

I'm not saying there is a ratio, basically just wanted to know if more callibration frames equals less noise.

But your post sums it up perfectly, I shall en-devour to do more in future then!

Thanks! 

[frugal]

Take as many as you can, especially for Bias and Darks where you can keep them from session to session, I have a master Bias made up from 200 individual bias frames, and a master dark from about 200 darks frames. The last lot of flats I took were at 1/3000s against a twilight sky, so I took 120 of them.

There is an interesting article about Bias frames on Blackwater Skies: http://www.blackwaterskies.co.uk/2013/09/pixinsight-dslr-workflow-part-1-bias.html?m=1 Where he shows the pattern of different numbers of bias frames, up to 330!.

[johnrt]

I use a cooled mono CCD instead of dslr, but the theory remains the same. I aim for a minimum of 100 bias frames and about 50 flat frames regardless of how many lights I have to work with. Don't forget to subtract the bias from your flats (and darks) before calibrating your lights with them.

[ollypenrice]

Yes, this is a very odd myth as it seems to me. A master dark is a master dark, a master bias a master bias and a master flat a master flat.

As regards darks and bias, the more you take the more, on average, the more accurately they'll represent and remove the noise in your lights. And for flats and lights, the more you take the more the non random noise in each will be neutralized by stacking and so the less will appear in the final image.

Since DSLRs are noisy a dithered guide would make a huge difference to final noise. WOrth looking into.

Olly

[narrowbandpaul]

As others have said there is no formula. The ideal number is an infinite amount of all frames. If you get near this you will have done well.

I like combining a ton of bias frames. It can show sub electron fixed pattern noise. I once used >1000 which was interesting. Using the subframe feature in maxim kept download times and file size down.

Paul

[narrowbandpaul]

I have no idea where this idea of a ratio came from. I assure you it's a myth, with no foundation in science.

Olly,,are you saying that stacking many frames will reduce the non random noise? When you stack many it's the random stuff you reduce so as to not inject any noise when you subtract or divide out the fixed stuff (vignetting and hot pixels etc)

[all-on-black]

I use a cooled mono CCD instead of dslr, but the theory remains the same. I aim for a minimum of 100 bias frames and about 50 flat frames regardless of how many lights I have to work with. Don't forget to subtract the bias from your flats (and darks) before calibrating your lights with them.

Very interesting stuff! I take it that to subtract bias from flats, I simply run a stacking process of bias and flats, as if it were a normal stack, then use the master flat as the flats instead of multiple flats?

I'm going to sort myself a library of bias and dark frames whilst it's cloudy then, in order to save imaging time.

[PhotoGav]

Here's another hypothesis to add to the mix:

It's better to stack an odd number of frames.

No idea whether this is true, it's something that I came across in my research at some point. Anyway, I stick to it and always use an odd number of subs. I have made a dark library of a selection of exposure times, using 49 frames for each, I always take 39 flats and have a master bias made from 199 frames.

Anyone know if this is a valid hypothesis?!

[m37]

I think the idea of the odd number is if a median calculation is made then there is a middle one with an odd number rather than the mean of 2 values being used but I can't see how this makes any practical difference. Might well be wrong on this point though.

I think if you're taking longer exposures e.g. 10 minutes or above, the law of diminishing returns would certainly be applicable; is it practical to take another 10 darks at 30 minutes each for the ever dwindling returns? Especially if you have work in the morning!

I always aim for 10 light frames minimum with the same number of darks, minimum of 50 bias frames (these are very quick!) And then a minimum of 20 or more often 30 flats.

Am I on the right lines here?

If dityhered guiding is what I think it is, it is a very interesting and ingenious idea.

[PhotoGav]

Dithered guiding is just a bit of reframing between each sub so you use a slightly different bit of the chip for each frame. This averages out the imperfections so that you end up with more signal from your target (please correct me if I'm wrong!). Easy to do in BackyardEOS, just click that option and make sure that you 'Enable Server' in PHD.

Definitely the thing to do... Any little trick that helps improve the final image is well worth doing.

[johnrt]

The odd number of frames theory is an easy one to prove or disprove. Upload two stacked images, one calibrated with an even number of subframes, and another stack calibrated with 1 less sub frame of each type. I'll run it through Pixinsight's statistics tool for the results.

Personally I think it will have no effect whatsoever - but I am prepared to eat my words!

[davew]

I've heard of the odd number Median idea. Not a clue how true it is.

If your chip, be it DSLR or cooled mono, is noisy ( Hot pixels ) then dithering gets you out of taking Darks in the main. A DSLR may have Amp or heat glow. Try it. I now use SD mask and NO darks and my chip will be far noisier at 30 minutes than almost any DSLR at 5 minutes.

Regarding diminishing returns...... The second sub is the start of diminishing returns  You don't get half the noise for twice the exposure time. Paul may will be able to explain it better than me so I hope he chimes in.

Dave.

[lensman57]

Hey,

I was just wondering what sort of ratio people tend to image with? I usually do about 40% of my lights for the darks/bias/flats.

However i've noticed, even at lower ISO's i'm still getting a fair bit of noise from my DSLR at prime focus.

If I up my darks/flats/ bias to perhaps 70%, will that reduce noise and increase quality of the images?

Thanks

Hi,

There is no ratio, the more the better if time allows. There is no problem with a CCD as normaly darks are not required, last night once I finished the session I just parked the laptop and the DSLR in the garage, put the power supply on and left it to do the darks at the same temp as the lights were taken as the garage is not heated and has a lot of large gaps for the cold to get through.

A.G

[narrowbandpaul]

The law of diminishing returns comes from the fact that in principle the SNR increases proportional to the square root of time or combined images.

Compared to a single sub, to double the SNR we need 4x as many subs, to double it again we need 4x as many again, and so on. A 10x improvement over a single sub will require 100 subs. So to get better and better you need to throw a disproportionate amount of subs at it.

Consider a single sub that detects S photons in a pixel. The random error on this is sqrt (S). And so the signal to noise is s/sqrt(S). I.e. Sqrt(S). If you take one more sub, or double the exposure length then you have twice the signal (as that is linear) and the random error is sqrt (2S). The signal to noise then is 2S/sqrt(2S), I.e. SNR=sqrt2*sqrt(S). But sqrt(S) was the SNR of our single sub. So in other words SNR for 2 subs or of twice the exposure length = sqrt(2)*SNR for the single sub. This is trivially extended for combining N images, the factor is just sqrt(N).

This assumes that the random error, or shot noise is the only factor. Read noise plays a part and analysis including this shows that it is beneficial from a SNR point of view to take a few long subs rather than many shirt ones (for the same total integration time). So a single 60min sub will be better than 60 one minute subs.

In practice this is borne out. However when stacking pixel rejection algorithms are used and these rely on having several subs. So this may be the best overall compromise

Hope that helps,

Cheers

Paul

[m37]

Dithered guiding is just a bit of reframing between each sub so you use a slightly different bit of the chip for each frame. This averages out the imperfections so that you end up with more signal from your target (please correct me if I'm wrong!). Easy to do in BackyardEOS, just click that option and make sure that you 'Enable Server' in PHD.

Definitely the thing to do... Any little trick that helps improve the final image is well worth doing.

Brilliant, cheers dude.

[glowingturnip]

agree with Paul - theoretically speaking, and for most platforms, as many of each type of calibration frame as possible is better, but the square root law does bring in diminishing returns , there's really not much point churning out your second or third hundred lots of darks coz they won't add much more.

I can think of one scenario where taking an equal number of lights and darks may be the best route though - on my noisy horrible DSLR, I can see the amp glow in the corner growing frame by frame until the chip reaches 'working temperature' when the difference between frames starts levelling out.  Same for the darks.  If i'm limited to only taking a handful of lights, then each frame is going to have different amp glow due to warm-up, so I should be using the same number of darks, taken from cold, to have the same warm-up, and strictly should be calibrating light1 with dark1, light2 with dark2 etc.  Doesn't apply for bias and flats, as many of those as possible still.

Clearly the proper way to do this would be to expose for many of each, and throw away the uneven ones at the start.  Actually an even better way would be to chuck my inadequate camera and buy a proper one, but that's sadly out of budget.

I'm not convinced that flats need as many takes as lights, darks and bias do though - the S/N for flats is already much higher than the others - exposed to about 1/3 of saturation, though the noise is only going to be as much as on a light frame, so S/N many times higher than a light frame.  However, they're quick and easy to take, so may as well bang off a load anyway.

[Davey-T]

The law of diminishing returns comes from the fact that in principle the SNR increases proportional to the square root of time or combined images.

Cheers

Paul

I think we've had this discussion before and agree with the above, I did a few experiments a while back and didn't see any improvement after 16 calibration frames.

As there are so many other variables with equipment, software, seeing and competence of the imager there's plenty of other stuff to screw up our images that need sorting rather than waste time taking hundreds of calibration frames.

Just my opinion  

Dave

[dph1nm]

Hmm - there is a ratio argument. If you have equal numbers of lights and darks (or biases), then when you subtract the master dark you will increase the read noise contribution in your final light stack by sqrt(2). If you have 10x as many darks than lights then the increase will be negligible. I think the same applies to the random component of the thermal noise.

NigelM

p.s. this statement ignores any effects of dithering.

[ollypenrice]

I have no idea where this idea of a ratio came from. I assure you it's a myth, with no foundation in science.

Olly,,are you saying that stacking many frames will reduce the non random noise? When you stack many it's the random stuff you reduce so as to not inject any noise when you subtract or divide out the fixed stuff (vignetting and hot pixels etc)

Ah no, I was trying to say what you said but I put a superflous 'non' into my sentence.       Humble apologies. Clearly the fixed pattern noise is reduced by dark subtraction and the random by muliple stacking.

Senior moment!

Olly

[ollypenrice]

I'm not going to shoot 100 thirty minute darks! I'm just not!!   

Actually I use AstroArt 5 for stacking and have gone over to a bad pixel map and a master bias as my calibration method. AA advise you to clip 2000 ADU off the bottom of an appropriate master dark to make a BPM. The software has a box to support its use. I find that a 15 min based BPM is perfectly good when used on 30 minute subs, too. Despite our noisy full frame Kodak chips we're getting remarkably clean stacks. Recently, touch wood, I have had literally no cosmetic cleaning to do on one of the camera's stacks. The older one has some difficult columns which do survive.

I used to find classical darks a bit unpredictable. Sometimes they seemed quite invasive and destructive. Sometimes not. I've no explanation to offer.

Olly

[narrowbandpaul]

Hi Olly,

I assumed there was a typo in there. Well all have our moments.

I have never used a bad pixel map but will experiment at some point I'm sure. As for doing 100 30 min darks. What's so wrong with that. It just takes 2 days of continuous imaging . I feel even worse for Tim with 2 hour subs!

It raise the point that the law of diminishing returns applies not only to light frames, but to darks, flats and bias. And costs. Think about how much you would have to spend to get just a small improvement in image quality. Beyond a certain point improvements whilst small come at great expense.

Paul

[all-on-black]

Well, this thread soon got extremely technical! All brilliant reading though, but I have to admit.. most of it went straight over my head!

Definitely looking forward to trying different levels to see what I can get.