DSLR imaging - Do you really need Dark frames?

[Bizibilder]

Just as an experiment I have processes my recent "Witches Broom" subs both with and without dark frames in the mix.  The two images are shown here, cropped full size.  The only processing has been one identical levels stretch to bring out the background a little.  The images are presented in JPEG format.

Each stack contains the same 28 light frames, 30 flats and 28 bias.

I know which I think has the "better" background noise - which do you prefer?

 

[Alien 13]

2nd one but i have a feeling it might be a trick question i have never used a dark or flat yet but i might be willing to try it sometime.

Alan

[wuthton]

Personally I prefer the second one (the one with NO DARKS DSS in the file name  )

In the interest of full disclosure has any dithering/kappa SC gone on here? I've had my doubts about darks in the past, I don't bother with the Atik.

Matt.

[jambouk]

What dslr is it? Modified/cooled?

I can't see much difference (on my phone) but i can barely see any broom either.

James

[ronin]

May be a stupid question but if it was from a DSLR is the noise reduction feature turned off as otherwise the DSLR will be taking a dark itself and subtracting that from the image before you get the final image presented to you.

In effect the DSLR could be doing the dark anyway, and so you actually have an image which is processed with a dark.

[Bizibilder]

These were stacked from RAW images, no dithering or other tricks and no camera subtracting its own darks.  The camera is a modded Canon 1000D.  And, yes, the "no darks" is the right hand image.

[jambouk]

How long was each light frame? Have you tried to process the stacked images harder to get more detail in the broom? Might the need for darks be more apparent then?

James

[D4N]

Depends how long the subs are I guess, with 30 second subs and cooldown time between them the sensor shouldn't heat up too much.

I've noticed my darks are pretty dark at 30" with very little noise at all.

Sent from my iPad using Tapatalk

[Alien 13]

Thats very interesting and has made my mind up to carry on without darks and even lights nice and simple.

Alan

P.S I normally shoot continuos subs in live view mode and not had any real problems with noise.

[Bizibilder]

Sorry - should have said - all subs and lights are 5mins with 5 seconds between.  Bias are 1/4000 sec. Flats are taken with the camera on "Av" - in this case they were 1/40 sec

[themos]

If you dither between light frames, it makes the requirement for dark frames even less pressing.

[Alien 13]

A quick question i have not yet taken bias frames should i do this or not bother if i am not using darks?

Alan

[jambouk]

It would be interesting to see a similar comparison with shorter subs; 60-120 second subs.

Do you use hot pixel detection and other "noise reducing" methods in stacking and in processing?

[Knight of Clear Skies]

I prefer the second, but there isn't much in it.

You might get different results with darks depending on the number of subs. With 28 subs the noise should average out pretty nicely anyway.

I'm not sure how much my darks are helping my images either, the ones without aren't noticeably noisier although I haven't tried a scientific test. As a mobile imager it's only practical to take my darks while travelling home with my camera in a bag, so the temperature might not match that well. I give the camera a bit longer between shots to try and compensate, but it's all a bit reckon. I wonder if slightly off darks are worse than none at all.

[Knight of Clear Skies]

One more thought - it might be worth digging into the EXIF data to see how much the sensor temperature fluctuates over the imaging and dark runs.

A quick question i have not yet taken bias frames should i do this or not bother if i am not using darks?

Alan

In theory, yes, bias frames should help remove any systematic errors from the sensor regardless of whether dark frames are used.

[wxsatuser]

What happens if you stretch it more?

[andrewluck]

Craig Stark found some interesting anomolies with Canon DSLRs when he compared them to CCD cameras in this article: http://www.stark-labs.com/craig/resources/Articles-&-Reviews/DSLRvsCCD_API.pdf.

In particular, he surmises that "Canon is using the mean signal in the shaded (optical black) portions of the sensor to re-scale the image". Note that this is a RAW image he's talking about. The net result of this is that while you may remove some hot pixels by subtracting a dark frame you will add noise to the image. For this reason, I always rely on dithering and a hot pixel / bad pixel filter to remove noise with a DSLR and never bother with dark frames. Flats are far more important.

Andrew

[Bizibilder]

Thank you for that link!!  I knew I had seen an article in the dim and distant past but no amount of searching brought that one up! (Probably my poor search technique  ).  I'll give that a read tomorrow when I'm a little less sleepy!

[wxsatuser]

A quick question i have not yet taken bias frames should i do this or not bother if i am not using darks?

Alan

Look at the calibration process, bottom of page.

http://deepskystacker.free.fr/english/theory.htm

[Psychobilly]

I ended up not using darks with my modded 1000D either just dithering  plenty of subs and Kappa sigma stacking in DSS...

Peter...

[dph1nm]

I think the answer is  it depends. I never use darks at ISO1600 for my 1000D, but at lower ISO they are needed as large scale bands start to appear which are not present in the bias.

BTW - if you use flats you will need to take either biases or darks.

NigelM

[IanL]

Going to be the (slightly) dissenting voice here.  I don't think the second (no darks) image is much better than the first image, partly because it's not really a direct comparison, and partly because it is worse in some respects.  Let me explain:

- The histogram stretches are not the same between the two images.  Judging by eye, the no darks image has a lower mean background level.  I can't really tell whether the foreground in the no-darks image is brighter and , but either way there is a bit more contrast in the no darks image, which combined with the slightly darker background makes it look more visually appealing.  A minute or two of processing the histogram stretch on the first (darks) image would easily achieve the same mean background level and the same foreground contrast.  Only if that was done would it be a fair test between the two methods, at the moment you're not comparing apples and apples.

- There are loads of hot pixel artefacts in the no darks image.  I've taken the liberty of highlighting a couple of the big/obvious ones below, (the two pairs of yellow boxes/lines).  There are others if you look carefully. There are lots of smaller ones all over too: if you look at the top yellow box, the top half is the darks image and the bottom half is the same part of the no darks blown up a bit - you can see plenty of speckles of red, yellow and green hots in there - I don't think they are down to the contrast issues mentioned above though, as I have done similar tests myself.  Now you can certainly remove hot pixels without darks (e.g. I have done it pretty well using PixInsight cosmetic correction), but a good master dark will do the job just as well.  In theory, dithering ought to kill off hot and cold pixels after stacking (since they appear in different places each time they average away or can be rejected), but in practice in my well dithered images I still get the same sort of coloured blobs as you have if I don't use darks.

(Click image to enlarge so you can see the detail).

- The acid test is how much more noise the darks add vs. the improvement they bring (if any).  Again looking at the top yellow box I can't see much difference between the two (the no-darks image background looks slightly noisier to my eye, but it might just be the contrast levels making it more obvious).  Really you need to measure the noise to prove things one way or the other - I'm working on it myself with my own images, but progress has stalled due to work and other things.

As an aside, my tests have shown that the EXIF temperature of lights and darks (for my Canon 500D and probably most other Canons at least) is absolutely no use for matching darks to lights.  Measuring the EXIFs and noise levels on a large library of darks, I can tell you for a fact there is no correlation between the two.  (The noise does correlate well to temperature and isn't affected by the Canon on-camera processing in the Craig Stark article). From what I have read, the temperature sensor monitors one of the processing chips to prevent it overheating in live view mode and bears no real relation to the sensor temperature which is what you want to know.  The best method I can come up with is to measure the Median Absolute Deviation (MAD) of the dark frames (which is a good way of measuring the noise) and match those that have the same or similar values, since that indicates the sensor was at the same temperature.

I'm using PI's dark scaling to try matching darks to lights, but I can't think it would be easy to do the same in something like DSS.

By the way, none of this is to say I am convinced you must use darks with a DSLR - I'm still very much in two minds about it myself, and have certainly managed to get similar results without them - albeit you do need to take extra processing steps to deal with hot/cold pixels, etc.

[Psychobilly]

 Light's , flats and bias were all I used to use... Mis-matched darks used to cause more problems than they were worth so I used to spend the time getting as many lights as possible...

Peter.....

[Alien 13]

Has anyone tried the canon inbuilt dark option i know it takes up imaging time but in theory it must produce a perfectly matched dark for each sub?

Alan

[D4N]

Isn't it immediately after the light? I would have thought that this would mean the sensor is warmer for the dark.

Sent from my iPad using Tapatalk