software to calculate darks for various temperatures?

[alcol620]

Hi Folks

Does anyone know of any software, free or otherwise, that will take existing dark subs and modify them for other temperatures?

Thanks

[Cornelius Varley]

I wouldn't have thought that possible.

[D4N]

Yes PI does that.

Sent from my iPad using Tapatalk

[alcol620]

THanks D4N

Anything else anyone? Is there any likelihood that it can be added to DSS?

[perfrej]

If you're talking DSLR (Canon), they really aren't scalable as the camera is indeed doing some magic stuff behind the scenes. Craig Stark did a great analysis of it and found tons of fun stuff...

/per

[sharkmelley]

If you're talking DSLR (Canon), they really aren't scalable as the camera is indeed doing some magic stuff behind the scenes. Craig Stark did a great analysis of it and found tons of fun stuff...

/per

They certainly are scalable - I use it as part of my main processing sequence.  Dark scaling really is the only way to use darks on any camera that doesn't have set-point temperature cooling (such as DSLRs and some cooled astro-CCDs).  In fact I can't emphasise this enough - any other method will result in disappointment.  Point blank.

PixInsight, IRIS and DSS all do dark scaling (by optimisation) but these are the only ones I have experience of.  My guess is that most other astro-processing packages also do it.

Mark

[alcol620]

Hi Mark.

I use DSS with my DSLR frames, does it do the scaling automatically or are their settings that have to be tweaked? How does DSS know the temperature of the Dark frames to start with?

Regards

Alec

[sharkmelley]

You've identified the key issue!  The software does not have to know the temperature.  Instead it works by taking the each light frame in turn and working out what scaling to apply to the dark (before subtraction) to minimise overall noise in the light frame./

For DSS, read the section on "Entropy-based Dark Frame Subtraction" here:  http://deepskystacker.free.fr/english/technical.htm   It even provides a link to a mathematical document describing how this works.

The only "gotcha" is that you should create your master dark at a ambient temperature slightly higher than your typical imaging night because DSS will only scale a dark downwards i.e. it will only use a scaling factor in the range 0-1.

I update my (Canon) master dark once a year because of changes in the sensor itself (due to ageing).  I'm now migrating to Sony where I will need 2 master darks - one for Bulb mode and one for non-Bulb mode - that's because Sony Bulb mode uses some additional raw processing.  On Nikon, darks don't work effectively because of black clipping (unless you have installed the  Nikon "hack").

This is another good argument for Canons!

Mark

[alcol620]

Hi Mark

Thanks for that. Good news indeed. I have been trying to build up a library of Darks at different temperatures so that they can be matched up roughly to the lights. Obviously this is a bit of problem because at different times of the year the temperature of a DSLR can be significantly different due to the overall ambient temperature. If DSS does what you say, then there seems no requirement to build up this library?

Presumably it would be important to have all the individual Darks at a similar temperature to enable DSS to produce the master dark. What would the result be if they were at different temperatures.

My next action is to read the article you referenced.

Regards

Alec

[sharkmelley]

I completely agree that agree that a library of darks is a waste of time.  How does one know what temperature your lights were taken at?  Look at the following graph of how dark current increases with time when taking successive long exposures (5 minute exposures in this case) - you would need a different dark from your library for each light - I'm really not sure if that is even possible:

I'm not exactly sure how DSS creates its master darks from a set of darks at differing sensor temperatures.  To be on the safe side I would take approx. 4 hours of darks at a stable room temperature and use the final 2 hours worth.  Whilst taking the darks, keep the camera in the dark (e.g. a cardboard box) to prevent any kind of light leakage.

Mark

[alcol620]

Thanks Mark

Just a follow up. When taking darks, does the camera need to be attached to the scope or simply have its lens cover on and be covered up.

I read the article, got the flavour of what was said, but all a bit beyond my academic level.

I have been taking darks at at ISO800 and a number of exposure levels then setting up a file for similar temperatures as the camera temperature rises ie 18-20 degrees, 21-24 degrees etc etc. However I soon realised that I would have to have an enormous database to cover the range of temperatures that the camera could be at. And a large number of dark frames for each temperature. Hence my original question seeking a better way forward

Regards

Alec

[sharkmelley]

No need for the camera to be on the scope - just cover the lens and make sure there is no possibility of light leakage.

Mark

[alcol620]

Thanks Mark

[perfrej]

Hey guys, I am not trying to smoke-screen this or ruin your day, but read this article. I found it very interesting.

http://www.stark-labs.com/craig/resources/Articles-&-Reviews/CanonLinearity.pdf

And: http://pixinsight.com/forum/index.php?topic=4432.0

Scaling works perfectly well with CCDs, though.

/per

[alcol620]

Thanks perfreg

All very complicated and beyond me! Obviously not as simple as first thought.

Just have to accept what DSS does for the time being and hope for reasonable images. 

Will dithering the scope between images reduce the requirement for darks?

[D4N]

I think dithering is pretty much essential for DSLR imaging. I've not had much joy with darks on mine, the dark noise just seems to change too much. Flats and bias frames work well though.

Sent from my iPad using Tapatalk

[sharkmelley]

Hey guys, I am not trying to smoke-screen this or ruin your day, but read this article. I found it very interesting.

http://www.stark-labs.com/craig/resources/Articles-&-Reviews/CanonLinearity.pdf

And: http://pixinsight.com/forum/index.php?topic=4432.0

Hi Per - very interesting discussions in those links - thanks.  I was already aware of Craig's article.

I think the conclusion is that dark scaling is not absolutely perfect on DSLRs.  But the problems they refer to are only second-order effects:  dark scaling is still far and away the best method for dealing with the fact that sensor temperature (and hence dark noise) changes during over the course of an imaging session.

Will dithering the scope between images reduce the requirement for darks?

I think dithering is very important - I always use it.  Combined with dark scaling it is a very effective way of dealing with dark noise.  It then allows you to do much greater stretching on the faint data.  Once you reach that stage, you will find that the Canon banding issue becomes the next biggest issue to deal with.

Mark

[michaelmorris]

It is perfectly possible to build up a library of darks of different temperatures for a DSLR.

Canon DSLRs store the sensor temperature data in the frames EXIF data. Darks Master free software (http://sourceforge.net/projects/darkmaster/) can then be used to match dark frames to your light frames and export the matched file list to DSS.  

Additionally the excellent Backyard EOS and APT software packages can be configured to save the sensor temperature in the filename automatically.

I try to take sets of darks when it's really cold as these are the hardest to get.  l leave a nice long gap between darks subs to help to reduce the rate of heat accumulation in the camera.  I've also tried putting the camera in the fridge to take darks (make sure it's sealed in a bag with some desiccant).

You can cool the DSLR with a homemade cool box as well.

http://www.astronomymark.com/dslr_cooler.htm

http://www.swashastro.co.uk/peltier_cooler_box.html

http://www.scsastro.co.uk/catalogue/orion-dslr-camera-cooler-52095.htm?gclid=CKvygoSwvMUCFSnLtAodS3kA5A

I hope this helps

[alcol620]

HI Michael

Thanks for that, I might give that piece of software a go. The coolers look interesting.

Has anyone else had experience of DarkMaster? Anyone any ideas on what would constitute "darks at the same temperature". How many degrees would be an acceptable range for "the same temperature"? Obviously individual light frames are different temperatures.

Regards

Alec

[sharkmelley]

An EXIF temperature  driven dark frame library gives the appearance of being a good scientific way of performing dark matching.  However it relies on the key assumption that the temperature recorded in the Canon EXIF header is a good predictor of dark current and therefore thermal noise.  In my experience (and I have tested this) the relationship doesn't hold at all well.  You don't even need to take my word for it.  For instance, look at Table 3 in Roger Clark's review of the Canon 7D II (http://www.clarkvision.com/reviews/evaluation-canon-7dii/). There is a large range of dark current for identical EXIF temperature readings e.g. at 23C we see dark currents of 0.084 or 0.146 which is a huge discrepancy - nearly a factor of 2.  Such differences would make a complete nonsense of dark frame matching.

A more extreme example is here: http://pixinsight.com/forum/index.php?topic=7079.0  A cold finger had been used to cool the sensor to well below freezing but still the recorded EXIF temperature was 12C.  It certainly looks unlikely that the recorded EXIF temperature is actually the sensor temperature and since Canon have never stated exactly what the EXIF temperature represents (at least to my knowledge), some have speculated that it might actually be the temperature of the DIGIC processor instead, or even some other part of the camera.

Stick to dark frame scaling (which I believe is the best method) or taking a set of darks immediately after the end of every imaging session (with the camera still outside) - this is also very effective but doesn't allow for the fact that the sensor temperature increases over the imaging session.

Mark

[alcol620]

Mark

Interesting that the pixinsight example you highlight suggests that the EXIF temperature has no relationship to the camera sensor. If this is indeed proven beyond doubt then there is no data available from the output from the camera that allows any meaningful scaling of the dark frames for temperature. Apart from the scaling that goes on within software packages like DSS.

If the EXIF temperature is so far out from the actual temperature of the sensor, is there any point in recording the temperature at all??

[sharkmelley]

The temperature is recorded in the EXIF header automatically anyway - so no worries there.  The only question is how useful it is.  I suspect it wasn't put there for astronomers!

Mark