Now that I can autoguide, how to decide best exposure time?

[BrendanC]

I'm finally getting decent results now that I've got autoguiding working properly. I'm currently using 3m exposures but I've tested 4m and 5m and they're good too, which implies I can probably go even further.

However, this throws up yet another quandary! Which is: even if I can do these long exposures times, should I?

I'm aware that longer exposures will bring out the fainter details for example in nebulae, but could blow out the brighter stars. Also, they bring out the sky fog more, which I need to be wary of in a Bortle 4 zone.

I've looked but there don't seem to be any hard and fast rules (not that there tend to be anyway with astrophotography!). It would help if there was a formula or calculator of some kind? Or is it really another case of, try it and see what works?

Any advice?

Thanks, Brendan

[endless-sky]

Hi,

the "fast" rule is to expose until the histogram peak, as seen from the back of the camera, is 1/4 to 1/3 detached from the left. This ensures you are rendering the read noise contribution of your camera/chosen ISO to a minimum percentage in your final stack.

The "long" rule is to determine the read noise of your camera at your optimal ISO (photonstophotos.net is a great resource for this, or you can use BasicCCDParameters script, if you have PixInsight), and then expose such that the mean ADU is around 5*RN^2 to 10*RN^2 (these are the limits I usually try to respect).

Lower exposures imply that you do not swamp the read noise enough and that you won't be able to pull out all the details that your sky conditions allow for.

Higher exposures imply that you are blowing (clipping) too many stars and also reducing your total dynamic range (because you are shifiting the histogram to the right, giving it less room to stretch).

Edited December 21, 2020 by endless-sky

[BrendanC]

Thank you.

I'm shooting with a DSLR hooked up to my laptop, running APT (probs should have mentioned that).

Presumably the histogram I see on APT will be the equivalent of what I would see on the camera?

Also, for read noise, given that the https://www.photonstophotos.net/ site looks like a great resource but will take a while for me to digest, are there any tools other than Pixinsight that will give me an ADU reading for a photo? I've asked Ivo about APT and it doesn't do that, which is a tad disappointing.

[vlaiv]

Depends on your setup.

- simplest rule - as long as you can while still having tight stars.

- complex rule - determine read noise for your camera (in electrons) and see if LP noise or thermal noise is higher for your conditions (will depend on ambient temperature and light pollution levels) and then make exposure such that read noise is 1/5 of higher of the two - thermal noise or light pollution noise (very similar to above given advice - except you should not do it in ADUs but rather electrons and you have to see if thermal noise (dark noise) is higher than LP noise).

Or simply go with 4-5 minute exposures

 

[endless-sky]

I use KStars and EKOS (because I control my equipment with a Raspberry Pi). Their FITS Viewer give you the mean ADU of the picture, as soon as it's completed and downloaded. So I can compare the ADU of the actual shot to the theoretical ADU that I know my camera works best at.

For Windows, maybe N.I.N.A. offers you the option of seeing the ADU of a RAW shot, without stretching it/converting it. But I haven't ever used it, so I am not 100% sure.

This is the actual formula to calculate your ADU:

DN = (Nread^2 * Swamp / Gain + Offset) * (2^16/2^Bits)

Where:

DN = required background signal in 16-bit DN.
Nread = read noise in e-
Swamp = swamping factor
Gain = camera gain in e-/ADU
Offset = bias offset in ADU
Bits = ADC bit depth

Credit: Jon Rista.

You'll have to determine first what the optimal ISO is for your camera (I don't have any experience with it, so I am not sure). Then find the corresponding Gain (in e-/ADU) and Read Noise (in e-). The offset is the mean ADU value you get when you take a bias. Swamp should be 5 (for minimum) and 10 (for maximum).

Hope this helps!

[vlaiv]

13 minutes ago, endless-sky said:

Swamp should be 5 (for minimum) and 10 (for maximum).

I disagree on this one. If you use above formula and want to compare signal levels versus noise levels (their relationship is quadratic, for both poisson and gaussian distribution, noise is square root of signal) - I would suggest using 25 instead of recommended values as that is square root of 5.

Noise adds like linearly independent vectors - square root of sum of squares.

Let's examine all three cases - 5, 10 and 25 as "swamp" factor. This translate into sqrt(5), sqrt(10) and 5 higher noise level. Let's call this factor C.

Let's your light pollution noise be LP.

Then your light pollution and read noise combined will be:

sqrt( (LP)^2 + (LP / C)^2 ) = sqrt( LP^2 + LP^2/C^2) = sqrt(LP^2 * ( 1 + 1/C^2)) = LP * sqrt(1+ 1/C^2)

This means that resulting noise is the same as if we had higher LP noise for factor of sqrt(1+1/C^2). Let's calculate this factor for our example values of C.

Swamp factor of 5 gives sqrt(1.2) = ~1.095 or 9.5% increase in LP noise. This corresponds to 20% increase in LP brightness.

Swamp factor of 10 will give sqrt(1.1) = ~1.049 or 4.9% increase in LP noise. This corresponds to 10% increase in LP brightness.

Swamp factor of 25 will give sqrt(1.04) = ~ 1.0198 = ~1.02, or 2% increase in LP noise. This corresponds to 4% increase in LP brightness.

So you see - there is no maximum really - it is only how much you want to increase effective LP noise (or any dominant noise source - or total noise combined with read noise). With x5 difference in noise value (equivalent to 25 swamp factor above) you get 2% increase in noise which would be equivalent to 4% increase in LP (for LP dominated scenario).

For some reason I find it more acceptable than above recommended values.

[ollypenrice]

Stellar cores will usually be saturated in any exposure long enough to get the faint stuff. The solution lies in post-processing. You can still have attractive stars if you pull their colour into the core from the outside. How do you do that? Buy Noel's Actions for Photoshop! (Now called Pro-Digital Astronomy Tools, I think.) You can do it longhand by using Ps Layers but I just leave it to Noel!

My advice would simply be to experiment. My own conclusions disagree with most of the published advice...

Olly

Edited December 21, 2020 by ollypenrice

[endless-sky]

I was suggesting 10 as a swamping factor because I was under the impression that it meant that the read noise in the final stack only contributed 5%, when compared to the other noise sources. Does that correspond to your calculations, Vlaiv? Of course, if 25 would be 2.5x better, so the 5% becomes 2%, that's even better. But the more we push the exposure, the less dynamic range we are left with. So we have to compromise. To me, 5% left read noise in the total stack seemed like a good enough compromise.

[BrendanC]

Thanks all - lots to read up on and think about!

[vlaiv]

13 minutes ago, endless-sky said:

I was suggesting 10 as a swamping factor because I was under the impression that it meant that the read noise in the final stack only contributed 5%, when compared to the other noise sources. Does that correspond to your calculations, Vlaiv? Of course, if 25 would be 2.5x better, so the 5% becomes 2%, that's even better. But the more we push the exposure, the less dynamic range we are left with. So we have to compromise. To me, 5% left read noise in the total stack seemed like a good enough compromise.

Indeed it does:

40 minutes ago, vlaiv said:

Swamp factor of 10 will give sqrt(1.1) = ~1.049 or 4.9% increase in LP noise.

 

40 minutes ago, vlaiv said:

Swamp factor of 25 will give sqrt(1.04) = ~ 1.0198 = ~1.02, or 2% increase in LP noise.

Dynamic range per exposure is completely irrelevant. Olly already mentioned one way to deal with saturated stars - another is to take very small set of very short exposures and use that for saturated parts of main image. These short exposures will capture bright parts of the image with very good SNR (as those are bright and have plenty of signal).

We increase overall dynamic range of image by stacking, so single exposure dynamic range is really not important for final result. That is why 12bit cameras work every bit as good as 14bit or 16bit.

[endless-sky]

Thank you, Vlaiv!

[rnobleeddy]

3 hours ago, vlaiv said:

Depends on your setup.

- simplest rule - as long as you can while still having tight stars.

- complex rule - determine read noise for your camera (in electrons) and see if LP noise or thermal noise is higher for your conditions (will depend on ambient temperature and light pollution levels) and then make exposure such that read noise is 1/5 of higher of the two - thermal noise or light pollution noise (very similar to above given advice - except you should not do it in ADUs but rather electrons and you have to see if thermal noise (dark noise) is higher than LP noise).

Or simply go with 4-5 minute exposures

 

As an aside, and something I was going to ask you in a different thread...

My old mount was poor and would occasionally jump by many arcseconds in DEC and would take so long to recover that the subs were unusable. Keeping subs short was essential because I could just throw away the ones that had a large jump.

 

My new mount isn't amazing, but is pretty consistent over time - if the guiding is RMSE of 0.8" then it's pretty much bouncing around all the time, rather than guiding well sometimes and having the odd bad spell. Is there are reason why, for example, 1 x 20 min sub would suffer more than 4 x 5 minute subs? Naively the signal would just add, so once I'm exposing long enough to average out the random movement, it won't matter how longer I expose for?

[vlaiv]

17 minutes ago, rnobleeddy said:

My new mount isn't amazing, but is pretty consistent over time - if the guiding is RMSE of 0.8" then it's pretty much bouncing around all the time, rather than guiding well sometimes and having the odd bad spell. Is there are reason why, for example, 1 x 20 min sub would suffer more than 4 x 5 minute subs? Naively the signal would just add, so once I'm exposing long enough to average out the random movement, it won't matter how longer I expose for?

I'm not sure what you are asking here, but here is breakdown of what happens:

If there is read noise in camera - only noise source that adds per exposure and is not dependent on exposure length - stack of few longer exposures will have better SNR than stack of many shorter exposures both summing up to same total exposure time.

This is because all other noise sources depend on time and stacking them is no different then exposing for longer. Only one that does not behave like that is read noise. It has the same intensity regardless if you expose for 0.1s or 100s.

Just how much difference there is between few longer subs vs many shorter subs totaling same overall exposure time depends on few factors - biggest one being ratio of read noise to other noise sources combined or rather biggest of rest of them in simpler case. Other factors include stacking algorithms used - sometimes sigma clip produces better results than regular average and it works better when there are plenty of samples to establish statistics.

There are other things that play in short sub favor - one of them being how well your mount guides. If your mount guides well for 5 minutes - that does not mean that it can go all night long as there are factors that are on larger scales than that. What if you have differential flexure that becomes obvious on exposures longer than 5 minutes?

So, guiding/tracking can and will set upper limit. Second thing is wasted subs. What if something interrupts your imaging session - a bird landing on your scope, sudden gust of wind, neighbor turning on their light, earthquake (yes that does happen), ... You can either loose 20-30 minutes of imaging or loose 4-5 minutes. This is down to how often you expect for odd thing to happen.

Choice of exposure length is ultimately a balance, and yes you should go with as much exposure length as it makes sense with all above criteria.

[rnobleeddy]

45 minutes ago, vlaiv said:

I'm not sure what you are asking here, but here is breakdown of what happens:

If there is read noise in camera - only noise source that adds per exposure and is not dependent on exposure length - stack of few longer exposures will have better SNR than stack of many shorter exposures both summing up to same total exposure time.

This is because all other noise sources depend on time and stacking them is no different then exposing for longer. Only one that does not behave like that is read noise. It has the same intensity regardless if you expose for 0.1s or 100s.

Just how much difference there is between few longer subs vs many shorter subs totaling same overall exposure time depends on few factors - biggest one being ratio of read noise to other noise sources combined or rather biggest of rest of them in simpler case. Other factors include stacking algorithms used - sometimes sigma clip produces better results than regular average and it works better when there are plenty of samples to establish statistics.

There are other things that play in short sub favor - one of them being how well your mount guides. If your mount guides well for 5 minutes - that does not mean that it can go all night long as there are factors that are on larger scales than that. What if you have differential flexure that becomes obvious on exposures longer than 5 minutes?

So, guiding/tracking can and will set upper limit. Second thing is wasted subs. What if something interrupts your imaging session - a bird landing on your scope, sudden gust of wind, neighbor turning on their light, earthquake (yes that does happen), ... You can either loose 20-30 minutes of imaging or loose 4-5 minutes. This is down to how often you expect for odd thing to happen.

Choice of exposure length is ultimately a balance, and yes you should go with as much exposure length as it makes sense with all above criteria.

Thanks. I was referring to the error introduced by the mount, and that itself was in relation to the help you offered in another thread, where for my CCD, it would take 20 minute subs would keep the read noise at the level you recommend.

The point around differential flexure is one I'd forgotten. 

Either way, I managed to get 10 x 20 minute subs last night with pretty poor guiding and the subs are OK (as an aside, I believe this was a combination of moderate gusts of winds but I appear to have had auto-meridian flip turned off, and for some reason the tracking degraded after the mount passed the  meridian. Once I spotted it was about to crash and flipped it, guiding was a lot better). There's a slight elongation in the stars in one axis, but I'll have a play with dropping some of the worst subs from just before I flipped from the stack. 

So I guess that's a long way of saying I think I'll be fine when guide performance is better, so just need to balance the exposure duration with % of lost subs.

 

[BrendanC]

On 21/12/2020 at 13:25, vlaiv said:

- complex rule - determine read noise for your camera (in electrons) and see if LP noise or thermal noise is higher for your conditions (will depend on ambient temperature and light pollution levels) and then make exposure such that read noise is 1/5 of higher of the two - thermal noise or light pollution noise (very similar to above given advice - except you should not do it in ADUs but rather electrons and you have to see if thermal noise (dark noise) is higher than LP noise).

 

Right, so, this is one of those 'backburner' projects that I've decided to have a think about.

Firstly, I shoot everything at ISO800 because according to this site, it's the best for an EOS1000D (it actually says it's ISO200 but that's ridiculous, and says to check out ISO800 too, so that's what I use): http://dslr-astrophotography.com/iso-values-canon-cameras/

I've found my camera at photonstophotos: https://www.photonstophotos.net/Charts/ReadNoise_e.htm#Canon EOS 1000D

Looks like read noise at ISO800 is 5.65 electrons with a bit of interpolation between the given values.

So, I have one part of this. Now, how do I determine the light pollution noise, or thermal noise? Any pointers?

Edited December 31, 2020 by BrendanC

[vlaiv]

2 minutes ago, BrendanC said:

Right, so, this is one of those 'backburner' projects that I've decided to have a think about.

I've found my camera at photonstophotos:  https://www.photonstophotos.net/Charts/ReadNoise_e.htm#Canon EOS 1000D?

Looks like read noise at ISO800 is 5.65 electrons with a bit of interpolation between the given values.

So, I have one part of this. Now, how do I determine the light pollution noise, or thermal noise? Any pointers?

Find e/ADU value for ISO800

Take one bias and one dark sub.

Convert to electrons by multiplying with e/ADU value for given ISO. Subtract bias from dark and measure mean value.

Divide with dark exposure length - that will give you dark current at given temperature. Dark current depends on temperature and roughly doubles or halves each 6 degrees C° or down respectively.

That will give you means to calculate dark current for given exposure time in given ambient temperature. Dark current noise will be square root of that.

Light pollution will require that you take exposure of actual sky. It will vary depending on where you point your telescope - so point it to where you usually expose targets (maybe do few exposures in different directions).

Again - convert to electrons, calibrate with darks and select background sky in center of the frame (that avoids need to do flats) and get median value. If you can skip stars in selection - if not, median value will deal with them quite good (as it ignores outliers).

This will give you LP signal per exposure - divide with exposure length to get per second - and again square root will be associated noise (once you multiply with wanted exposure length).

[BrendanC]

It's never easy is it...?

Thank you again @vlaiv, I'll get back to you when I've scratched my head some more about this!