selecting the 'best' iso setting - sorry here we go again

[SteveKinder]

This is my first post so please be gentle with me.

I have done a lot of research first; honest I have!

I have only been active in astronomy for about 6 months. It is only even more recently that I have been trying to get into astrophotography. However, like the rest of the people on this forum I am not thick and I have my obsessive side... I have done a huge amount of background reading and understood most of what I have read. I have a lot more to learn but need to try and put what I think I know into practice first.

However,........ I am finding it impossible to reconcile all that I am being told regarding ISO settings. The message is hugely inconsistent. I have read respected books that say stick with 400 and 800 and others that say use 1600 and 3200. A significant minority promote 100 and 200 settings!?

There are a number of discussions around on this subject but I am unashamedly starting a new one as I want to ask questions and challenge the received wisdom.

I am keeping my question deliberately simple and maybe a little inflammatory but am desperate for a better understanding.

Why have expensive cameras with high ISO sensitivities available if you are not going to use them when you have a lack of light?

Thanks,

Steve.

My kit is

William Optics FTL132

William Optics FLT98

CGEM Mount

Cannon 500D (unmodified)

Lost of other bits and pieces to be detailed later.

[Photosbykev]

imo, and it is only my opinion, capturing astroimages is mainly about controlling all the variables and balancing them to optimise your final result. Things like iso, aperture, guiding accuracy, seeing conditions, camera noise control etc all need to be considered.

The bottom line is use as low an ISO as you can without compromising the final image because your polar alignment or guiding was poor.

I typically use 800-1600 on a 5D MkII and its noise control is excellent but I do make sure I take plenty of lights, darks and flats to help process the image. As I improve my techniques and the guiding becomes accurate then I'll start looking to reduce the iso and increase the exposure length. On lower end cameras I probably wouldn't want to go much beyond iso 400, purely because of the noise issues BUT others will certainly disagree.

[jgs001]

With experimentation, at different focal lengths and focal ratios, I know what I can get away with. I really need to use iso 800 with my 80ed, but with my nifty at f/3.2 or f/4 I can get away with iso 200 or 400. Although the theory is that you get the same number of photons so the iso doesn't matter (very paraphrased and simplistic view) I've found the too low an iso with the 80ed means I jaw to really push the data thy much harder to get the image out and that means it gets noisier (probably doing something wrong in processing )

[dph1nm]

For astronomers changing ISO really only has two effects:

(1) If you increase ISO then you saturate bright objects (stars mainly) in a shorter exposure time.

(2) If you decrease ISO the read noise goes up.

Whether either of these two things worry you depends on what you are doing I guess!

NigelM

[simon hicks]

You are quite right...there is a huge amount of folklore about this, manly because there are so many variables....what works for one person works because of their combination of variables.

Unfortunately there are very few serious studies of this. I have seen a couple in papers. What is clear to me is that the characteristics of the camera (noise, gain, readnoise, etc), the characteristics of the background light pollution, the characteristics of the object you are trying to capture (a bright cluster vs a narrowband nebula), and the amount of imaging time you have available.

I'll try to look up the links to the papers.

[reddoss]

I have a modified Canon 450D and use differnt F ratio scopes for imaging. I find that in light polluted areas iso 800 gives an awful lot of noise with fast (f4.7 and F5.3) reflector scopes but is not too bad in the F7.5 refractor. It's a bit of a trade off as you need increased sensitivity to battle against the glare from the sodium haze. However with a light pollution filter and darker skies, I find an iso of 800 just fine.

[simon hicks]

Here's a couple of really useful links. They are really about optimising the sub exposure duration....but that is very related to ISO setting...its all wrapped up together in a sense. I like the second paper best and I've included a quote from the author of the second paper that was on the PixInsight forum.

Happy reading!

http://www.hiddenloft.com/notes/SubExposures.pdf

Finding the Optimal Sub-frame Exposure - Article

This is Charles Anstey, the author of the paper in question. I have been trying to find a simpler way to explain what is going on and why sub-exposure length matters.

Let's consider each term separately because they follow superposition.

Target signal : Goes up linearly with time so it depends purely on total time Tgt(t). So 1 100-minute shot or stacking 100 1-minute shots are equal.

Light pollution signal : Goes up linearly with time so it depends purely on total time LP(t). So 1 100-minute shot or stacking 100 1-minute shots are equal. However we can simply subtract this out from the image as a bias.

Light pollution noise : Goes up with the SQRT(time) so it depends purely on total time Elp(sqrt(t)). So 1 100-minute shot or stacking 100 1-minute shots are equal.

At this point if we had a noiseless camera, there is no advantage to shooting longer sub-frames and the implications are pretty astounding. When advanced amateurs start using noiseless cameras, astro-imaging will take another huge leap forward in quality. At that point, DSO imagers will be using 10-30 frames per second just like planetary imagers, collecting as many frames as they want, and then stacking only the best of the best until all the desired detail has been attained. Look up "Lucky camera" to see the results of such a camera.

However, the current real-world "affordable" cameras all have read noise. The total read noise in an image depends purely on the number of frames stacked and goes up by the SQRT(#frames)

Read Noise : RN*SQRT(# sub-exposures). So 100 1-minute shots has a lot more read noise than 1 100-minute shot.

The goal of the formula is to balance the number of sub-exposures with respect to read noise and light pollution. Reducing the sub-exposure length and having more total frames results in more total noise from the increase in read noise with no gain in target signal, fixed by total time. Increasing sub-exposure length and stacking fewer frames gains basically nothing because the faintest signal discernible above the total noise has already been limited by the total exposure time and dominated by LP noise so lowering total read noise will not help.

From a practical point of view when you don't know how much total time you wish to spend on an object, shooting longer sub-exposures allows you to keep adding more frames if you decide to keep at it with the only penalty being that if the frame is lost because of bad guiding or other error, you lose a larger piece of time. Use the formula to determine the practical upper limit of the total time for a given sub-exposure and know that if you are thinking of possibly putting in even more total time, you need to use a longer sub-exposure to be safe.

The problem with John Smith's formula is that it is only valid if you are stacking exactly two frames. Eventually if you stack enough frames, read noise will become significant enough that for the same total time you would get better results for the faintest signals shooting fewer frames for longer time. Also the implication is that shooting in LP or during the full moon somehow limits how deep you can image compared to dark skies. Completely incorrect and an imager can create an identical result under the full moon compared to darkest skies if they image with enough total time. The increase in total time may be a factor of 10+ but what else are you going to do for that night, nothing? Seeing and transparency determine the absolute limits of image quality and depth. You can't stack a bunch of 3.5" FWHM images and wind up with a 2.0" FWHM image no matter how much total time. Don't skip great nights of seeing just because the moon is out. The only caveat is the camera has to have enough well depth to cover the LP signal and have plenty of bits left over for the stretch to an 8-bit display. Cameras with only 8 or 12 bits are likely to be limited by excessive or even moderate LP.

[themos]

NigelM talks sense on this, I think.

Add the ingredient that you move from read-noise-limited conditions to photon noise-limited conditions as you increase exposure times. You "want" to be photon-noise-limited.

have a play with this simulator (it's for microscopy and doesn't allow you a change in ISO setting but you can vary the read noise, which, as NigelM said, goes up with decreasing ISO).

Nikon MicroscopyU | Interactive Java Tutorials | CCD Signal-To-Noise Ratio

To summarise perhaps: you don't want to be read-noise-limited so you can't have ISO too low (but perhaps for your conditions of sky, optics and electronics even the lowest ISO keeps you photon-noise-limited).

And you don't want to lose dynamic range so you don't want ISO too high (but perhaps for your blah blah even the highest ISO does not saturate the interesting parts of your image).

That's my current understanding and it will probably evolve.

[SteveKinder]

Thank you all very much; particularly to Simon and Themos for their extra effort and the references. I will read them you have my word. The intellectual muscle, and breadth and depth of knowledge, of the contributors to this forum is humbling.

Steve.

[GJBC]

As time progresses and you get more accomplished you will get a better feeling for the conditions around you on the evening of your image aquisitions and make you setting choices to fit those conditions. ie location of object if lower to the horizon you might need lower ISO to loose noise but take longer exposures. if the object is high and in good skies it might be longer exposures again but you will be able to use a higher ISO. there is no set rule for iso or exposure length (not IMO) i had the same questions early on but like you did not get far in finding out, everyone has their own opinion. So i started on 800 ISO and stuck with it for a while as i did not want to get stuck on technicalities. i also learnt the other requirements i.e. Darks flats Bias, all these helped over time as i got to grips with them. but stuck with ISO 800. once i was getting fairly decent results following an image aquision routine that was the same every time, i started to tinker. i also learnt to discipline my self to one object at a time or one per night or part of a session ie before 12midnight or after midnight i might get two set of images in. i see so many taring around the sky taking an image here an image there and saying its not that good, stick with oine object per session and nail it. I would take a light frames at 60 seconds each at each ISO setting and then dependent on results aquired would choose the best looking image it only took five minutes and i had the right setting for that object. No scientific mumbo jumbo as i had learnt the rest of the process it all followed that after.

my acquisition train was as below after set up etc and using pre gathered flat frames.

1 light frames test to choose ISO setting may be five images 60 seconds

2 try exposure lengths with same ISO ie 90 seconds, 120 seconds or more then choose best again.

3 take series of light frames number of your own choice but the more the merrier.

4 Take a series of dark frames minimum of 9 same iso focus etc.

5 take bias frames same amount as light frames.

if I had time and the skies remain clear i might move to another object and start over again checking focus is still the same no shift in image train and i will do the ISO test again and take it from there.

it is a steep learning curve sometimes and you can drown yourself in the amount of knowledge. get out there and try i did and i got there over time. sadly don't do it so much these days health wont allow it. my first and last images of Andromeda are attached so you can see it can be done taken from my back yard in south wales ps the skies here are not to bad. but if you look at the last image you will see a single RAW sub ISO 200 80 seconds there are ten of these plus i used darks and bias no flats were used i used DSS and CS2 1st oner taken oin 2005 last one in 2007

oh one other thing, don't delete your acquisition images you might be surprised what is there as you get better with processing them i know i have re visited what i thought was poor data in 2005 only to find i get an image out in 2008. it also allows you to keep your hand in when it raining.

i know it was a ramble and it only touched on you ISO question but i hope my 2p worth helps you take the right path, others will have their own method and paths.

good luck

[GJBC]

sorry forgot the last image with the raw data one sub of ten light 90 seconds ISO 200 note the red sky beginning to show it was worse with a HIGHER iso so i choose my setting for each image set

used TMB 105 and Canon 20Da all on an EQ6

Grahame

[dph1nm]

Another point worth making is that I see various comments on various astro forums about high iso images looking too noisy, so people use lower iso. But this is not what matters - it is the signal-to-noise that concerns us, not just the noise. So a high iso image will have higher noise in terms of the numbers coming out of the camera, but the signal-to noise will just the same (or slightly better if you are read noise limited) as any other iso.

NigelM

[simon hicks]

Just to back up NigelM's good point, its also the dynamic range that you've got in the part of the image you are interested in. So if I am narrowband imaging a faint nebula then its ISO1600 because I can't get enough dynamic range otherwise even with 20 minute exposures.

[themos]

Now this is the bit I don't understand. Dynamic range? But you get MORE dynamic range if you go down in ISO from the highest setting. The reason? At highest ISO you get a lot of ADUs per electron. ADU is the value that will be recorded on the digital file. But there is a fixed (whatever ISO you use) maximum value that CAN be recorded on the file, usually a power-of-two-minus-one. So assume that you want to capture a scene where some parts produced 1 photon and some parts produced 1000 photons. If you scale that up too much and end up with 100 ADU and 100,000 ADU, you will find that the 100,000 value cannot be recorded and you end up with something like 65,535 (the largest integer that will fit in 16 bits). You've lost dynamic range whereas if you used an ISO that produced 20 ADU and 20,000 ADU, you'd have preserved it.

If this is wrong, tell me!

[simon hicks]

I was referring to the dynamic range only in the part of the image that you are interested in. So if you are narrowband imaging a faint nebula and you are trying to capture the faint dust trails in the background (at the expense of losing dynamic range in the stars), then if you can only capture enough light to give you say 10 counts (bit levels or whatever) then this part of the image will be somewhat posterised.

If however you image at a greater ISO then that 10 counts can be turned into 100. The stars will be blown out...but I've captured the nebula. I wouldn't use ISO1600 on a star cluster.

You are right that the dynamic range of the whole image (including stars etc) will not be greater, and may actually be lower due to a higher background. So it depends on what you are trying to capture in that particular session.

[themos]

Again, I don't understand:

Turning a scene that goes 9,10,11,9,13 to one that goes 90,100,110,90,130 achieves nothing that can't be done in software post-processing.

The only reason I can see to stop me from turning down the ISO setting is that my read-noise increases so that I am no longer photon-noise limited (or I get into quantisation problems where the gain is less than 1 ADU per electron and my photoelectron will not be written to the file and be lost).

[simon hicks]

The first will be 9,10,11,9,13. It is limited to integer values. The second one could be 92,97,113,94,132. The data in the second one is better, it has more resolution.

Post processing can smooth between the first set of points, but that's different to acquiring real data between those points.

[themos]

I don't understand that. The ISO setting is essentially a gain knob you use when reading how many photoelectrons you have collected. It doesn't miraculously give you more original photoelectrons. The extra apparent "detail" in the 92,97,113,94,132 is purely down to noise in the read-out/gain electronics. It is made in the camera, not in the sky.

[simon hicks]

Good point Themos. I think this is why some people advocate going to the point of 'unity gain', i.e. where gain = 1 ADU/e-. (I'm not sure I support this view.)

This seems to assume one photon results in one electron. This is where my expertise (or major lack of it) ends. But I would be really surprised if a Canon camera is really doing single photon counting? It might have one photoelectron per ADU, but that doesn't mean each and every single photon creates a photoelectron with 100% efficiency. I would have really thought there was a certain number of photons (on average) that are needed to generate 1 photoelectron (on average).

There is a secondary aspect to this as well. If you raise the ISO then you reduce the exposure time and get more subs to stack. The stacking process introduces real increases into the measured bit depth, i.e. two 1s or 0s averaged can give three levels (0, 0.5, and 1), and so on if you get my drift. Therefore taking the ISO from 800 to 1600 means I can take two images vs one in a given time. The 800 case will have levels of 9, 10, 11. The 1600 images could have levels of 90,100,110 and 90,90,110 giving a stacked image of 90,95,110. So my 1600 ISO stack has a real measured bit depth greater than the 800 stack.

[geordieskies]

I use Compact and Bridge cameras and don't have access to a DSLR. I've seen recommendations to use high ISO settings of 1600 and 3200 for DSLRs but my cameras produce so much noise at these settings that I must stick to a maximum of 400 ISO. I'm about to embark on some planetary imaging using a Canon S90, I will post some pics on the forum as soon as possible. :-)

[simon hicks]

Hi Geordieskies,

If you want to do a comparison, then you need to compare a stack of say 10 x 2 minute subs at ISO1600 with a stack of 5 x 4 minute subs at ISO800, because they will both have the same total exposure time over all the subs and the same exposure level within each sub.

A good experiment though. Good luck.

Cheers

Simon

[geordieskies]

well, there's another problem Simon, I'm stuck with a maximum exposure time of 15 seconds. Oh dear, I feel a DSLR purchase looming :-)

[simon hicks]

You won't look back!

[themos]

Simon, the electron "liberated" by the photon is all we have. We can read in the specs about how many photons, on average, it takes to yield one e-. I think that's what's quoted as Quantum Efficiency, in percentage terms (so, maybe 30% chance that an incident photon will result in an e-).

As for your last paragraph, I am not convinced. The reason is that this argument, as it stands, would suggest that it would be even better to shoot at some crazy ISO like 256,000 for very short durations and increase our dynamic range without limit. I think we should use more realistic numbers: 9,10,11 for the ISO 800 run, ok. If you instead run 2 exposures at ISO 1600 (with the same photons arriving) you will catch the equivalent of 4+5=9, 4+6=10, 6+5=11 (a likely split for the arrival of the photons) which will be doubled to 8,8,12 and 10,12,10 (because we've doubled the gain by doubling the ISO and before read-noise messes them up, but let's ignore that for now). Averaging these two still gets you the same information that you had originally = 9,10,11. The reason you don't really see the nice gap of 2 (as in 10,12,10) in the actual images in the higher ISO setting is the read noise which smears the quantisation.

[dph1nm]

I use Compact and Bridge cameras and don't have access to a DSLR. I've seen recommendations to use high ISO settings of 1600 and 3200 for DSLRs but my cameras produce so much noise at these settings that I must stick to a maximum of 400 ISO.

This depends what you mean by noise. If you mean they have such a high dark signal that a significant fraction of your pixels are near saturation you are a bit stuck. However, the fact that compact cameras have high shot noise for daytime shots (and hence look horrible at higher ISOs) isn't really relevant to astro shots, especially if you are shooting afocally.

Nigel