Dynamic Range - The Maths

[Scooot]

Not sure if this is the correct place to post this.

I’ve been trying to understand the Dynamic Range of my Canon 450D

This website says it’s about 62db http://www.astrosurf.com/comolli/strum45.htm

Sensorgen says it’s about 10.4 stops (I realise it varies with ISO) https://web.archive.org/web/20180830114507/http://sensorgen.info/CanonEOS-450D.html

wiki gives a good table comparing stops to dB, https://en.m.wikipedia.org/wiki/Dynamic_range

I have managed to finally understand how the wiki table arrives at  the results, my maths skills are fairly basic. 

EG 2^10 stops = 1024

1024 log10 x 10 = 30.1db

So on the same basis 10.4 stops should be

2^10.4 = 1351.17

1351.17 log10 x10 = 31.31 

So about half the 62db on the first link, so I’m guessing I should be multiplying by 20 instead of 10 for the photographic db calculation but I don’t understand why?

Can anyone explain where I’m going wrong following this?

[Martin Meredith]

Hi

The dB definition depends on whether you are dealing with power-like quantities or root-power-like quantities: 

dB = 10 log10 (power quantity)

db  = 20 log10 (root power quantity)

Since power = root power squared, the two are equivalent since log10 x^2 = 2 log 10 x

As I understand it (willing to be corrected!) a power quantity would be something like voltage * current whereas voltage alone, or something proportional to it (your situation, I imagine) is a root power, hence the use of 20 rather than 10 as the multiplier.

Martin

[wxsatuser]

The formula for DRdb  is 20xlog(FWC/RN)

To get DR in stops divide by 6

[wxsatuser]

At ISO 100 your 450D has a FWC of 26614 and a read noise of 19.4
Plug these into the formula and you get 62.7db

Divide by 6  gives 10.45 stops.

[Scooot]

Thanks very much Martin and Mike. 

Mike I’ve worked that through Ok  

I had seen the FWC/RN formula, that would’ve been my next puzzle which you anticipated. It was the multiplication by 20 which I couldn’t understand as it seemed to be at odds with the wiki table. I still don’t follow why the root power uses a multiplier of 20 instead of 10 but I suppose I don’t need to now  

[Martin Meredith]

Hi Scooot

As you say, it doesn't matter now ?, but in case it it helps anyone, just consider the definition of decibel to be

   20 * log10 (amplitude-like-quantity)

Since power is amplitude squared, amplitude is square root(power). Substitute that in the above

   20 * log10 (square root power) 

ie 20 * log10 power^0.5

ie 0.5 * 20 * log10 power

ie 10 * log10 power

The two things are completely mathematically equivalent, but you need to know if you are computing dB from a power value or an amplitude value in order to apply the right formula.The confusion arises because it is as common to see the definition as 10 log10 power as 20 log10 amplitude. (Just one additional thing to note is that since in some domains (e.g. sound pressure) amplitude can be negative, we usually take the root mean square of amplitude in that case).

Martin

[Scooot]

1 hour ago, Martin Meredith said:

Hi Scooot

As you say, it doesn't matter now ?, but in case it it helps anyone, just consider the definition of decibel to be

   20 * log10 (amplitude-like-quantity)

Since power is amplitude squared, amplitude is square root(power). Substitute that in the above

   20 * log10 (square root power) 

ie 20 * log10 power^0.5

ie 0.5 * 20 * log10 power

ie 10 * log10 power

The two things are completely mathematically equivalent, but you need to know if you are computing dB from a power value or an amplitude value in order to apply the right formula.The confusion arises because it is as common to see the definition as 10 log10 power as 20 log10 amplitude. (Just one additional thing to note is that since in some domains (e.g. sound pressure) amplitude can be negative, we usually take the root mean square of amplitude in that case).

Martin

Ah got it  

So FWC/RN = 26614/19.4 = 1371.85

1371.85 log 10 x20 = 62.746db

1371.85 is the amplitude. Square it for Power = 1,881,972.42

1,881,972.42 log 10 = 6.2746 x 10 = 62.746db

Many thanks.  

[Scooot]

Trying to convert to stops I think this is right  

Stops are expressed in log 2. 

2 log10 = 0.301

So to get back to stops

62.746db/20 = 3.1373

3.1373/0.301 = 10.4229 stops

or as Mike says divide by 6 for an approximation 

[vlaiv]

Very informative thread.

I would like to know what is the point of Dynamic range as a measure? What does it tell us?

[Scooot]

5 minutes ago, vlaiv said:

Very informative thread.

I would like to know what is the point of Dynamic range as a measure? What does it tell us?

I find it easier to understand in stops. The range of light from the lightest pixels to the darkest that the camera can detect. So as my dynamic range is about 10, if I expose on a grey card, anything brighter or darker than 5 stops either side won’t be captured, either over exposed or invisible. That’s my understanding.  

 

[vlaiv]

2 minutes ago, Scooot said:

I find it easier to understand in stops. The range of light from the lightest pixels to the darkest that the camera can detect. So as my dynamic range is about 10, if I expose on a grey card, anything brighter or darker than 5 stops either side won’t be captured, either over exposed or invisible. That’s my understanding.  

 

Ah, I see - it's useful in terms calculating "stops offset" from a given middle value (gray), if we accept that floor is equal to read noise floor and top is saturation point.

[Scooot]

Basically a camera with a greater dynamic range is better in that regard. Stacking increases the dynamic range but it’s obviously better if you start from a higher figure in the first place.

I was curious at first because I’d always thought the dynamic range was higher with a lower ISO & with my camera it isn’t between ISO100 and ISO 400. (Or not by much) One thing led to another as I was reading about it and I ended up getting curious about the maths. 

[vlaiv]

That was my problem in the first place - notion that higher dynamic range is somehow better or that dynamic range is even comparable between cameras.

Let's take for example camera

a) Having read noise at 2e and FWC of 2048 (nice round numbers) - it has dynamic 10 stop dynamic range (I would say 10bit, but let's go with stops, bits imply ADC or ADU values)

b) Having read noise at 8e and FWC of 8192, again has 10 stops of dynamic range.

Cameras are otherwise equal (same QE, same pixel size, ...)

Camera A has significant disadvantage compared to camera B - as highest SNR signal it can record in single exposure (light dominated regime) is much less - 2048 / sqrt(2048 + 2^2) = ~45.21, while for camera B it would be 8192 / sqrt(8192 + 64) = ~90.16.

To me ratio of FWC and RN is simply meaningless and up until your previous answer I could not figure out where it is useful - but you did provide useful answer - in terms of regular photography and range of stops one can go up/down with regard to some baseline.

[Scooot]

31 minutes ago, vlaiv said:

That was my problem in the first place - notion that higher dynamic range is somehow better or that dynamic range is even comparable between cameras.

Let's take for example camera

a) Having read noise at 2e and FWC of 2048 (nice round numbers) - it has dynamic 10 stop dynamic range (I would say 10bit, but let's go with stops, bits imply ADC or ADU values)

b) Having read noise at 8e and FWC of 8192, again has 10 stops of dynamic range.

Cameras are otherwise equal (same QE, same pixel size, ...)

Camera A has significant disadvantage compared to camera B - as highest SNR signal it can record in single exposure (light dominated regime) is much less - 2048 / sqrt(2048 + 2^2) = ~45.21, while for camera B it would be 8192 / sqrt(8192 + 64) = ~90.16.

To me ratio of FWC and RN is simply meaningless and up until your previous answer I could not figure out where it is useful - but you did provide useful answer - in terms of regular photography and range of stops one can go up/down with regard to some baseline.

I think you’ve helped me with something else. I was also wondering if there was any advantage to using ISO 100 instead of the sensorless ISO of 400, given there’s very little difference in dynamic range. 

Using your formula:

Maximum SNR at ISO 100 is 162 

26614 / sqrt (26614 + 19.4^2)

Maximum SNR at ISO 400 is 90.88

8305 / sqrt (8305 + 6.8^2) 

So on the face of it, when light levels allow ISO 100 is worth using for the potential higher SNR even though the dynamic range is also the same. 

Have I got this right?  

 

[vlaiv]

21 minutes ago, Scooot said:

I think you’ve helped me with something else. I was also wondering if there was any advantage to using ISO 100 instead of the sensorless ISO of 400, given there’s very little difference in dynamic range. 

Using your formula:

Maximum SNR at ISO 100 is 162 

26614 / sqrt (26614 + 19.4^2)

Maximum SNR at ISO 400 is 90.88

8305 / sqrt (8305 + 6.8^2) 

So on the face of it, when light levels allow ISO 100 is worth using for the potential higher SNR even though the dynamic range is also the same. 

Have I got this right?  

 

Have no idea what sensorless ISO 400 means.

You have to be careful when doing max SNR calculations - as above formulas are only valid for electron count, or e/ADU of 1.

If you collect 100 electrons, intrinsic (Poisson) noise will be 10e and signal to noise ratio will be 100/10 = 10. If you convert that to ADU with e/ADU factor of 0.5, you will get 200 ADU but noise expressed in ADU will be 20, so SNR stays the same (200/20 = 10). ISO is essentially e/ADU factor (each ISO has corresponding e/ADU or gain value) - and these don't change signal associated noise nor snr ratio - even when accounting for read noise.

Apart from being multiplicative constant, ISO also serves another purpose - it "scales" ADC window. Usually cameras don't have enough bits in ADC unit to fit whole FWC. For example if FWC is 26614 and ADC is 14bit - it can only fit 16384 distinct values / levels. In this case, one ideally wants gain range from 1 to 26614 / 16384. ISO is often defined in 100,200,400,800, .... because of "backward compatibility" - people are used to these sensitivity markings from days of analog film. Because of this gain range is "extended" over ideal range, and high ISO settings will actually give you less levels of value than ADC bit count is capable of recording.

This "scaling" of window is what you should take into account when doing SNR calculations. Here is example:

ASI1600 that I have, has FWC of 20000 and 12bit ADC. At minimum gain, max recorded ADU will be 4095 but that will correspond to 20000e and at this gain setting read noise is about 3.6e so SNR of max signal would be:

20000 / sqrt(20000+3.6^2) = ~141.37

At unity gain (meaning e/ADU = 1, one electron - one ADU) this max SNR would be:

4095 / sqrt(4095 + 1.7^2) = ~63.97

It looks like minimum gain has advantage over unity gain - single exposure can achieve better SNR, and this would be true if we had unlimited ADC bit count. But our ADC is limited to 12 bits and in order to record those 20000e with 12 bits (values 0-4095) we need to have certain e/ADU - it is around 5 (20000/4095 = ~4.884, it is closer to 5 because FWC is a bit more than 20000e - but ZWO manual quotes it at 20ke so we can't know for certain, and from e/ADU gain graph it looks like value is about 5 so it matches). But what does this tell us? It tells us that there is going to be some serious round off error. If pixel captures 1e it is going to end up as - 0 ADU (adu units are round), if pixel captures 2,3,4e - ADU will still be 0. This is bad thing as we are taking random value of captured electron count and rounding them all to same value, we introduce certain error that is not so random in nature. Stacking works when we have good random distribution - or random value that follows certain distribution - gaussian for read noise, or poisson for signal. If we change that distribution in non random way, stacking will give us wrong results.

Let's say that you have light source that gives of 0.1e in exposure duration. What will your subs be like in terms of recorded values? 0,0,0, ...,0,0,1,...,0,0,1,0,0....,0,0...,1,0,0,2,0,0,0 .... On average just a bit over every tenth frame will capture 1 electron, very rarely it will happen that captured value will be 2e, very very rarely it will be 3e and very very very rarely it will be 4e. All these values will end up as 0 and stack of 0 will be 0 - you will not detect signal at all!

For this reason, regardless of dynamic range or max SNR that you can capture in single exposure - I advocate using unity gain with ASI1600. In astro imaging we rely on multiple exposures anyway and number of those exposures will shape dynamic range of resulting stack. With gain values above unity (e/ADU less than 1, or high ISO values) - there is still round off error and it's still not random in nature, but here rounding error tends to be small and usually gets masked by read noise (total error looks closer to regular random distribution) so it's not a big deal, but you end up getting even smaller max signal or dynamic range so why use it?

 

[Scooot]

27 minutes ago, vlaiv said:

Have no idea what sensorless ISO 400 means.

You have to be careful when doing max SNR calculations - as above formulas are only valid for electron count, or e/ADU of 1.

If you collect 100 electrons, intrinsic (Poisson) noise will be 10e and signal to noise ratio will be 100/10 = 10. If you convert that to ADU with e/ADU factor of 0.5, you will get 200 ADU but noise expressed in ADU will be 20, so SNR stays the same (200/20 = 10). ISO is essentially e/ADU factor (each ISO has corresponding e/ADU or gain value) - and these don't change signal associated noise nor snr ratio - even when accounting for read noise.

Apart from being multiplicative constant, ISO also serves another purpose - it "scales" ADC window. Usually cameras don't have enough bits in ADC unit to fit whole FWC. For example if FWC is 26614 and ADC is 14bit - it can only fit 16384 distinct values / levels. In this case, one ideally wants gain range from 1 to 26614 / 16384. ISO is often defined in 100,200,400,800, .... because of "backward compatibility" - people are used to these sensitivity markings from days of analog film. Because of this gain range is "extended" over ideal range, and high ISO settings will actually give you less levels of value than ADC bit count is capable of recording.

This "scaling" of window is what you should take into account when doing SNR calculations. Here is example:

ASI1600 that I have, has FWC of 20000 and 12bit ADC. At minimum gain, max recorded ADU will be 4095 but that will correspond to 20000e and at this gain setting read noise is about 3.6e so SNR of max signal would be:

20000 / sqrt(20000+3.6^2) = ~141.37

At unity gain (meaning e/ADU = 1, one electron - one ADU) this max SNR would be:

4095 / sqrt(4095 + 1.7^2) = ~63.97

It looks like minimum gain has advantage over unity gain - single exposure can achieve better SNR, and this would be true if we had unlimited ADC bit count. But our ADC is limited to 12 bits and in order to record those 20000e with 12 bits (values 0-4095) we need to have certain e/ADU - it is around 5 (20000/4095 = ~4.884, it is closer to 5 because FWC is a bit more than 20000e - but ZWO manual quotes it at 20ke so we can't know for certain, and from e/ADU gain graph it looks like value is about 5 so it matches). But what does this tell us? It tells us that there is going to be some serious round off error. If pixel captures 1e it is going to end up as - 0 ADU (adu units are round), if pixel captures 2,3,4e - ADU will still be 0. This is bad thing as we are taking random value of captured electron count and rounding them all to same value, we introduce certain error that is not so random in nature. Stacking works when we have good random distribution - or random value that follows certain distribution - gaussian for read noise, or poisson for signal. If we change that distribution in non random way, stacking will give us wrong results.

Let's say that you have light source that gives of 0.1e in exposure duration. What will your subs be like in terms of recorded values? 0,0,0, ...,0,0,1,...,0,0,1,0,0....,0,0...,1,0,0,2,0,0,0 .... On average just a bit over every tenth frame will capture 1 electron, very rarely it will happen that captured value will be 2e, very very rarely it will be 3e and very very very rarely it will be 4e. All these values will end up as 0 and stack of 0 will be 0 - you will not detect signal at all!

For this reason, regardless of dynamic range or max SNR that you can capture in single exposure - I advocate using unity gain with ASI1600. In astro imaging we rely on multiple exposures anyway and number of those exposures will shape dynamic range of resulting stack. With gain values above unity (e/ADU less than 1, or high ISO values) - there is still round off error and it's still not random in nature, but here rounding error tends to be small and usually gets masked by read noise (total error looks closer to regular random distribution) so it's not a big deal, but you end up getting even smaller max signal or dynamic range so why use it?

 

Thanks for the detailed explanation, nice of you to take so much trouble.

I probably used the wrong phrase when I said “sensorless ISO” I meant ISO-less as explained n this site. http://dslr-astrophotography.com/iso-values-canon-cameras/ on my canon ISO 400 or thereabouts seems to be the right spot.

They also have a differing view regarding unity gain you might be interested in. http://dslr-astrophotography.com/worry-unity-gain/

I take your point about bit size and the maximum SNR, so remain undecided for the time being of using ISO 100 sometimes or not.  

[vlaiv]

28 minutes ago, Scooot said:

They also have a differing view regarding unity gain you might be interested in. http://dslr-astrophotography.com/worry-unity-gain/

I've read that article, but I don't think it addresses point I made above, at least in case of ASI1600.

There is natural dithering in form of read noise. But here is quote from wiki on that:

Quote

In order to make the quantization error independent of the input signal, noise with an amplitude of 2 least significant bits is added to the signal. This slightly reduces signal to noise ratio, but, ideally, completely eliminates the distortion. It is known as dither.

At minimum gain for that camera, like we said e/ADU has value of about 5. Two LSB value in ADU will be 3 (both bits turned to 1, or highest in 0-3 range) which translates back to ~15e of noise needed to dither 0-4e values. ASI1600 has read noise of about 3.6e at this gain setting - about 4 times less than needed.

[Scooot]

Thanks again Vlaiv, I’ll have another read later to see if I can follow and understand it.  

[vlaiv]

2 hours ago, Scooot said:

Thanks again Vlaiv, I’ll have another read later to see if I can follow and understand it.  

I probably need to stand corrected here. I just run some tests and it appears that 3.6e is enough to dither signal at 5 e/aDU gain. Will need to revisit theory myself to understand where I got it wrong. Test was fairly basic one - I created artificial image with some text in it. Text intensity was set to 0.4e, and I made 256 images with this text and I applied Poisson noise to it. I also added 3.6e Gaussian noise to each sub. Divided each sub with 5 and rounded value. After stacking these images, I was still able to discern text from background (it was not readable but shape of some letters was there and one could clearly see it resembling original text - much as one would expect from low SNR). There is definitive SNR impact from rounding alone - I have not measured it yet, but dithering with only 3.6e works.

When I think about it, that is fairly unsurprising - engineers designing Panasonic sensor probably know what they are doing

 

[Martin Meredith]

My understanding is that you don't lose anything via rounding so long as you have a noisy offset in the signal. Here's a quick simulation. The signal is the circle with a level of 0.1 in each sub, buried in a noise signal, and you are looking at a stack of 1000 such images. The circle is not visible in any individual sub but emerges as expected from the stack. In the top figure no rounding has been applied while in the lower figure each sub has been rounded to integers prior to adding to the stack. Clearly, there is no adverse effect of rounding so long as there is an offset. The reason is that a signal of level 0.1 means the photon will be caught on 1 in every 10 subs on average. By the same token, it will be rounded up from offset to offset + 1 on 1 in every 10 subs. So in effect that very faint signal is not lost. If the offset is 0 it is a different story. Then the photon is effectively lost.

 

[vlaiv]

2 minutes ago, Martin Meredith said:

My understanding is that you don't lose anything via rounding so long as you have a noisy offset in the signal. Here's a quick simulation. The signal is the circle with a level of 0.1 in each sub, buried in a noise signal, and you are looking at a stack of 1000 such images. The circle is not visible in any individual sub but emerges as expected from the stack. In the top figure no rounding has been applied while in the lower figure each sub has been rounded to integers prior to adding to the stack. Clearly, there is no adverse effect of rounding so long as there is an offset. The reason is that a signal of level 0.1 means the photon will be caught on 1 in every 10 subs on average. By the same token, it will be rounded up from offset to offset + 1 on 1 in every 10 subs. So in effect that very faint signal is not lost. If the offset is 0 it is a different story. Then the photon is effectively lost.

 

What gain did you apply in simulated case? I find it very odd that SNR with rounding and SNR without rounding is the same.

[vlaiv]

Here are some additional results in regards to dithering and non unity gain. I made following test case:

0.1 Signal with Poisson noise was generated and additive Gaussian noise of 3, 2 and 1 sigma was added. e/ADU of 5 was applied to each and rounding was performed. 256 of such images were stacked and mean value and standard deviation measured.

Expected signal in each case should be 0.02 (0.1 / 5), and expected noise in each case should be sqrt(0.1 + 9) / 5 * 16 = ~0.0377, sqrt(0.1 + 4) / 5 * 16 = ~0.0253, sqrt(0.1 + 1) / 16 = ~0.0131

Measured values for signal and noise:

Value: 0.019811630, Stddev: 0.041914366
Value: 0.018644333, Stddev: 0.029275529
Value: 0.007413864, Stddev: 0.008475984

It is clear that resulting error is somewhat larger than calculated error - due to quantization error introduced with rounding - for first two cases. As a reference, I did stack of first case, with gaussian noise of sigma 3 without rounding to confirm above calculations and results obtained are value:  0.01982021 with stddev: 0.037817369 - good match to theoretical. It is also clear that 1 sigma dithering for e/ADU 5 is not enough to produce good results - obtained mean value is 2.5 less than it should be.

 

[Martin Meredith]

So this is simply a signal value of 0.1 on the circle circumference, 0 elsewhere, added to uniform noise in the interval [0, 3], rounded prior to adding to stack in one case and not in the other i.e. sum (signal + random)  vs sum(round(signal + random)). No gain concept applied. Just illustrating that we don't lose 'fractional' photons long as there is a noise 'carrier'.

 

[vlaiv]

2 minutes ago, Martin Meredith said:

So this is simply a signal value of 0.1 on the circle circumference, 0 elsewhere, added to uniform noise in the interval [0, 3], rounded prior to adding to stack in one case and not in the other i.e. sum (signal + random)  vs sum(round(signal + random)). No gain concept applied. Just illustrating that we don't lose 'fractional' photons long as there is a noise 'carrier'.

 

Ah ok, I was misled by SNR inscription on top of the images supplied.

[Martin Meredith]

See also discussion 3 paras up from base of this article http://keithwiley.com/astroPhotography/imageStacking.shtml