Swamp read noise

[Pitch Black Skies]

I think I've read that swamping the read noise X5 is a safe exposure length guideline.

How do I know I am swamping it X5?

Is it a case of measuring the ADU from a bias frame and then taking an exposure until it is 5x that ADU?

My camera is 533MC Pro. I use unity gain 100 under Bortle 4.

[vlaiv]

Select a piece of background without much "stuff" in it (few stars is ok, but try not to have nebulosity or galaxies or such) in one of your subs and measure median ADU value (median is used because of any odd star that ends up in selection - it will "ignore" it).

Multiply that value with e/ADU for your gain - in case you are using unity gain - then it is easy - value is 1e/ADU and you don't have to multiply anything - just use median ADU.

Take square root of that number - that is your LP noise.

You can get read noise value from published data - for ASI533MC Pro it is:

about 1.5e for unity gain.

You can go the other way around - starting with read noise. You have 1.5e of read noise. This means that you need 7.5e (x5) of LP noise or square that to get signal - 56.25e for signal.

Compare this number to actual number you measure from background with median and that will give you idea by how much you need to extend or if you can shorten your subs.

Just be careful that you need to properly calibrate your sub. Use one of the channels for measurement - green is probably the best as it carries the most of luminance information (humans are most sensitive to noise in luminance data).

[900SL]

Expose around 1200 ADU above the bias mean seems to be the rule of thumb for the 533 mc

My bias ADU is around 2500 on the ASIAirpro

 

[Pitch Black Skies]

My results:

 

Mean ADUs

Bias 2797

30s 2822

60s 2847

120s 2896

180s 2948

300s 3047

 

The closest I can get to 56.25e was with the 300s sub.

√3047 = 55.19e

I didn't feel confident about exposing longer than 300s but I need to do that to reach 56.25e. Although is 300s+ not a bit excessive for an optimal exposure time with this camera?

 

1200 ADU above the bias mean ADU doesn't seem practical from the test results. For 300s I was only 250 ADU above it..

Edited April 19, 2022 by Pitch Black Skies

[Fegato]

Vlaiv is spot on.

I have used a slightly more relaxed method of this to work out a reasonable range of ADU values for my subs to use "live" with NINA (which will report median ADU on each frame) based on 3x and 10x swamp values. This uses the maths as Vlaiv describes above and came from research and various bits of reading. e.g. I found Jon Rista's contribution to this thread on CN useful https://www.cloudynights.com/topic/597682-i-need-a-primer-on-read-noise-calc-asi1600/#entry8200280

He suggests x3 as an absolute minimum and x10 as "optimum".  Where you want to go in the range (or longer) will depend on your target and your equipment. How bright is the target? Are my stars blowing out? Can I risk guiding for this long? Have I got enough processing power and disk space to process all these subs? 

To get the values to work with "live", you just need use the formula in the link above. This adds the bias (offset) back in to the result you would get using a calibrated frame. And in practice, the median of the whole sub will usually be very close to that of a section of background . The only issue I had was that my "offset" number was already a 16 bit ADU value, so as I have a 14 bit ADC (same with 533), rather than

DN = (Nread^2 * Swamp / Gain + Offset) * (2^16/2^Bits)

I used

DN = (Nread^2 * Swamp / Gain) * (2^16/2^Bits) + Offset

(otherwise the offset is multiplied by 4 twice)

So with my ASI2400MC Pro at Gain 140 (1.2 e) and Offset 15 (480 ADU), and a x10 swamp factor, I get  (1.5^2 * 10/1.2) * 4 + 480 = 75 + 480 = 555. I also used Sharpcap's sensor analysis to check my camera numbers, and they were very close to the ZWO specs.

WIth these modern CMOS sensors, you may find that the optimum is a lot less than you might expect.

Robin

 

[Fegato]

8 hours ago, Pitch Black Skies said:

My results:

 

Mean ADUs

Bias 2797

30s 2822

60s 2847

120s 2896

180s 2948

300s 3047

 

The closest I can get to 56.25e was with the 300s sub.

√3047 = 55.19e

I didn't feel confident about exposing longer than 300s but I need to do that to reach 56.25e. Although is 300s+ not a bit excessive for an optimal exposure time with this camera?

 

1200 ADU above the bias mean ADU doesn't seem practical from the test results. For 300s I was only 250 ADU above it..

I think you may have that wrong.

300s sub = 3047.   Take off the bias / offset, 3047 - 2797 = 250 ADU

So your median signal is 250. You need to take your Gain into account in order to work out where this is on the swamp scale. See my post above.

Also worth saying that 300s may be too long or too short for the camera. As well as the camera, the exposure needed is massively affected by the scope being used (focal ratio in particular) and the filters being used. 

 

[Fegato]

So if you're using Gain 100, a figure of 90 ADU plus offset would be x10 swamp factor. That's near your 120s exposure?

[vlaiv]

One thing that needs to be taken into account is bit depth of camera.

Actual ADU values might be multiplied with 4 or 16 (14bit or 12bit camera) - this will skew results.

@Pitch Black Skies

Can you post single raw / uncalibrated sub from one of your sessions and single bias raw sub - again straight from camera?

[Fegato]

1 hour ago, vlaiv said:

One thing that needs to be taken into account is bit depth of camera.

Actual ADU values might be multiplied with 4 or 16 (14bit or 12bit camera) - this will skew results.

@Pitch Black Skies

Can you post single raw / uncalibrated sub from one of your sessions and single bias raw sub - again straight from camera?

Yes the 533 is 14 bit, so the "raw" value will be multiplied by 4.  But also, as I said above, if you take the bias ADU reported from the software it will already be converted to 16 bit - this is why I changed the formula slightly for the calculation.

[Pitch Black Skies]

@vlaiv

Bias_1.0ms_Bin1_533MC_gain100_20220420-140701_-10.8C_0002.fit

Light_M101_300.0s_Bin1_533MC_gain100_20220402-231650_-10.0C_0002.fit

[Pitch Black Skies]

On 19/04/2022 at 09:21, vlaiv said:

Just be careful that you need to properly calibrate your sub. Use one of the channels for measurement - green is probably the best as it carries the most of luminance information (humans are most sensitive to noise in luminance data).

I don't understand. Do I need to do some calibration as part of the experiment?

[Pitch Black Skies]

I just want to add a screen shot of a bias and a 300sec sub of LP from my ASIair Pro too, to make sure I am reading it correctly.

[Fegato]

I prefer to use Median rather than Average (mean), not sure if ASIAir can show that?  Although not much difference with this Light, as it's fairly dark. With some very bright targets the mean may get a bit high for representing the background.

Anyway, I get 2796 median for the bias. The Light frame you posted is a different one to yours above? It has a median of 3216 (mean = 3212).

Your bias at 2796 does seem a bit high. I wonder if you could reduce the offset a bit, just to give a bit more room for signal, although maybe no issue given the dynamic range of the camera.

[Pitch Black Skies]

19 minutes ago, Fegato said:

The Light frame you posted is a different one to yours above?

Yes, the screenshot is from the LP test last night, just a blank patch of sky. I didn't save the subs, I just screenshot them for the ADU readings.

The fit file is M101 from a session a few weeks back.

I haven't ever adjusted the offset as I don't understand it.

Edited April 20, 2022 by Pitch Black Skies

[Fegato]

10 minutes ago, Pitch Black Skies said:

Yes, the screenshot is from the LP test last night, just a blank patch of sky. I didn't save the subs, I just screenshot them for the ADU readings.

The fit file is M101 from a session a few weeks back.

I haven't ever adjusted the offset as I don't understand it.

Yes I didn't understand it for a while. This is quite a good explanation:

https://daleghent.com/2020/08/understanding-camera-offset

You're basically just adding a number to the ADU count to avoid measuring zero in any pixel. There's no right value as such, but I think a value of a few hundred up to a thousand is usually suggested as about right. This will equate to a number that ZWO give, and I don't know what the relationship between the two is I'm afraid! For my ASI2400MC Pro I have offset value of 15 and get a bias ADU median of 480.

As in the linked article, if you take a bias frame with Offset 0, record the median value, and you can then increase the offset and see what you get.  But this isn't going to make a huge difference anyway - it will just give you a little bit more well depth to play with.

 

[Pitch Black Skies]

On 20/04/2022 at 16:30, Fegato said:

Yes I didn't understand it for a while. This is quite a good explanation:

https://daleghent.com/2020/08/understanding-camera-offset

You're basically just adding a number to the ADU count to avoid measuring zero in any pixel. There's no right value as such, but I think a value of a few hundred up to a thousand is usually suggested as about right. This will equate to a number that ZWO give, and I don't know what the relationship between the two is I'm afraid! For my ASI2400MC Pro I have offset value of 15 and get a bias ADU median of 480.

As in the linked article, if you take a bias frame with Offset 0, record the median value, and you can then increase the offset and see what you get.  But this isn't going to make a huge difference anyway - it will just give you a little bit more well depth to play with.

 

I think I've read that the default offset for this camera is 70 as per ascom driver?

I can't wrap my head around Jon Rista's formula. What am I doing wrong here?

Btw does DN mean data numbers?

Edited April 25, 2022 by Pitch Black Skies

[vlaiv]

3 hours ago, Pitch Black Skies said:

I don't understand. Do I need to do some calibration as part of the experiment?

Ideally, you want to measure background LP on calibrated sub - as it contains only light signal and all other signal removed.

However, we can measure uncalibrated sub and then remove offset measured from bias and ignore dark current as it is probably very small and won't affect results too much.

This is the reason I asked you for just bias and light sub.

Your bias has mean value of ~699.23ADU

Although camera is set to offset of 70 - it does not really mean anything - it is just number / percent / something in drivers. Actual offset is measured from bias sub.

You measured mean bias value to be ~2800ADU - but that value is 14bit value padded with two zeros to make it 16bit - and is hence multiplied with 4 (don't worry about this bit - just remember, 14bit cameras multiplied with 4 and 12 bit cameras multiplied with 16. 16bit cameras are as is).

2800 / 4 = 700, so actual value is above 699.23ADU - that is our offset.

Next I took your light. Split into channels, took green channel and made selection somewhere where there is not much vignetting (again - we are working with uncalibrated data - so aim to get the least vignetted region) and not much target:

Here I measured median value of 829 (again - measured value /4).

Our LP signal is thus 829 - 699 = 130

From read noise we saw that you need 1.5 * 5 = 7.5, 7.5^2 = 56.25 background signal value and yours is 130

130 / 56.25 = ~2.3111 = ~2.3

You are currently exposing about 2.3 times longer than you need to.

Since you are exposing 300s - you can expose for 130s or even 2 minutes and you will be fine.

Should you reduce exposure time to 2 minutes? Well that depends on you. If you feel comfortable working with 5 minute exposures - no need to reduce it to two minutes, but if you are loosing subs due to guiding issues or wind or whatever reason and you feel more comfortable with shorter subs - you can use shorter subs down to about 2 minutes.

[900SL]

You can't adjust the offset using an ASIAIRPRO.

There's a spreadsheet for subecxposure for the 533mc here, download the latest one. You enter your focal length, aperture and sky brightness etc, also gain. You can ignore the center columns

 

[900SL]

Ok won't let me cut and paste the link. Search Google for 533mc exposure calculator, it's on Cloudy nightz

[vlaiv]

1 minute ago, 900SL said:

Ok won't let me cut and paste the link. Search Google for 533mc exposure calculator, it's on Cloudy nightz

Why? We just calculated it above on real data.

 

[900SL]

Can't take you out in the field, Valve

[Pitch Black Skies]

44 minutes ago, vlaiv said:

From read noise we saw that you need 1.5 * 5 = 7.5, 7.5^2 = 56.25 background signal value and yours is 130

Awesome, thanks so much.

Does this mean that if I went with a recommendation of 10x swamp factor, I would be underexposing by almost 58%?

1.5 * 10 = 15, 15^2 = 225

130/225 = ~.58

[vlaiv]

Just now, Pitch Black Skies said:

Awesome, thanks so much.

Does this mean that if I went with a recommendation of 10x swamp factor, I would be underexposing by almost 58%?

1.5 * 10 = 15, 15^2 = 225

130/225 = ~.58

Yes, but why would you want to go with such large factor?

If you've seen such large factor being used - its probably not the factor for read noise, but rather background level.

Signal and associated noise are related by following equation (for Poisson type noise) - Signal = noise^2 or noise = sqrt(signal)

(by square root / square function)

If you apply factor of 10 to signal - that is the same as applying factor of sqrt(10) to noise - which is ~x3.16

Here is quick calculation in how much "total" difference is made by read noise for different "swamp" factors.

Noise swamped by x3 - there total noise increases by ~5.4%

Noise swamped by x5 - total noise increases by ~1.98%

Noise swamped by x10 - total noise increases by ~0.5%

We can't visually notice increase in noise of less than 5-10%, for this reason it is probably enough to swamp read noise by x3, but I prefer to swamp it by x5.

Swamping it by x10 is really an overkill as you certainly can't perceive increase of less than half of 1% in noise.

What you are probably referring to is "swamping background signal" kind of thing where x10 means signal not noise - and is equivalent to swamping noise by sqrt(10) = ~x3.16 - a bit more than x3 which is on edge of what we are able to detect.

Which ever number you choose in x3-x5 - you'll be fine. Going over x5 really makes no sense as improvement is minimal and can't be perceived by human.

[Pitch Black Skies]

2 hours ago, Pitch Black Skies said:

I think I've read that the default offset for this camera is 70 as per ascom driver?

I can't wrap my head around Jon Riatas formula. What am I doing wrong here?

Btw does DN mean data numbers?

I was writing the gain as 100 rather than 1. The gain is supposed to be expressed as e-/ADU which is 1 for me at Unity Gain. Vlaiv actually explained this to me earlier.

I'm still not getting the end part of the formula (2^16/2^bits). Jon just multiplies by 16 in his equations.

That leads me to another question, should I be multiplying by 14 as my camera is 14bit, or even 13.5 as that's what unity gain brings it down to?

Here's my latest attempt:

[Pitch Black Skies]

On 20/04/2022 at 19:30, vlaiv said:

Yes, but why would you want to go with such large factor?

If you've seen such large factor being used - its probably not the factor for read noise, but rather background level.

Signal and associated noise are related by following equation (for Poisson type noise) - Signal = noise^2 or noise = sqrt(signal)

(by square root / square function)

If you apply factor of 10 to signal - that is the same as applying factor of sqrt(10) to noise - which is ~x3.16

Here is quick calculation in how much "total" difference is made by read noise for different "swamp" factors.

Noise swamped by x3 - there total noise increases by ~5.4%

Noise swamped by x5 - total noise increases by ~1.98%

Noise swamped by x10 - total noise increases by ~0.5%

We can't visually notice increase in noise of less than 5-10%, for this reason it is probably enough to swamp read noise by x3, but I prefer to swamp it by x5.

Swamping it by x10 is really an overkill as you certainly can't perceive increase of less than half of 1% in noise.

What you are probably referring to is "swamping background signal" kind of thing where x10 means signal not noise - and is equivalent to swamping noise by sqrt(10) = ~x3.16 - a bit more than x3 which is on edge of what we are able to detect.

Which ever number you choose in x3-x5 - you'll be fine. Going over x5 really makes no sense as improvement is minimal and can't be perceived by human.

I think I am confusing Jon Rista's reccomendation for swamping read noise squared * 10.

Edited April 25, 2022 by Pitch Black Skies