More questions on ASI1600MM Pro - Gain

[Adam1234]

Hi, I've got some more questions on using the ZWO ASI1600mm Pro...this time on gain.

Is gain similar to ISO on a DSLR?

I've seen most people have been recommended to start out using unity gain, so I started off with that the other night, and took 28 x 300s subs with the L filter on M63.

Tonight I've been taking subs with the R filter (180s) on M63, however, I increased the gain to 250 when focusing and didn't change it back to unity gain when I started imaging. Will the mismatch in gain in the L vs R be a problem when it comes to combining the L with the RGB?

Should I continue with gain 250 for the G and B subs and keep the L subs that I shot at unity, redo the L at 250, or redo the R at unity? 

Any other general advice regarding best gain to use would also be appreciated.

Thanks!

 

Edited May 27, 2020 by Adam1234

[Miguel1983]

Not sure, i'm no expert by any means, but i would think the R data is going to be much brighter than the rest shot at unity.

Personally i shoot LRGB at gain 76 and narrowband at 139, because that seems to be the consensus.

The higher the gain, the lower the full well capacity, so high gain low exposure time, like ISO on a camera.

This post might be useful as a guide.

 

[Merlin66]

I stick to unity gain (139) for all my solar imaging.

I think of gain as a signal multiplier....the signal hasn't changed just they way it's presented.

Just my 2c

 

[Starwiz]

I've got the ASI1600mm-Pro myself and have only recently been getting my head around all of this.

A few things I've learned:

Gain doesn't change the sensitivity of the camera.

Each pixel can be thought of as a bucket (or a well - hence the term 'well-depth') that collects photons in a similar way that rain drops would collect in a real bucket.

At the end of the exposure, the fuller the bucket, the higher the ADU value for the pixel (higher ADU = lighter, lower ADU = darker).

Gain determines how quickly the bucket gets filled.  A higher gain means the bucket will fill more quickly, but there will be less levels possible within the bucket, so a higher gain means less dynamic range and lower gain means more dynamic range, although it's not a linear curve.

Offset determines how full the bucket already is before the exposure is started.  An empty bucket to start off with is undesirable in case there are no photons collected.

I'm currently experimenting with the HDR (Highest Dynamic Range) settings for the camera (Gain = 0, Offset = 10), but haven't yet processed anything.

John

 

[Adam1234]

10 minutes ago, Starwiz said:

I've got the ASI1600mm-Pro myself and have only recently been getting my head around all of this.

A few things I've learned:

Gain doesn't change the sensitivity of the camera.

Each pixel can be thought of as a bucket (or a well - hence the term 'well-depth') that collects photons in a similar way that rain drops would collect in a real bucket.

At the end of the exposure, the fuller the bucket, the higher the ADU value for the pixel (higher ADU = lighter, lower ADU = darker).

Gain determines how quickly the bucket gets filled.  A higher gain means the bucket will fill more quickly, but there will be less levels possible within the bucket, so a higher gain means less dynamic range and lower gain means more dynamic range, although it's not a linear curve.

Offset determines how full the bucket already is before the exposure is started.  An empty bucket to start off with is undesirable in case there are no photons collected.

I'm currently experimenting with the HDR (Highest Dynamic Range) settings for the camera (Gain = 0, Offset = 10), but haven't yet processed anything.

John

 

Thanks John, it will be interesting to see what you get with the HDR settings compared with say unity gain

[Starwiz]

4 hours ago, Adam1234 said:

Thanks John, it will be interesting to see what you get with the HDR settings compared with say unity gain

First indications are that it looks a lot smoother when I zoom in, but I have a significant gradient having not applied dark frames, so hope to get these done in the next day or so.

John

[daemon]

Brilliant analogy from John there! 

I have had the same camera for about a year and have only ever used Unity Gain (139) apart from one time when APT decided to flick it up to 200 without my knowledge!

For most targets I have shot so far I have found this to be fine, and to be honest my line of thinking has been that there were far too many other issues/parameters to worry about first before dwelling too much on gain. Just my thoughts though.

Cheers,

Steve

[Adam1234]

On 27/05/2020 at 05:15, Miguel1983 said:

Not sure, i'm no expert by any means, but i would think the R data is going to be much brighter than the rest shot at unity.

Personally i shoot LRGB at gain 76 and narrowband at 139, because that seems to be the consensus.

The higher the gain, the lower the full well capacity, so high gain low exposure time, like ISO on a camera.

This post might be useful as a guide.

 

What offset do you use at these gains?

[Miguel1983]

I don’t use offset

[Merlin66]

Using the ASI 1600 for solar imaging, don’t use offset either.

 

[wimvb]

One more parameter to calculate in: number of subs. At high gain, you lose dynamic range. In order to increase this, you need more subs. Can your computer handle this? With my ASI174MM-Cool, I use low gain (20), low offset (10) and still take more than 100 x 2 mins L-exposures. I also keep gain and temperature constant for all filters (except Ha, which I rarely use). This makes maintaining a dark library much easier.

[vlaiv]

21 minutes ago, Merlin66 said:

Using the ASI 1600 for solar imaging, don’t use offset either.

 

You should if you do full calibration. Darks for planetary / lunar / solar are rather short and so are flat darks.

Short darks are almost the same as bias (not enough dark current to raise signal level) - and there is a good chance there would be clipping to the left if you don't adjust offset.

[Merlin66]

Vlaiv,

In FireCapture I haven't had any issues/ concerns about darks etc. Flats for solar are a completely different ball game.

Do you honestly think setting an offset would benefit solar imaging??? Typically 0.009 sec exposures .....

[vlaiv]

15 minutes ago, Merlin66 said:

Vlaiv,

In FireCapture I haven't had any issues/ concerns about darks etc. Flats for solar are a completely different ball game.

Do you honestly think setting an offset would benefit solar imaging??? Typically 0.009 sec exposures .....

Yes I do.

It is about dark calibration (and implicitly flat calibration since it depends on dark calibration).

Here is an example:

Here I generated what is very close (in signal distribution) dark frame. I used sigma 1 and mean value of 2.

Distribution looks ok - it is nice bell shape, but some values are below 0. Camera can't record such values - it uses unsigned numbers as result (photon count is non negative value).

Look what happens when I limit value to 0 or above:

This raises mean value of dark - as if someone added DC component to the image (and also does some nasty things to distribution and hence stacking of master dark won't work as it should if distribution was unaltered).

You won't have that DC component in your lights as signal acts as offset and there won't be histogram clipping. Dark (and bias) component in ligths won't be clipped and there won't be additional DC offset signal.

Dark calibration fails for this reason and after dark subtraction your lights don't contain only light signal but also negative DC component that we just saw.

Now when you apply flat calibration - you are correcting both light, that is affected by attenuation and DC component that is not affected by attenuation - as it did not come in form of light thru the objective and pixel lens on camera. This makes your flat calibration fail as well.

In general - you want to avoid all of the above - and that is the reason you can adjust offset in the first place.

Short subs imaging is more susceptible to this than long exposure - because in long exposure, dark current can be enough to overcome this and still put histogram on right side of zero and prevent clipping.

[jimjam11]

On 27/05/2020 at 02:40, Adam1234 said:

Hi, I've got some more questions on using the ZWO ASI1600mm Pro...this time on gain.

Is gain similar to ISO on a DSLR?

I've seen most people have been recommended to start out using unity gain, so I started off with that the other night, and took 28 x 300s subs with the L filter on M63.

Tonight I've been taking subs with the R filter (180s) on M63, however, I increased the gain to 250 when focusing and didn't change it back to unity gain when I started imaging. Will the mismatch in gain in the L vs R be a problem when it comes to combining the L with the RGB?

Should I continue with gain 250 for the G and B subs and keep the L subs that I shot at unity, redo the L at 250, or redo the R at unity? 

Any other general advice regarding best gain to use would also be appreciated.

Thanks!

 

I never use unity gain; 200 for narrowband, 0 for lum and 76 for rgb. 300s at unity gain is likely to be dramatically overexposed. I switch gain so I can use 60s subs for lrgb without massively over exposing. The tables linked (on cloudynights) are a brilliant resource.

Make sure your calibration frames match exactly.
 

If you have the newer Asi1600 it defaults to offset 50 for all gains. It can essentially be ignored unless you go fiddling!

[Starwiz]

16 hours ago, daemon said:

Brilliant analogy from John there!

Thanks for saying so, Steve.

John

[daemon]

37 minutes ago, jimjam11 said:

I never use unity gain; 200 for narrowband, 0 for lum and 76 for rgb. 300s at unity gain is likely to be dramatically overexposed. I switch gain so I can use 60s subs for lrgb without massively over exposing. The tables linked (on cloudynights) are a brilliant resource.

Make sure your calibration frames match exactly.
 

If you have the newer Asi1600 it defaults to offset 50 for all gains. It can essentially be ignored unless you go fiddling!

Attached is a cooled (-20c) unstretched 300s luminance sub at 139 gain (ASI1600MM bought about a year ago).

It is of M101 and you can hardly see the galaxy, so I don't see it as massively over exposed. Or am I missing some point here?

I am still experimenting with this camera so will bow to superior knowledge, but none of my tests have yet involved adjusting the gain for any of the filters. I will experiment further on that though. 

L_Lum_2020-05-26_23-48-16_Bin1x1_300s__-20C.fit

[RayD]

Steve Chambers from Atik does a pretty good and simple to understand presentation on CMOS gain here.

[jimjam11]

25 minutes ago, daemon said:

Attached is a cooled (-20c) unstretched 300s luminance sub at 139 gain (ASI1600MM bought about a year ago).

It is of M101 and you can hardly see the galaxy, so I don't see it as massively over exposed. Or am I missing some point here?

I am still experimenting with this camera so will bow to superior knowledge, but none of my tests have yet involved adjusting the gain for any of the filters. I will experiment further on that though. 

L_Lum_2020-05-26_23-48-16_Bin1x1_300s__-20C.fit 31.27 MB · 2 downloads

Assuming this is at offset 50 which is typically about 800DN your median background is 6384. This means your background skyfog is approx 5600DN above bias. Optimal at gain 139 (taken from the CN thread) is approx 450DN above bias. You are therefore overexposed to the tune of 5000DN, a lot. If you look at the unstretched sub you can see tons of stars, these are over-exposed and you lose colour in them.

I ran the binarize PI process on your sub and converted all pixels > 0.95 (> 62000 DN) to white. These are all overexposed and will clip to white. You could easily use much shorter subs and as long as your total integration time is the same you will not get a noticeable drop in SNR. You will gain in terms of star clipping, lose less subs to clouds etc and probably see improved resolution. The downside is you end up with more subs which take longer to process!

As per the CN thread an optimally exposed sub will look terrible but this doesnt matter once stacked. This is one of my unstretched subs, 60s L , gain 0 @ F4.5. Raw frame is top left, binarized is bottom right:

 

Edited May 28, 2020 by jimjam11

[daemon]

40 minutes ago, jimjam11 said:

Assuming this is at offset 50 which is typically about 800DN your median background is 6384. This means your background skyfog is approx 5600DN above bias. Optimal at gain 139 (taken from the CN thread) is approx 450DN above bias. You are therefore overexposed to the tune of 5000DN, a lot. If you look at the unstretched sub you can see tons of stars, these are over-exposed and you lose colour in them.

I ran the binarize PI process on your sub and converted all pixels > 0.95 (> 62000 DN) to white. These are all overexposed and will clip to white. You could easily use much shorter subs and as long as your total integration time is the same you will not get a noticeable drop in SNR. You will gain in terms of star clipping, lose less subs to clouds etc and probably see improved resolution. The downside is you end up with more subs which take longer to process!

As per the CN thread an optimally exposed sub will look terrible but this doesnt matter once stacked. This is one of my unstretched subs, 60s L , gain 0 @ F4.5. Raw frame is top left, binarized is bottom right:

 

Thanks for that feedback jimjam. It is very interesting and I had not considered such parameters.

Anyway, the main reason for the long subs that night was to test my auto-guiding and I would not normally shoot such long subs for LRGB.

I will certainly experiment with the gain settings you suggest now that I understand a little more.

Apologies if I have hijacked Adam's thread - but hopefully this information is useful for him also!

[vlaiv]

1 hour ago, jimjam11 said:

Optimal at gain 139 (taken from the CN thread) is approx 450DN above bias.

Do you have any idea how they arrived to that number?

It is better to understand why particular number is chosen than to just use number.

1 hour ago, jimjam11 said:

You are therefore overexposed to the tune of 5000DN, a lot.

Not really - that is not over exposure at all.

You are right - you have shown areas of the image that are over exposed by using a bit of pixel math - mostly stars.

You are also right that it will lead to color distortion in the stars that are saturated.

There is very simple procedure to remedy that - one should take just a few short exposures at the end of the session to use as "filler" subs - for those over exposed stars.

Since we are dealing with limited full well capacity in any case - at any gain - there will always be some stars that saturate our sensor for given exposure - it is therefore better to just adopt imaging style that overcomes that for any case - few short exposures.

Is 5000DN background a bad thing? No, it is not - if subs are ok at that sub duration - overall SNR will improve for same imaging time over short subs. It will be very small improvement over recommended value (I would say 1156DN as a "better" number), but there would still be some improvement. Only when read noise is 0 there is no difference.

 

[jimjam11]

41 minutes ago, vlaiv said:

Do you have any idea how they arrived to that number?

It is better to understand why particular number is chosen than to just use number.

Not really - that is not over exposure at all.

You are right - you have shown areas of the image that are over exposed by using a bit of pixel math - mostly stars.

You are also right that it will lead to color distortion in the stars that are saturated.

There is very simple procedure to remedy that - one should take just a few short exposures at the end of the session to use as "filler" subs - for those over exposed stars.

Since we are dealing with limited full well capacity in any case - at any gain - there will always be some stars that saturate our sensor for given exposure - it is therefore better to just adopt imaging style that overcomes that for any case - few short exposures.

Is 5000DN background a bad thing? No, it is not - if subs are ok at that sub duration - overall SNR will improve for same imaging time over short subs. It will be very small improvement over recommended value (I would say 1156DN as a "better" number), but there would still be some improvement. Only when read noise is 0 there is no difference.

 

I wouldn't want to misquote but i believe the model was based on a number of different methodologies for arriving at the optimum dn above bias. The post from 12/06/17 17:53 states:

However, the "expected ADU above bias" results in the last columns are similar to those from the well established John Smith/Stan Moore formula (~10*RN*RN/gain)*16 and Jon Rista's approximation (20*RN/gain)*16. The exception seems to be that (as shown in the attached table) Jon's method tends to underestimate at low gain and overestimate at high gain, compared to the other methods.

 

I guess its purpose is to find the minimum sub length where read noise is adequately swamped whilst preserving as much dynamic range as possible. This leads to much shorter subs than people expect. Beyond this swamping point snr gains by going longer per sub is minimal?

Another good reference is the Dr Robin glover presentation on youtube:

 

 

Edited May 28, 2020 by jimjam11

[jimjam11]

43 minutes ago, vlaiv said:

 

Is 5000DN background a bad thing? No, it is not - if subs are ok at that sub duration - overall SNR will improve for same imaging time over short subs. It will be very small improvement over recommended value (I would say 1156DN as a "better" number), but there would still be some improvement. Only when read noise is 0 there is no difference.

 

1156? How did you reach that value because I reckon that more closely matches where I end up with LRGB imaging! With LRGB I dont like going below 60s because the acquisition overhead starts to become troublesome (interframe delay). As a result I usually end up in the range of 1000DN above bias so very close to you. Do you adopt that DN above bias for all gains?  

 

 

[vlaiv]

5 minutes ago, jimjam11 said:

I guess its purpose is to find the minimum sub length where read noise is adequately swamped whilst preserving as much dynamic range as possible. This leads to much shorter subs than people expect. Beyond this swamping point snr gains by going longer per sub is minimal?

It is fairly simple explanation really and one worth understanding.

Noise adds like linearly independent vectors - which really means square root of sum of squares (like finding length of a vector using projection of it in X and Y coordinates - here X and Y are linearly independent - you can't specify X with Y and vice verse).

It is also the same as finding hypotenuse in triangle. Here image will help:

Longer one side is with respect to the other - smaller the difference between hypotenuse and longer side. It is up to you to define what larger enough means - and based on that you can define how much background signal you want.

In this particular case we are putting read noise against LP noise - as two dominant noise sources, or rather - we are letting LP noise become dominant noise source with respect to everything else and observing how much larger it is with respect to read noise.

ASI1600 has read noise of about 1.7e at unity gain. My rule is - make LP noise x5 larger than read noise. It is very "strict" rule - you can choose x3 rule which is more relaxed - but let's look how each behave:

With x5 rule - we have 1.7e read noise and 8.5e LP noise. LP signal is going to be square root of LP noise, so LP signal is now 72.85e. Since we are at unity gain, but ASI1600 uses 12 bits - we need to multiply that with 16 to get DN - that gives us 1156DN

But let's see in percentage how much read noise + LP noise is larger than LP noise alone: sqrt(1.7^2 + (5*1.7)^2) = 1.7 * sqrt( 1+ 5^2) = 1.7 * sqrt(26) = ~ 1.7 * 5.09902

Difference being 5.09902 / 5 = 1.0198039027185569660056448218046, or about 1.98 = ~2%

Only two percent of difference between sky noise and sky noise + read noise - hence read noise makes minimal impact

Let's do the math for x3 rule:

again we have sqrt(1+9) = sqrt(10) = 3.1622776601683793319988935444327 / 3 = 1.0540925533894597773329645148109 = ~5.41% - now we have 5.41% difference between pure LP noise and LP + read noise. Btw x3 rule will give us: (1.7*3)^2 * 16 = ~416DN

That is the point behind any particular number - how much difference you want for read noise to make. I usually use 2% difference rule - but you can say, I'm happy with 10% difference - and I'll use rather short exposures.