ASI1600mm cool

[derrickf]

I noticed this in a thread on the PI forum about a different topic here.

About halfway down the first page Vicent Pieris indicates that PI will have a process for correcting amp glow; this may be of use to owners of the ASI 1600 :

Quote from: IanL on 2017 January 09 10:17:16

- If your CMOS camera suffers from repeatable banding, amp glow or fixed pattern noise, then you should turn off PI's dark frame optimisation and use a time and temperature matched master dark.  Otherwise it will likely over or under-correct and those artifacts will only be partially corrected (or indeed made more visible).  This is certainly the case for my CMOS-based ZWO ASI1600MM-cool.  A temperature and time-matched master dark without optimisation works very well and deals with the amp-glow that becomes visible and increases after about 60 seconds of exposure.  With optimisation turned on, PI under-corrects by applying a scaling factor of about 0.45 despite a matched dark frame.  This is unsurprising since the optimisation process is global and I expect it is being thrown by the significant variation across the frame.

Hi,

We're designing an amp glow correction right now, so in the future you'll be able to optimize the amp glow subtraction.
 

HTH

Derrick

[Gina]

I followed IanL's advice in this forum about turning off PI's dark frame optimisation and my amp glow was almost perfectly corrected   But if amp glow is going to be directly targeted in PI that is very good news

[Magnus_e]

I have a question to anyone using this camera on Windows. 

When looking at posted settings for this camera, something called Unity(Gain) is often mentioned. I'm trying to track down the settings for the asi on linux with INDI, and so far I know Gain=Gain, Brightnes=Offset. The INDI driver also has Gamma, witch looks like a setting not used (implemented) in ascom.

However, when I asked about Unity on the zwo forum, the reply was there are no setting named Unity.

http://zwoug.org/viewtopic.php?f=17&t=6756

Does anyone know what this unity or unity gain I have seen posted is? Is it a setting that can can be toggled on and off, a setting that takes a numerical value, or is it just a therme used about low gain settings (or something else)

Thanks in advance, Magnus

[Gina]

Unity gain refers to a Gain setting of 139.  It's the value that produces 1 ADU per electron.  Doesn't mean very much in practice except that no extra dynamic range is obtained by going below a gain of 139, as I understand it.

[Guest]

1 hour ago, Magnus_e said:

I have a question to anyone using this camera on Windows. 

When looking at posted settings for this camera, something called Unity(Gain) is often mentioned. I'm trying to track down the settings for the asi on linux with INDI, and so far I know Gain=Gain, Brightnes=Offset. The INDI driver also has Gamma, witch looks like a setting not used (implemented) in ascom.

However, when I asked about Unity on the zwo forum, the reply was there are no setting named Unity.

http://zwoug.org/viewtopic.php?f=17&t=6756

Does anyone know what this unity or unity gain I have seen posted is? Is it a setting that can can be toggled on and off, a setting that takes a numerical value, or is it just a therme used about low gain settings (or something else)

Thanks in advance, Magnus

This question regarding unity gain drove me nuts when I first got this camera LOL!

It was continually referred to as if it was some secret that only the pro's would be "hip" to if they had done their homework :-)

But it turns out a gain setting of 139 (unity) is really useful with this in all conditions I've used it (narrow band, LRGB). I've also used an offset of 5 (I think, maybe 10) as the one time I moved that the results were poor.

The thing I like about this standard (?) setting is that I didn't have time for guess work or experimentation when I first got the camera - I just wanted to see if it worked and needed some basic settings to get started.

Now I've used it for a while - I have no excuse and may start experimenting :-)

David

[alexbb]

1 hour ago, Gina said:

Unity gain refers to a Gain setting of 139.  It's the value that produces 1 ADU per electron.  Doesn't mean very much in practice except that no extra dynamic range is obtained by going below a gain of 139, as I understand it.

There's this image describing several characteristics and their values:

The 139 gain value corresponds to ~1e-/ADU - 1 electron recorded and converted as a value of 1 by the Analog-to-Digital Unit (ADU). The maximum Dynamic Range (DR) is obtained at 0 gain, though, if you don't saturate the signal with an exposure, you cover a higher range of digital values with a higher gain (>0, but <= 139). And you can recover the dynamic range by stacking multiple subs.

There are 3 values that are more important: Highest Dynamic Range - at 0 gain, Unity Gain - at 139 gain and Lowest Read Noise - at 300 gain. You can read further about the read noise and e- rms: http://www.hamamatsu.com/jp/en/community/life_science_camera/references/technical_guides/read_noise/index.html and http://www.andor.com/learning-academy/read-noise-understanding-scmos-read-noise 

And about the offset - you could raise it until the darkest pixel is considered black: http://www.cloudynights.com/topic/542048-gain-and-offset-settings-for-zwo-cameras/

HTH,

Alex

[Filroden]

So am I right in thinking in layman's terms

 

For gain (which multiplies signal/noise):

- if I was imaging something very bright and my image was being saturated, even at lower exposures - reduce exposure (where possible) and gain (when not)

- if my image was faint and never saturating, even at longer exposures - continue to raise exposure (where possible) and gain (when not) until you are close to, but not quite saturating

 

For offset (which adds to signal/noise):

- if my image contains any pixels that have 0 value, raise offset until no pixels have 0 value

- if none of my pixels have zero value, reduce offset until they care close to, but not quite 0 value

 

That said, I stick to the three presets (0 gain with 10 offset, 139 gain with 21 offset, 300 gain with 50 offset) as I figure ZWO have put some effort into determining these are good values. Though I guess offset depends heavily on background sky signal and whether filters are being used, as high background light means little or no offset will be needed to raise pixel values high enough to prevent readings of 0.

[alexbb]

5 minutes ago, Filroden said:

For offset (which adds to signal/noise):

- if my image contains any pixels that have 0 value, raise offset until no pixels have 0 value

- if none of my pixels have zero value, reduce offset until they care close to, but not quite 0 value

If your image contains any pixels that have 0 value, lower the offset. You could consider the offset as the low clipping point.

[Filroden]

1 minute ago, moise212 said:

If your image contains any pixels that have 0 value, lower the offset. You could consider the offset as the low clipping point.

I thought it would be the other way around? The offset value is added to all pixels so even if I exposed an image in a perfectly dark room, no pixel would have a value lower than the offset value. This allows for noise, which can vary up and down from the true value. If noise could reduce a pixel value by 10, then I need an offset of at least 10 to prevent black clipping. Therefore, in my real image, if I see any pixels with 0 value then I still don't have a high enough offset to compensate for my noise.

[alexbb]

31 minutes ago, Filroden said:

I thought it would be the other way around? The offset value is added to all pixels so even if I exposed an image in a perfectly dark room, no pixel would have a value lower than the offset value. This allows for noise, which can vary up and down from the true value. If noise could reduce a pixel value by 10, then I need an offset of at least 10 to prevent black clipping. Therefore, in my real image, if I see any pixels with 0 value then I still don't have a high enough offset to compensate for my noise.

I don't believe that noise could reduce the value, only increase. And the default offsets for 0 gain, 139 and 300 are 10, 21 and 50 if I remember well. So as the signal is multiplied more as the gain increases, I suppose that the lowest value increases as well. That's why the default offset is higher for higher gain?! I thought the "offset" word was used (wrong) to describe a low clip - the point above where the signal is converted to a >0 value and therefore was subtracted from.

Hmm, this is the logic that I followed, I might be wrong. I can't figure out why for a lower gain you should use a lower offset.

[Filroden]

27 minutes ago, moise212 said:

Hmm, this is the logic that I followed, I might be wrong. I can't figure out why for a lower gain you should use a lower offset.

Gain will multiple noise as well as signal, so you need a larger offset to prevent clipping.

[Guest]

Excellent discussion on gain and offset - I wasn't aware of the ZWO suggested combinations (in particular gain 139 and 21 offset - I've been using 5 or 10 I think)

I'll give this a try

David

 

[Allinthehead]

My own observations have led me to use an offset of 15 for higher gain eg 300. I've found that using a lower offset keeps stars smaller and seems to give better contrast. 

[alexbb]

49 minutes ago, Filroden said:

Gain will multiple noise as well as signal, so you need a larger offset to prevent clipping.

Yes, that's what I did not understand from the ASCOM defaults. Since you already multiply the signal (real signal + noise), I expect that for higher gain values to pass the digital converted 0 value with a larger one.

I mean:

At gain 0: (10(offset) + X(real signal + noise) * Y(gain multiplier) >= 1

At gain 139: (21(offset) + X(real signal + noise) *Z(gain multiplier) >= 1; Z > Y

Why would you need a larger offset for a higher multiplier value? What am I wrong with?

[Filroden]

I don't know in what order gain and offset are applied. If offset is applied first then it will be multiplied by gain and a smaller offset would be better at higher gain. If applied after gain then it will need to be higher.

I'd assumed offset would be applied after gain otherwise it would eat up too much of the dynamic range but based on @Allinthehead observations it would seem offset is being applied before gain.

[alexbb]

If offset is applied (added) after gain in order to rise the minimum digital value to 1, then why would it even be configurable?

For a higher gain, the signal (real + noise) would be higher than for a lower gain. Adding offset at any point (before or after multiplying the signal) would result in a higher value. The only possible solution for this would be that some input value is <0 before the multiplication is done, but I don't see which value could be negative.

[IanL]

You've got it backwards.  Offset is (effectively) the number of electrons (after amplification, determined by gain) that corresponds to an ADU value of zero.  So if I read out a pixel and get a value of 15 electrons, with an offset of 14 I will get an ADU of 1, an offset of 15 gives me an ADU of 0, and an offset of 16 also gives me an ADU of 0 since everything with an  electron value below 16 will be clipped to zero ADU (black).

That's why as you increase gain you increase the offset.  Higher gain values lead to higher minimum electron values (i.e shifting the histogram to the right), so the offset is applied to shift it back to the left and make maximum use of the number of bits available in the A/D process (10 or 12 bits for the 1600 depending on which mode you use).

If you see pixels with an ADU of zero in your uncalibrated frames (other than dead/cold pixels) the you need to reduce the offset to move the black level to the left.

[Filroden]

So offset is subtractive, not additive? So if I have any zero value pixels I have to reduce offset until there are none? And offset is applied after gain?

Like ((s * g) - o)?

[alexbb]

2 minutes ago, IanL said:

You've got it backwards.  Offset is (effectively) the number of electrons (after amplification, determined by gain) that corresponds to an ADU value of zero.  So if I read out a pixel and get a value of 15 electrons, with an offset of 14 I will get an ADU of 1, an offset of 15 gives me an ADU of 0, and an offset of 16 also gives me an ADU of 0 since everything with an  electron value below 16 will be clipped to zero ADU (black).

That's why as you increase gain you increase the offset.  Higher gain values lead to higher minimum electron values (i.e shifting the histogram to the right), so the offset is applied to shift it back to the left and make maximum use of the number of bits available in the A/D process (10 or 12 bits for the 1600 depending on which mode you use).

Yep, that's how I thought it initially. In the software configuration, when you increase the offset you actually subtract the offset value from the ADU value.

[alexbb]

1 minute ago, Filroden said:

2 minutes ago, Filroden said:

So offset is subtraction, not additive? So if I have any zero value pixels I have to reduce offset until there are none? And offset is applied after gain?

Like ((s * g) - o)?

 

I'm not sure of the terms in English, but I think that an offset voltage is added when you increase the gain and then the ADU compares the signal*gain with the offset. So it behaves as a subtraction in the end.

[IanL]

8 minutes ago, Filroden said:

So offset is subtraction, not additive? So if I have any zero value pixels I have to reduce offset until there are none? And offset is applied after gain?

Like ((s * g) - o)?

Yes just edited my post to say that as you posted yours.

7 minutes ago, moise212 said:

Yep, that's how I thought it initially. In the software configuration, when you increase the offset you actually subtract the offset value from the ADU value.

It's important to understand that the offset is not subtracted from the ADU value.  That would be pointless as you'd just shift the unused bits in the ADU from the left (low) end of the range to the right (high) end of the range.  The offset is the number of electrons that is subtracted from those read from the pixel before it is put through the analog to digital converter.  Thus:

Number of Electrons in Sensor Well

:

Number of Electrons in Sensor Well x Gain

:

Number of Electrons in Sensor Well x Gain - Offset Number of Electrons

:

Analog to Digital Conversion

:

ADU Value in Image File

 

That way you end up using the full range of possible ADU values rather than leaving some empty at one end or having voltages that are too high to be measured by the A/D converter being clipped to the maximum ADU value at the other.

[Filroden]

1 minute ago, IanL said:

That way you end up using the full range of possible ADU values rather than leaving some empty at one end or having voltages that are too high being clipped to the maximum ADU value at the other.

This bit I can understand  I just find that terminology tends to be driven by the engineers that develop the kit and not for the customers who use it.

I like simple things. If my image can only contain values from 0 to 4096, I only want to see values ranging from 1 to 4095 (so I know I'm not clipping). I want to turn the two dials (currently called gain and offset) to expand my image so it occupies the most values without clipping. I think I get confused because of the whole ADU element of the process. To me, that just needs to be a black box and all I see in my image summary is the min/max pixel values in my image. I need to adjust my exposure length (first preference but not always possible because of mount limitations) and my gain/offset to spread this range as much as possible. I just want to know which way to change my dial depending on what I see in the min/max pixel summary. Gain seems intuitive, offset seems counter intuitive!

[IanL]

1 minute ago, Filroden said:

offset seems counter intuitive!

Only because people think of it in terms of the offset being applied to the pixel values, whereas if you think of it correctly as the offset being applied to the location of the zero value it makes perfect sense.

[Filroden]

I think users will naturally think in terms of how it affects the output (pixel values) because they care about the result they see and not the way it was made. But that's not how an engineer would logically design it.

It's the constant work battle I face getting engineers to think like users and not building interfaces that are counter intuitive. PixInsight is a perfect example. The number of times I've seen a slider that you move it to the right (usually signalling an increase in something) to reduce something (e.g. in the image analysis script you increase the sensitivity slider (and it's value) to reduce the number of stars detected!!). I'm sure this represents some pure mathematical expression (no dount sensitivity is a divisor) but it's confusing to a user.

Move slider to right means increase, turn dial clockwise means increase. 

Increase gain sees pixel values increase.

Increasing offset should likewise see pixel values increase, so either change the operator in the code or change the scale to be negative  

[Guest]

Thanks guys - I've got the next script for "Yes, Minister" LOL!

(don't be offended - I mean that in a good humour way - the terminology vs. translation to layperson terms is hilariously difficult :-)

David