ZWO ASI1600MM-Pro - Darks/Bias Necessary?

[mikey2000]

As ever, with the fast moving world of sensors, I'm struggling to discover the current, up to date view about the need for Bias and Dark frames for this newish Mono sensor.

 

I keep reading different things!    Can I skip the bias frames?  Will I need them to properly calibrate the lights and flats?   With my old DSLR sensor, I'd have bias, darks, flats and lights and seemed to get by.

When I make flats, do I need to make one for each filter (I have LRG, B and Ha in the filter wheel)

 

As it's gloomy here, I'm just thinking of building a library of masters for my new 'proper' astro camera.  

 

Any advice much appreciated!

 

Just for fun, here's a screen grab of an integrated set of 128 bias shots.  It looks sort of stripy with a bit of gradient.

[mikey2000]

Just for interest, here are the stats for that master bias file

[SyedT]

I have the ASI 1600MM-C V3; darks are necessary particularly if you're doing longer exposures to cut out the amp glow; they really do help a lot from my experience. I didn't really take flat frames initially as with my 2 inch filters I wasn't getting any vignetting, but since I've started doing flats I've noted that the images seem to have a better illumination profile, and of course all optical train dust bunnies are removed. According to my understanding, bias frames are subtracted from flat frames to keep only the illumination profile, so bias frames would definitely help. Each filter needs to have its own set of flats as they will let different amounts of light through (particularly LRGB vs NB). You'll only be able to build a master for your darks/bias for use later unless your imaging rig never rotates/collects dust (as that's what flats help to reduce); I take flats for the appropriate filters after I complete imaging sessions.

Edited February 5, 2018 by SyedT

[vlaiv]

Yes you can skip bias frames, not only that you can skip them, they are not needed with set point cooled cameras, if you use darks of matching length and temperature.

Calibrate like this:

master dark = avg(darks)

master flat dark = avg(flat darks)

master flat = avg (flat - master flat dark) or
master flat = avg(flat) - master flat dark

calibrated light = (light - master dark) / master flat

Use as much calibration frames as you can. Use at least 32bit precision, and that is it.

[Demonperformer]

Thanks for that .

I was reading your dithering thread this morning and a lot of it was way over my head, but this makes the requirements very clear for the 1600 (which is also my camera).

[ollypenrice]

I must say that I usually use luminance flats for all filters and that they work fine. If, on occasion, they don't I shoot flats per filter - but this is exceptional.

I don't follow Vliav's procedure at all, but that's with CCD, set point cooled. I use a master bias as a dark and a master bias as a flat-dark. I use a defect map as well and an aggressive hot pixel filter when stacking. For me this works better than the standard dark subtraction routine. Much better, in fact. So all I can usefully suggest is that you experiment to see what works best for you. Don't get hung up on the theory. You won't be posting the theory on the imaging boards.

Olly

[mikey2000]

Thanks for the tips everyone.  Vilav - is your procedure meant for CCD sensors?  I have a CMOS.

I think Olly may have hit the nail on the head - Experiment and see what happens. 

 

I think I was just being lazy - someone else must have done this before me, with this camera!

[wimvb]

Even with the PRO version of the ASI1600, there is amp glow. The only way to calibrate this out is with matching darks. In my case, with the ASI174, "optimised" darks will not completely remove the glow, but matching darks will (temp, time, gain, offset).

My procedure is:

Lights and darks at the same gain, offset, temperature and exposure time. Flats and dark flats at 0 gain and matching exposure time. Intensity for flats set to 25 000 ADUs.

No bias frames.

So far I've used a set of flats for each filter, but I must try Olly's method of only using lum flats. That would save some processing time, and disk space.

[vlaiv]

3 hours ago, mikey2000 said:

Thanks for the tips everyone.  Vilav - is your procedure meant for CCD sensors?  I have a CMOS.

I think Olly may have hit the nail on the head - Experiment and see what happens. 

 

I think I was just being lazy - someone else must have done this before me, with this camera!

It is meant for both CCD and CMOS sensors, well, at least "well behaved" ones. It is proper calibration method in sense that it removes all but light signal from light frame and correctly applies flats (only on light signal).

I have ASI1600 myself and that is what I use for calibration.

On a side note, well behaved sensor is one that:

- has predictable bias signal (read noise is random but bias signal is always same for every pixel)

- has dark signal dependent on temperature and exposure duration (linearly grows with exposure length on fixed temperature). Can have uneven dark current (amp glow), but it needs to behave as said per pixel

- has linear response to light.

Edited February 6, 2018 by vlaiv

[wimvb]

6 hours ago, vlaiv said:

has dark signal dependent on temperature and exposure duration (linearly grows with exposure length on fixed temperature). Can have uneven dark current (amp glow), but it needs to behave as said per pixel

Here is where theory goes awry. Amp glow does not scale linearly with time. Amp glow is due to an uneven heating of the sensor. A local increase in temperature leads to a local increase in dark current. If the local temperature were constant, amp glow would increase linearly with time. But temperature increases with time, probably until a steady state is reached, and amp glow will increase more than linearly with time. This is the reason why amp glow in ASI cameras can't be calibrated out completely with darks taken with a different exposure time.

[vlaiv]

2 hours ago, wimvb said:

Here is where theory goes awry. Amp glow does not scale linearly with time. Amp glow is due to an uneven heating of the sensor. A local increase in temperature leads to a local increase in dark current. If the local temperature were constant, amp glow would increase linearly with time. But temperature increases with time, probably until a steady state is reached, and amp glow will increase more than linearly with time. This is the reason why amp glow in ASI cameras can't be calibrated out completely with darks taken with a different exposure time.

Very good point, I have not look into it but it looks like I'm going to have to.

There is another explanation why ASI1600 would not be able to calibrate properly with different exposure darks. It turns out there is sort of "floating" bias with ASI1600. Whether this is something implemented in drivers or it is feature of sensor itself, I have no idea, but it turns out that bias level drops with exposure length. I've seen while measuring sensor data that average levels for 1m exposure are lower than that of straight bias for same settings and temperature. I was under the impression that it stabilizes at some point, but it might not be so. Only thing that I've noticed is that after 30s or so average value starts rising - as one would expect due to buildup of dark current.

If sensor indeed changes bias level depending on exposure length (and it seems so) it would also be a reason why different exposure darks would fail at calibration. One needs bias removed prior to adjusting dark current levels.

On the amp glow topic - it sounds reasonable what you are saying but that would have another implication. There would be difference in amp glow between darks of same exposure and settings taken under different surrounding temperature conditions. Equilibrium in amp glow would depend on efficiency of heat dissipation from the system - which is higher in cold environment than it is in hot. I would expect amp glow to be less in darks taken on a cold night outside vs darks taken at room temperature. I'm shooting my darks indoors, and calibrate with them lights taken in at least 10C colder conditions. I've not noticed residual amp glow after calibration. I'm not saying that it is not there, I just have not seen it, and it might well be because I was not paying attention.

If it turns out that there is indeed dependence of amp glow to ambient on same temp and settings, I guess one would need to have couple of sets of master darks taken at different ambient temperatures (like summer darks and winter darks and one set for spring / autumn or something like that).

I don't really know the nature of amp glow in CMOS sensors. With CCD sensors it was indeed amp glow - since CCDs have single amp unit, and if that electronics was too close to sensor it would cause problems - heat travels via circuitry back towards the CCD (metal conducts electrons one way but equally good heat in opposite direction). With CMOS sensors there is amp unit associated with each pixel, so I doubt what we see and call amp glow is indeed related to amp unit(s) with CMOS.

[vlaiv]

On topic of CMOS amp glow:

https://www.cloudynights.com/topic/599475-sony-imx183-mono-test-thread-asi-qhy-etc/page-3

There is some interesting discussion and data.

One thing that intrigues me is ray like feature in amp glow in CMOS sensors. That is not something I would expect from heat. It looks more like there is some sort of electron build up due to electric / magnetic fields from adjacent circuitry rather than heat. Due to fast readout there are components in CMOS sensors that operate in Mhz range. Could this all be due to rouge EM radiation?

[wimvb]

7 minutes ago, vlaiv said:

Equilibrium in amp glow would depend on efficiency of heat dissipation from the system - which is higher in cold environment than it is in hot. I would expect amp glow to be less in darks taken on a cold night outside vs darks taken at room temperature. I'm shooting my darks indoors, and calibrate with them lights taken in at least 10C colder conditions. I've not noticed residual amp glow after calibration. I'm not saying that it is not there, I just have not seen it, and it might well be because I was not paying attention.

If it turns out that there is indeed dependence of amp glow to ambient on same temp and settings, I guess one would need to have couple of sets of master darks taken at different ambient temperatures (like summer darks and winter darks and one set for spring / autumn or something like that).

With an active cooling element (peltier) rather than passive, fan cooling, ambient temperature should be less of an issue. Once the temperature us set, ambient temperature only determines how hard the cooling system has to work. The only times I've had problems with this was when shooting darks. I have my camera in a black box (no pun intended), and the temperature rises a few degrees during capture. If the set point is close to what the camera can handle, the set temperature may not be reached. But as long as the set temperature is welk witjin the camera's limit, I've never had a problem.

Btw, I suspect the ASI's temperature sensor to be a bit on the optimistic side.

[vlaiv]

1 minute ago, wimvb said:

With an active cooling element (peltier) rather than passive, fan cooling, ambient temperature should be less of an issue. Once the temperature us set, ambient temperature only determines how hard the cooling system has to work. The only times I've had problems with this was when shooting darks. I have my camera in a black box (no pun intended), and the temperature rises a few degrees during capture. If the set point is close to what the camera can handle, the set temperature may not be reached. But as long as the set temperature is welk witjin the camera's limit, I've never had a problem.

Btw, I suspect the ASI's temperature sensor to be a bit on the optimistic side.

Regardless of peltier cooling, if amp glow is produced by heat buildup - this means there is excess heat that is not being removed via peltier cooling. Such heat will be dissipated by regular means of heat transfer to surroundings (conduction and convection). Efficiency of such heat removal will depend on environmental temperature. Even if we look at camera as closed system, camera casing will be at different temperature (it is used to remove the heat from the system) depending on actual surrounding temperature. In cold environment casing will remain relatively cold. In hot environment casing will become hotter and hence everything inside casing will be at higher temperature, meaning less effective heat dissipation of amp glow producing component - bigger thermal build up in it - stronger amp glow.

I guess temperature sensor does not need to be accurately calibrated as long as it is "reproducible". Whether I'm shooting at -18C while it says -20C, I don't think it matters as long as each time I tell it to go to -20C it remains on -18C

 

 

[mikey2000]

I decided to set my camera to a temp of -20C.   After a bit of a cooling cycle, APT shows the sensor temp steady at -20.  I understand this to be an absolute temperature, not the difference from ambient. (Eg, when I switch it on indoors, the camera temp reads about 8 degrees, the same as the garage in which it lives).  The cooler fan switches on occasionally to maintain the set temperature. 

Very nice.

So it seems that maybe I should try this:

1. Shoot lights at -20C and whatever gain and exposure length. (Lets say 120seconds)
2. Shoot a stack of darks with the same -20C, same gain and same exposure length as the darks.
3. Shoot some flats (probably just one set based on the L filter - I don't see much vignetting yet but it's still cloudy here....)
4. Shoot some dark flats (same exposure length, gain, temperature as the flats)
5. Make some master Darks, Flats and Dark Flats.
6. Calibrate lights.
7. Stack.
8. Process

The I can keep the masters in a library for later re-use.

 

 

[mikey2000]

One further question.  Regarding shooting flats.   How do I measure the ADU to so I can go for 25000?  I've installed APT and I also have pixinsight now.

 

In my DSLR, I'd just set the exposure to get the histogram somewhere left of centre.  Is this more or less the same idea? 25000 sounds a bit like 1/3 of the way to 65536....

[wimvb]

53 minutes ago, vlaiv said:

In hot environment casing will become hotter and hence everything inside casing will be at higher temperature, meaning less effective heat dissipation of amp glow producing component - bigger thermal build up in it - stronger amp glow

Yes, the casing will be warmer, but the whole idea of Peltier is to keep a fixed temperature on the sensor. Your argument doesn't take Peltier power into account. With a higher ambient temperature, the heat flow will be larger.

[vlaiv]

7 minutes ago, wimvb said:

Yes, the casing will be warmer, but the whole idea of Peltier is to keep a fixed temperature on the sensor. Your argument doesn't take Peltier power into account. With a higher ambient temperature, the heat flow will be larger.

Not sure how is that related to amp glow if we assume it is due to build up of heat on adjacent component and it is being fed into sensor via metallic circuitry. Sensor is kept at constant temperature because all excess heat is taken away by Peltier. If amp glow is rising faster than dark current - that means that part of chip is at higher temperature than the rest of the chip, and it is accumulating heat (if it were only at higher temperature but stable we would not be in trouble, since there is time linearity at different temperatures). Some of that heat will indeed be taken away by Peltier via sensor, but if it is rising, it will also be in dynamic with its surroundings - some of the heat will dissipate elsewhere because not all of it is being drawn away by Peltier. Otherwise amp glow temperature would also be stable. And if it is dissipating heat by other means than Peltier - speed of dissipation will depend on temperature gradient between that component and wherever that heat goes to.

[spillage]

1 hour ago, mikey2000 said:

How do I measure the ADU to so I can go for 25000

In apt you can set this using the flat assistant in tools. Just tell it the adu level and which filters to use. It will run and then add an new flats setting (under the light settings in the camera tab). I think.

sorry its called ccd flats aid. If you want it to create a plan make sure that box is ticked.

Edited February 6, 2018 by spillage

[wimvb]

7 hours ago, vlaiv said:

Not sure how is that related to amp glow if we assume it is due to build up of heat on adjacent component and it is being fed into sensor via metallic circuitry. Sensor is kept at constant temperature because all excess heat is taken away by Peltier.

On ASI cameras, amp glow is generated by circuitry on the sensor, and this can't be shut off during exposure or read out. That's why ZWO have such a hard time trying to reduce it. The Peltier cools the whole sensor, but there are still variations across the sensor, hence amp glow.

Btw, the imx183 image you posted, looks more like radiation than amp glow. Very strange. Could this just be a faulty sensor? Or a light leak maybe.

[vlaiv]

1 hour ago, wimvb said:

Btw, the imx183 image you posted, looks more like radiation than amp glow. Very strange. Could this just be a faulty sensor? Or a light leak maybe.

I've seen it before, it looks like it is very common on Sony CMOS sensors. My ASI178MCC has amp glow that is cross between ASI1600 and that one on the image. Corners are affected like in ASI1600, but in each corner there is "source of rays".

Here is what it looks like:

And to my knowledge, my ASI178MCC calibrates ok regardless of strange amp glow.

1 hour ago, wimvb said:

On ASI cameras, amp glow is generated by circuitry on the sensor, and this can't be shut off during exposure or read out. That's why ZWO have such a hard time trying to reduce it. The Peltier cools the whole sensor, but there are still variations across the sensor, hence amp glow.

Ok, still not following (sorry about that). Well I do understand what you mean I just don't understand fully consequences of that.

So you are saying, there are parts of the chip that are at different temperature than the rest of the chip? Right?

Do you have any idea how that temperature behaves in time?

1. It is just a patch of sensor surface where temperature is higher then the rest of the chip but stable - let's say sensor is mostly at -20C but corners are at -18C

2. When there is no exposure going on, whole sensor is at -20C and when exposure starts, sensor remains at -20C but selected few patches in corners start rising in temperature

2a) they rise asymptotically to a certain value quickly (like in few seconds)

2b) they rise asymptotically to a certain value slowly (thus not really reaching that value in common exposure times)

 

[mikey2000]

I made two master darks with my camera.  One for 120s, one for 300s.  Both at -20C, Gain 0 (The driver calls it 'High Dynamic Range')  (Stretched to view with pixinsight STF)

They are clearly from the same camera but are clearly not the same either.

 

There's a sort of circle shape visible (middle/left)  Can you see that?  I wonder if I had a light leak?  I shot them in the garage with the light off and the lens cap on.

[mikey2000]

the semi-circle shape is easier to see if you zoom right out, making the images almost thumbnail size.

[mikey2000]

When shooting with this camera, should I pause between exposures, to allow the file transfer to complete before starting the next shot?  Does that reduce amp glow?  I thought that's what the 256MB of on-board DDR memory was for :-)

 

More experiments to follow!

[wimvb]

1 hour ago, vlaiv said:

So you are saying, there are parts of the chip that are at different temperature than the rest of the chip? Right?

Yes

1 hour ago, vlaiv said:

Do you have any idea how that temperature behaves in time?

No, just that temperature increases during an exposure, such that amp glow is worse (more than linear) at longer exposure times. You may have noticed that images with sub frame exposures in excess of about 600 s are very rare. Not only are they not needed because of the low read noise, but they are actively avoided because of the increased amp glow. Even if amp glow can be removed by careful calibration, the random shot noise that is associated with it can not.

How long it takes for the increased temperature to reach a stable level, I don't know, but it is probably several tens of seconds.