QHY163c - Strange ADU behaviour. Bias & Darks Median ADU are double the expected amount (Offset)

[Xiga]

Hi guys

I recently picked up a used QHY163c, and i noticed something strange while taking some calibration frames. I can't explain it, so thought who better to ask than the fine SGL community!

Basically, whenever i take some Bias or Dark frames, SGP shows the Median ADU amount as being precisely twice the value of what i am expecting. As SGP reports the ADU count in 16bit, and the QHY163c is 12bit, then AFAIK all i should have to do is divide by 16 and the result should be equal to (or very very close to) the Offset value. I tested numerous combinations of Gain and Offset, but the result was always twice as high as it should be. 

I've included links below to 10 Bias and 10 Darks. They were taken recently, indoors at night, at Gain 80, Offset 35, and at -15C. Nothing else was connected to the laptop other than the camera. SGP shows the Median ADU as 70, instead of 35. 

Everything is seemingly working fine. No crashes or hangups, and files download in about 5 secs, so everything seems ok on the surface. I wonder is it possible the firmware is doubling the Offset behind the scenes somehow? 

I'm convinced the camera is fine, it's just i can't explain what is going on here, and i want to make sure it's not anything silly that i am missing, or if it is even going to be an issue for me going forward. Note, i still haven't given the camera a proper first light test yet, but there's a chance i might be able to tomorrow night. 

Thanks in advance for any advice!

Bias:      https://1drv.ms/f/s!AhhWC3D3zU7BpQjaXQo83p953Thp

Darks:   https://1drv.ms/f/s!AhhWC3D3zU7BpQlBlAUwG-8L2szP

ps - The Darks were 180s long, and the Bias were 0.3s long (as i know Bias don't work at extremely short exp times with these cameras). 

Edited May 19, 2019 by Xiga

[wimvb]

On 20/05/2019 at 00:37, Xiga said:

The Darks were 180s long, and the Bias were 0.3s long (as i know Bias don't work at extremely short exp times with these cameras). 

Do darks and bias frames have the same values? Offset is a value that is added to the collected data in order to make sure that no pixels are clipped. The mean background value should be higher than the offset. But in the bias frames it shouldn't deviate too much from the offset. For darks, the mean value can and should be higher due to dark current.

[vlaiv]

I'll examine subs you posted, but there could be a simple explanation.

I can't remember now, but it was discussed sometime ago - offset can be added at both pre amp and post amp stage - meaning that it can be multiplied by gain or not. This could be one explanation.

Second explanation would be that since some vendors use gain in arbitrary units, offset might also be in arbitrary units?

I remember older QHY cams having 0-100 gain in some drivers. That did not translate into anything meaningful, or one needed to measure e/ADU values to get proper correspondence. Is there a "table" or formula for gain - e/ADU conversion for this camera? Maybe offset is implemented in drivers in similar way - meaning 0-100% of possible offset ranges (which are real numbers but something like 0-220e or something like that).

 

[fireballxl5]

Signal gain is usually a simple ratio and so is dimensionless

[vlaiv]

Just look at "bias" - it looks very nice.

Indeed 70e is median (mean is a bit lower - I'll address this) and from quick inspection I would say that it is a good value. SGP reports e/ADU of 1 (fits header), while it is clearly not - this means that driver is not reporting proper value - no biggie as long as you know from gain table what you are using.

Bias looks good, there are no zero values (except interesting issue) around. I can't explain why is actual bias level 70 when you set it to 35. e/ADU is ~1.8755, so if offset is applied prior to gain, expected value would be ~18.6617. If it is applied after, well expected value would be 35.

Only option that I can think of is that offset is not set in units of e, but rather some arbitrary units. You can do a experiment and set 3-4 different offset values and shoot dozen of bias files to see how median value changes with given offset. Maybe try changing gain also, until you figure out what is going on. This is only if you are curious - I would not mind using camera like this.

Now here is interesting thing - top 3 rows of pixels have 0 values in every single pixel. Like whole 3 rows have been "shut down" or not read or something like that. This might be due to drivers or has something to do with how camera was manufactured - electronic design. Do keep it in mind though - as it might cause you problems sometimes and it's good to know that artifact exists so you can take steps to circumvent any problems.

[vlaiv]

1 minute ago, fireballxl5 said:

Signal gain is usually a simple ratio and so is dimensionless

ZWO has very nice convention of using logarithmic gain values - with their cameras gain number represents actual gain expressed in 0.1dB.

[vlaiv]

I have to correct myself.

I now downloaded all 10 bias files, and things got more interesting.

First bias frame has first three pixel rows with pixels set to exactly 0. All other bias frames have only first row with all pixels set to exactly 1120 - or rather 70 if we divide with 16.

I don't know if this is due to camera or possibly due to storage / transfer (if you used failing usb memory stick or something and copied files one by one? just a thought).

[Xiga]

Thanks for all your input guys.

I asked over on the QHY forums and the answer I got direct from QHY was...'This is normal, it is because the OFFSET of this chip is added after the ADC, so it doesn't change with the gain'.

Does that explain things then?

@vlaiv that's interesting about the first few lines of pixels. I copied the files onto a 64GB USB3 flash drive before moving them to my desktop and uploading them to the cloud. There were no problems with copying them AFAIK. Although it's a little worrying that there is variation between the subs. How did you identify this if you don't mind me asking? And when you mention the possibility of this causing problems and taking steps to avoid it, what do you mean exactly?

@wimvb I had another look at a Dark and Bias in SGP. I've pasted below the statistics for both. Do these look normal? Based on what you said above, I would expect the Dark to have a higher Mean ADU than the Bias, but it's actually lower.

BIAS:   

DARK:

 

[vlaiv]

I just loaded files in ImageJ and did statistics on them. Initially I downloaded only first bias file.

It became apparent that something is off when I looked at the stats - it said that minimum value in frame is 0 while median was quite high - at 70. In my experience such high gain should not produce 0 values. This points to a dead pixel or something like that - single value that is set at 0. So I took the histogram to see what it looks like and when examining histogram I noticed that very large number of pixels had 0 value - that kind of surprised me.

So I started to examine image in close up trying to see if I can find any offending 0 valued pixels - and while I was doing that I noticed that top part of image is black and uniform - first three pixel rows. They all had 0 value.

Later when I downloaded other bias files and looked at the top of the image - I noticed that they all have 1120 values in first row (not first three rows).

Idea about usb memory stick being faulty (I'm not sure it is likely in this case) came from earlier. I exchanged quite a bit of PMs with Rodd about his setup and optimum sampling / workflow. He also attached some of his raw stacks and single subs and they had lines with 0 valued pixels in them - however in his case those were not full rows nor were they top rows - they were rather random, but were easy to spot since they were at light frames which have dynamics and higher background because of LP levels. We concluded that it was faulty USB stick because he did not have those artifacts on his work laptop but used usb stick to transfer files to computer connected to internet.

When I say that it might cause you trouble - this is what I mean, for example when you align such frames and you stack them you will have pronounced stacking edge artifacts on one side that may puzzle you. Or when you are trying to do statistics on pixel values - stats will be thrown off by same values in one row, and such.

Best way to deal with this if you indeed find it is related to camera is to crop your frames - just remove offending top 3 pixel rows or as many as needed.

On a separate note, thing that you mention that bias frames have higher mean value than darks is known thing for this sensor, and one of the reasons why bias should not be used. I think that if you increase duration of your "bias" at some point it will start acting normally.

Might be that it is related to the way how sub duration is measured. I remember reading that up to 1s exposures camera firmware is responsible for keeping time. Subs longer than 1s work in sort of "bulb" mode - computer tells camera to start exposure and then signals when exposure should end and reads off result. It might be that those two different timing mechanisms result in different offset values or something.

[wimvb]

7 hours ago, Xiga said:

I would expect the Dark to have a higher Mean ADU than the Bias, but it's actually lower.

That would be expected. But I can see one scenario where this may not be the case. Image sensors normally have non-active pixels along the edge. These pixels do not receive any light, but they are read after each exposure. The sensor has image processing capabilities on chip. The mean readings from the non-active pixels may be subtracted on chip from the signal from the active pixels. This is a very crude way of dark subtraction, which is aimed at keeping the highest value range in an image. Astrophotographers who use dslr cameras have noticed this at times. Btw, this is inherently different from in camera dark subtraction, where an extra image is taken by the camera after the normal exposure has ended.

The result of on chip dark subtraction is that the average dark signal is always about the same value, independent of exposure time. But faulty pixels and amp glow, etc are still present in the image. These faults still need ordinary dark calibration during stacking.