QHY10 color CCD purple borders ??

[jarbi]

Hi,

I am trying to figure out what is going on with my color CCD camera. As you see in the debayered and stretched frame attached, I get a purple border on the resulting image. The camera is calibrated for offset, I use 0 gain. 

This purple border is no showing up on the stretched flats, darks or Bias frames, just on the light frames. I am completely puzzled with this.. This cannot be vignetting as I see because the borders are square,what els can I do to narrow down the reason ?

 Any hint would be appreciated,

Clear skies,
Janos

[Thalestris24]

Hi

I'm guessing a bit but could it be condensation or ice on the sensor? That wouldn't show on bias or dark frames though you'd expect it on the flats - if you took them at the same time?

Louise

[vlaiv]

What is your calibration routine?

One thing that might produce such effect is Flat frame, not vignetting part (lack of light due to obstruction) but rather chip sensitivity part - it might be that certain regions of chip have different sensitivity due to manufacturing (it can be related to micro lenses, or amp in cmos chips, or stuff like that).

Take a look at this image (attached) - that is R, G and B components of one of my master flats (256 frames stacked) - debayered into R, G and B - it is OSC camera - ASI178mmc - but it clearly shows that besides vignetting part there are also some artifacts that are parallel to edges - like strips that are darker or lighter.

So the issue there can be related to those, but would also mean that something is wrong with your calibration routine (altering master flat in some way before applying it to lights, or something like that).

[Thalestris24]

2 minutes ago, vlaiv said:

What is your calibration routine?

One thing that might produce such effect is Flat frame, not vignetting part (lack of light due to obstruction) but rather chip sensitivity part - it might be that certain regions of chip have different sensitivity due to manufacturing (it can be related to micro lenses, or amp in cmos chips, or stuff like that).

Take a look at this image (attached) - that is R, G and B components of one of my master flats (256 frames stacked) - debayered into R, G and B - it is OSC camera - ASI178mmc - but it clearly shows that besides vignetting part there are also some artifacts that are parallel to edges - like strips that are darker or lighter.

So the issue there can be related to those, but would also mean that something is wrong with your calibration routine (altering master flat in some way before applying it to lights, or something like that).

I'd have suggested a flats problem too  but had the impression that the OP had posted a single light but I might be wrong!

Louise

[vlaiv]

Just now, Thalestris24 said:

I'd have suggested a flats problem too  but had the impression that the OP had posted a single light but I might be wrong!

Louise

I was under the impression that it was calibrated / debayered single frame, but now reading the post again, I'm not sure

[Thalestris24]

1 minute ago, vlaiv said:

I was under the impression that it was calibrated / debayered single frame, but now reading the post again, I'm not sure

Ah, I think it's actually a stack of 10 lights - Janos posted this image last year on the qhy forum... I think he may need to take longer exposure flats/biases?

Louise

[vlaiv]

2 minutes ago, Thalestris24 said:

Ah, I think it's actually a stack of 10 lights - Janos posted this image last year on the qhy forum... I think he may need to take longer exposure flats/biases?

Louise

I believe the this one is single frame - it looks more noisy / less detail then stack of 10 on qhy forum. And it looks like it's calibrated - if stack looks exactly like that - but less noise / more detail - it is produced out of calibrated frames.

Now having read the thread you found, I'm getting the feeling that indeed calibration / stacking procedure is the issue here - it says something about bias frame cutting of left hand side of histogram - I could relate this to using calibration with low precision math / only positive numbers (so when subtracting bias everything gets clipped at 0 if no negative values are calculated / allowed by software).

[jarbi]

8 hours ago, vlaiv said:

I believe the this one is single frame - it looks more noisy / less detail then stack of 10 on qhy forum. And it looks like it's calibrated - if stack looks exactly like that - but less noise / more detail - it is produced out of calibrated frames.

Now having read the thread you found, I'm getting the feeling that indeed calibration / stacking procedure is the issue here - it says something about bias frame cutting of left hand side of histogram - I could relate this to using calibration with low precision math / only positive numbers (so when subtracting bias everything gets clipped at 0 if no negative values are calculated / allowed by software).

Thanks for your replies, nice to see so many people ready to help ;-). So, it is indeed one single frame. It is also true that I have posted first a stack on the QHY forum, but I didn't find a solution yet. I try now to minimize any effect of processing, that is why I posted a single frame. Coming back to the flat question: my flats show some border, but they don't help to eliminate them in the light frames.

I have placed some fits and also debayered and stretched Flat frames here: 

https://drive.google.com/open?id=0B5rc5Ssg9hE-T0prWmxleGh6R00

I can  see some kind of border there but not in purple. These were made with gain =25 and with about 10,20,40 and 80% illumination.

As you see on the histogram of the FIT-s, I get a double "bump", can it cause this problem ?

 

Clear skies,

Janos

 

[jarbi]

I have created now a folder with a single light, flat, bias and dark frame:

https://drive.google.com/open?id=0B5rc5Ssg9hE-VEhnLWMzOGNIQkk

This way you have a complete set.

Coming back to an earlier suggestion of Louise: I don't think it is frost or dew, because it happens with higher sensor temperatures as well. Yesterday evening I have checked the chip after making calibration frames at -25C, an it was not frosted.

 

Cheers,

Janos

[vlaiv]

Ok, I've inspected your frames and indeed flats are the cause of this.

You are using OSC camera, and it has Bayer matrix - meaning 3 different pixel values will be present in your flats, or should I say 3 different histogram peaks should be there - each color concentrating around some value (each color has a bit different sensitivity). It is important that you use the same settings for flats that you used for other frames - meaning offset / gain. Only thing that you should vary is exposure time. Now it is also important to get histogram of your flats so that all three peaks are present. For example in second link you posted, flat frame has histogram clipping to the left  - all Blue (I think it is blue, blue is usually the least sensitive one in OSC) is clipped to the left. You might not notice this, but I think that you did not subtract bias / flat dark from your flat frame, but when you do - you will see that it clips. In first link for example - 80p is clipped to the left (there are only two peaks) - one color is clipping all the way - bunch of 65536 values for pixels - whole peak is slammed to the right on histogram.

Also your flats should be calibrated as well.

One thing that worries me is that your flats show really big level of non linearity. This could mean two things. First you used value of gamma other then 1, or the second thing camera has very non linear response.

What software are you using for capture and what are the settings for it?

Ideally the process should be as follows:

Set gain and offset to some values (gain should be set with respect to duration of your subs - longer subs less gain, so there is no clipping, and offset should be such that there is nice Gaussian in histogram when you take bias frame - so there is no clipping to the left in bias frames, but don't move it far to the right because you will use range). Set temperature as well to certain set point.

Capture bias frames. Capture light frames of certain exposure length. Capture dark frames of the same exposure length. (bias and darks, can be done later if you have CCD and set point cooling, for cmos sensors, depending on sensor, bias frames should be taken at the same time as lights without "reseting" the cmos - changing settings. With cmos sensors sometimes it is useless to take bias frames - for example ASI178mmc does not handle bias frames well because it seems there is some kind of internal bias subtraction in chip itself for long exposure).

Now when taking flats, keep gain, offset, temp settings the same, but find exposure that produces three distinct peaks in histogram with no clipping to the left or to the right. Also it should be as far right as possible. Also to check for linearity. If you find exposure length that produces three peaks - increasing a bit or decreasing a bit should have following effect on histogram - it should just move them left / right on histogram - but not change their relative positions or shapes, if you see that for example "distance" between peaks change when you change exposure length (middle one is closer to the right on some exposure settings and closer to the left one on other exposure settings) - that is not good thing - you either have gamma set to some other value then 1 or there is indeed non linearity in CCD response. This can also be compensated for (non linearity) but it is something that is a bit less straight forward (you would need to create bunch of "flats" at different exposure settings and see how histogram behaves and create curve that is used to calibrate out non linearity).

Now when you have found proper exposure time (three peaks and histogram to the right, but no clipping) - take your flats, and then take Flat darks (this is something you should do if you have longer exposure for flats and there is significant amount of dark current).

Ok, so when all of this is done - calibration goes like this:

Create master bias

Create master dark (stacked (dark-master bias))

Create master dark flat (stacked (dark flat -master bias))

Create master flat (stacked (flat - master bias - master dark flat)

And then calibrate your lights as  (light - master bias - master dark) / master flat

All of this should be done in raw without debayering any of the frames.

Then you have your calibrated lights.

Depending on software that you are using this can be automated process, I for example like to do my own calibration using ImageJ - this way I can process all frames in 32 bit format without loosing any precision (DSS for example does a lot of preprocessing / calibration in 16 bit, and if one is shooting a lot of frames - cmos + heavy LP kind of setup - you loose quite much by not doing full 32 bit precision math).

Hope this helps!

[Thalestris24]

Just to add, my qhy8l has an optimum gain and offset which you set and never normally change. Not sure if that applies the qhy10 but thought I'd mention (actually I just checked their manual and you do have to set them by the looks of it - see p22). Qhy have posted this re the qhy10 - can't say if it's relevant but the purple is suspiciously like yours. It might take some time for the cooled temperature to get stable so that's something to bear in mind particularly with calibration frames.

Um make sure you are using low speed downloads for image and calibration frame acquisition. Also, it looks like you have to expose for at least 9 secs in order to get a one whole frame. Not sure what the implication will be when taking bias/flat frames.

If in doubt - read the manual, thoroughly!

Louise

[jarbi]

Hi vlaiv and Louise,

Thank you for your advises, I need some time to digest your detailed answers and I will come to a solution soon.

Vlaiv, I would like to respond to some of your questions already. First about the clipping: I have attached the histogram of the debayered flatfield from my test set in my previous reply. As you see there all 3 colors are present, red is much weaker though. Now I know a possible reason for it: my idas LPS filter was in the imaging train when I made the flats, and that is doing some serious clipping in red I believe. I live in a very light polluted suburban place ( mag 18 "skies" )

I take my frames always in Sequence Generator Pro, so whenever a non linearity would be present it is caused by the camera I would say.  My offset was calibrated as in the QHY manual, with about 700 ADU mean pixel value on the BIAS.

I am familiar with the calibration process you describe ( you may have a look at my photo's so far on Astrobin ), but with this camera I couldn't get along very well so far.

I think you nailed it with your remark about the clipping on the flats ! My next move will be to take flats without the Idas filter with the same white flat panel, I should have thought about that earlier... I will come back here with hopefully good results. But until this weekend I will not have much time to work on it, working in IT is not a nine-to-five job mostly ;-),

 

Clear skies,

Janos

 

 

[Thalestris24]

Hi

I think the colours in the flats matter less than how you are taking them exactly? Your flats should obviously be taken with the exact and undisturbed same imaging train as your lights

Louise

[jarbi]

44 minutes ago, Thalestris24 said:

Hi

I think the colours in the flats matter less than how you are taking them exactly? Your flats should obviously be taken with the exact and undisturbed same imaging train as your lights

Louise

Hi Louise,

My one shot color camera is certainly more picky on this as I experience. Seeing the histogram of my flats, the red channel is falling out of the linear sensitivity range of my CCD, because it is much weaker than 30% of my full well capacity ( see histogram above ). Vlaiv has right,all three colors must be in the linear range to make good flats. But: with the idas filter attached it's impossible, because green and blue are getting saturated when red comes into the linear range.

I will make new flats without the idas filter and it will prove or disprove my reasoning soon,

 

Clear skies,

Janos

 

[Thalestris24]

41 minutes ago, jarbi said:

Hi Louise,

My one shot color camera is certainly more picky on this as I experience. Seeing the histogram of my flats, the red channel is falling out of the linear sensitivity range of my CCD, because it is much weaker than 30% of my full well capacity ( see histogram above ). Vlaiv has right,all three colors must be in the linear range to make good flats. But: with the idas filter attached it's impossible, because green and blue are getting saturated when red comes into the linear range.

I will make new flats without the idas filter and it will prove or disprove my reasoning soon,

 

Clear skies,

Janos

 

How exactly are you taking your flats - are you using a led panel? They are often short on red light. I find taking flats using just room light is better. I do all my stacking with dss which converts flats to greyscale anyway maybe you could try it.

Louise

ps there's another thread discussing sgp flats, if it's of any interest: 

 

[jarbi]

10 hours ago, Thalestris24 said:

How exactly are you taking your flats - are you using a led panel? They are often short on red light. I find taking flats using just room light is better. I do all my stacking with dss which converts flats to greyscale anyway maybe you could try it.

Louise

ps there's another thread discussing sgp flats, if it's of any interest: 

 

Hi Louise,

Thanks for your suggestion. I use a LED panel mostly, but I will try with room light next time and I will check out the thread you mentioned as well,

Cheers,

Janos