fine scale banding from QHY168C

[old_eyes]

My last imaging session collected data from M42 using my relatively new QHY168c OSC (I'm still getting used to its characteristics). After doing all the preprocessing in Pixinsight, integrating and using HDR composition to combine different exposures, I found some fine horizontal banding that I could not easily get rid of.

Poking around I can see that the banding is visible in the original light frames.

Here is a single light frame with 120 sec exposure. No calibration of any kind, just super-aggressively stretched to make the banding visible:

I have compressed it from full size, so just to prove it is not a processing artifact, here is a crop from the full size image:

Has anyone seen this before, or have any ideas what the source might be? It is not something odd with flats. darks or bias, as I didn't use any. Could it be noise somewhere, or a dodgy connection?

All suggestions gratefully received.

 

[vlaiv]

Looks like offset issue to me.

Can you take one of your darks and look at its histogram? There might be clipping to the left. If there is, or if histogram is too close to minimum value - can you increase offset in your camera driver options?

[carastro]

Many years ago I had a QHY camera and there is an option for fast download, which caused banding.  Do you have fast download switched on at all?

Carole 

[Craney]

How is it powered  ?     I have an QHY8pro and now have to power it on a 12V DC battery as opposed to a mains AC 12v supply due to some form of cross over RF interference or something not being properly earthed.     Maybe try a separate power source  (??)

[vlaiv]

Fast download and power could be cause of issues with CCD sensor - above is CMOS and should not be susceptible to either.

It is designed to give really fast download rates - like 30fps, and it does not have separate ADC that can suffer from power fluctuations - each pixel has its own ADC unit.

[old_eyes]

19 hours ago, vlaiv said:

Looks like offset issue to me.

Can you take one of your darks and look at its histogram? There might be clipping to the left. If there is, or if histogram is too close to minimum value - can you increase offset in your camera driver options?

Thanks for the suggestion I will take a look at one of the darks when I have a moment. Everything in the driver is currently left at default settings. I will check out whether I can offset and whether it makes any difference.

If I find a problem in the darks, then I don't even need a clear sky to see the effect of moving the offset. I am assuming that offset will be a visible setting in the ASCOM driver.

[old_eyes]

19 hours ago, carastro said:

Many years ago I had a QHY camera and there is an option for fast download, which caused banding.  Do you have fast download switched on at all?

Carole 

Hi Carole,

I will check whether there is a fast download option. Although the camera is fitted with USB3, I now use USB2 as that is what is available with the internal wiring of the CEM60EC, and I found that the pier mounted mini-PC I was using did not give reliable results with USB3. USB2 was much more stable and as a result I actually got faster downloads across a session and fewer framing errors.

[old_eyes]

18 hours ago, Craney said:

How is it powered  ?     I have an QHY8pro and now have to power it on a 12V DC battery as opposed to a mains AC 12v supply due to some form of cross over RF interference or something not being properly earthed.     Maybe try a separate power source  (??)

Since I have a battery on standby at the observatory this is an easy test to carry out and I will try it when I next have a chance. I agree with @vlaiv that it should not make a difference, but it will only take a couple of minutes to test.

[vlaiv]

2 minutes ago, old_eyes said:

Thanks for the suggestion I will take a look at one of the darks when I have a moment. Everything in the driver is currently left at default settings. I will check out whether I can offset and whether it makes any difference.

If I find a problem in the darks, then I don't even need a clear sky to see the effect of moving the offset. I am assuming that offset will be a visible setting in the ASCOM driver.

Yes it should be if it is adjustable. I know that ASI cameras have that - not sure if QHY ones do as well.

You will be able to test your darks for proper offset without shooting other subs and if you still have raw darks from that session - you can test if it is affected by wrong offset.

You won't be able to fix that session though. Changing offset requires redoing your calibration subs and darks taken at different offset (one that is not affected) - will not match lights. If offset is issue - lights are "affected" as well (not really affected because LP provides offset and there is great chance you won't have clipping in lights but if matching darks are affected - it will create artifact regardless).

[old_eyes]

Thanks for the guidance. I will look into it. 

Yes I still have the raw dark files and will check those out (although they were taken on a different day to the lights I used to show the effect - I need to check out some of the other imaging runs to see if it is always there).

Not worried about fixing the data with the problem. Non-astrophotography friends and family are still blown away by the images I have produced (they are not as critical as we are), and I suspect that when it is printed out rather than shown on a screen the effect will disappear into the background. But I know it's there and I wish it wasn't!

WOuld be nice to make the problem vanish so that I don't have to worry about it in future imaging sesssions!

[old_eyes]

On 04/01/2020 at 20:11, vlaiv said:

Looks like offset issue to me.

Can you take one of your darks and look at its histogram? There might be clipping to the left. If there is, or if histogram is too close to minimum value - can you increase offset in your camera driver options?

I checked the camera settings and they were different to the QHY recommendations for this camera. Histogram was possibly clipping.  Now set Gain 10, Offset 50. This has not yet solved the problem.

On 04/01/2020 at 20:13, carastro said:

Many years ago I had a QHY camera and there is an option for fast download, which caused banding.  Do you have fast download switched on at all?

Carole 

Fast download not an option.

I also replaced the supplied USB cable with a shorter (0.5m) gold contact cable. No change.

On 04/01/2020 at 21:07, Craney said:

How is it powered  ?     I have an QHY8pro and now have to power it on a 12V DC battery as opposed to a mains AC 12v supply due to some form of cross over RF interference or something not being properly earthed.     Maybe try a separate power source  (??)

Now this is interesting. On battery power the banding is not visible (If you strain you can maybe detect something, but is is so far down in the background that I am not sure it is really there and not my brain looking for a pattern I think might be there). However, it is not always visible when running of the 12V supply either.

Not sure why this should be. It could be the power supply  (a Nevada 30A switch mode), or it could be something connected to the same supply that is interfering. Could the pulse width modulation on the dew heater be doing it?

I would rather stick with the same supply as it enables me to use the internal wiring of the CEM60 to keep everything neat. I can try putting a couple of capacitors across the supply to the camera - say 470 microfarad and 0.1 microfarad. And see if that helps.

I also have another regulated 12V power supply that I can try to see if it fixes the problem. If that works, at least I can run another 12V cable from that through the conduit to the pier and avoid having large 12V batteries lying around.

More tests to carry out, but not tonight as it is relatively clear and I would rather collect more data. even if the quality is not great.

[Adam J]

On 20/01/2020 at 20:14, old_eyes said:

I checked the camera settings and they were different to the QHY recommendations for this camera. Histogram was possibly clipping.  Now set Gain 10, Offset 50. This has not yet solved the problem.

Fast download not an option.

I also replaced the supplied USB cable with a shorter (0.5m) gold contact cable. No change.

Now this is interesting. On battery power the banding is not visible (If you strain you can maybe detect something, but is is so far down in the background that I am not sure it is really there and not my brain looking for a pattern I think might be there). However, it is not always visible when running of the 12V supply either.

Not sure why this should be. It could be the power supply  (a Nevada 30A switch mode), or it could be something connected to the same supply that is interfering. Could the pulse width modulation on the dew heater be doing it?

I would rather stick with the same supply as it enables me to use the internal wiring of the CEM60 to keep everything neat. I can try putting a couple of capacitors across the supply to the camera - say 470 microfarad and 0.1 microfarad. And see if that helps.

I also have another regulated 12V power supply that I can try to see if it fixes the problem. If that works, at least I can run another 12V cable from that through the conduit to the pier and avoid having large 12V batteries lying around.

More tests to carry out, but not tonight as it is relatively clear and I would rather collect more data. even if the quality is not great.

You could get problems like that if you are pushing the power supply past its max rating. In any case some sort of power supply issue.

[old_eyes]

9 hours ago, Adam J said:

You could get problems like that if you are pushing the power supply past its max rating. In any case some sort of power supply issue.

Thanks for the suggestion.

Power supply well within ratings. 30A supply, drawing 3 A according to meter. But does look like some sort of power supply or interference issue. On pure battery power, the banding is very slight, but it is there.

It has been suggested that banding USA feature of CMOS detectors, but I haven’t chased that idea down yet.

[old_eyes]

Well I seem to have found out at least some of the problem.

QHYCCD say that mild banding is often found on CMOS sensors, but that it usually averages out when integrating multiple frames. I did not find this to be true. So I started methodically working through every step of the processing sequence trying to understand where the banding became bad.

It appeared to be a problem with the flats (which I must confess Ihad not looked at closely). When I debayered the master flat there was clear horizontal banding.

I was using the Flats Wizard in NINA to collect the flats and matching darks flats (I had read somewhere that CMOS cameras benefit from dark flats and so have been taking them. It doesn't add much time and NINA does it automatically). Looking at the exposures, they were really short, suggesting that my EL panel was too bright.

So I did two calibration and integration runs in Pixinsight on 62 x 120 subs of the Rosette nebula. One with flats and one without.

Here are clips from the integrated frames just autostretched in PI:

Top with flats, bottom without. Banding gone. Now I have horrible gradients and dust bunnies, but no banding.

OK, let's increase the flat exposure and see if it helps. Still using the wizard, but adding a couple of sheets of white paper.

Flat subs look OK, but when integrated the banding is there. Not as bad, but still there. Look at the dark flats and lo and behold, that is where my banding is coming from.

Switch to a conventional 120 sec master dark and everything works. See image below:

So the evidence seems to be that my camera does not like to take short darks. Short flats look OK, but short darks show this banding. When calibrated and integrated it gets into the master flat and causes all kinds of trouble.

Pixinsight with its calibrate and optimise options for darks in ImageCalibration seems to cope quite happily with the different exposures for flats and darks, so I am not sure why some people say you need matched dark flats to get good flats.

Of course it does not explain the milder banding I saw on the single frame used in the original post. That could be wyat QHYCCD were referrring to as typical CMOS banding that averaged out. I thought it was just getting worse with integration, but that seems to have been the flats issue.

This experiment was done with a master dark prepared about 6 weeks ago. What I have not yet done is prepared new 120 sec darks in case something has changed in the camera. Will do that tonight.

Lesson learned? When using a colour camera look at everything debayered. Although PI will use the raw master dark, bias and flat, you won't see the defects easily if you do not debayer and autostretch.

[old_eyes]

Some further progress and a surprising discovery!

Continued to test out various ideas.

On a cloudy night I ran a new set of darks at 120 sec (typical exposure for me with CMOS camera). Oops! distinct banding. Think back to when I had taken the set of darks with no banding. What had changed? Aha,  I had started using the CEM60 built in hub to minimise cable runs.

OK - let's go back and try a few things. 

I had not worried about using the USB2 hub as I previously found that the USB3 port on my Minix Z83-4 was unreliable and tended to hang or misframe subs

So start trying different connections. USB3 cable plugged directly into USB2 socket on PC - less banding but still some problems. Plug into USB3 socket on PC - virtually no banding at all! Plus the bus seems stable and reading the QHY168C steadily and accurately. What gives?

So last trial was a master dark created from 100 x 120 sec subs at -20C. Gain 10 and Offset 50. Looks and works fine. No subs rejected because of read problems.

As far as I know, no driver updates. Nothing except usual Win 10 updates. No idea what this means, except I seem for the moment to have found a workaround.

Recipe seems to be:

• Longer flat exposures
• Use long dark frame exposures and rely on Pixinsight to optimise for flats.
• Original USB3 cable as delivered with camera directly plugged into USB3 socket on mini-PC

If this turns out to be a stable solution I will probably use the trick of taking the polarscope out (I have a PoleMaster) and running a separate USB3 cable up through the mount to the saddle. MAy also need to consider a different mini-PC

Of course this could all be just temporary and the issue could return. The pleasures of a technically complex hobby!

[old_eyes]

Possibly the final comment on this thread.

The last test I did was to compare darks using the shared 12V 30A Nevada switching power supply (runs mount, focuser, and dewbands as well as main cmaera) with darks using the same configuration except that the main camera was powered via a separate 12V 5A linear power supply.

Master darks constructed from 50 x 120 sec exposures. Camera connected with USB3 cable direct from camera to USB3 socket on mini-PC.

This was all done on a cloudy night so as not to waste imaging time 😉.

Noise evaluation on the two masters showed noise figures only 0.5% apart. Putting the two through the subframe selector script also gave virtually identical noise figures.

I then used wavelet extraction to look at fixed pattern noise in the two master darks.

The image shows a side by side comparison at 1:1 scale of the central portion of the master darks. This is the wavelet layer 3, which gave the best contrast for the slight patterns. Right - Switch mode power supply. Left - linear power supply.

The banding is very slightly more obvious with the switched mode supply, but I think it is probably trivial.

So I think I have a route to reliable darks with no horizontal banding. That means good flats and fewr problems when calibrating lights.

Phew!

[old_eyes]

Sometimes when you finally solve a problem it is just plain embarassing.

What is the first thing to check when some computerised gear is behaving oddly? Check the cables! So I did swap the USB cables and I did go direct to the computer USB port (all of them) instead of the internal CEM60EC hub.

Which cable did I not swap, because it is so simple? The power cable. Switched to a different power cable for the camera and the last of the banding vanished. 

I think I am still better off with all the other things I found that made it better, but the root problem was a noisy power cable.

Sometimes I think this hobby hates me!