New camera flats help…pretty please?

[assouptro]

Hi Starpeople!

I’ve been imaging for years and managed the upgrade from dslr to ccd but now I’m using my new (to me anyway) qhy268m and I’m struggling with flats 

I used Nina flat wizard (for the first time)  and the light panel I have used with ccd for a couple of years now but when I integrated my first light in app I was shocked by the flat masters that have ruined the stacked images 

what am I doing wrong? 

i

I used the same temperature as the lights, same gain and offset  but I’m baffled by the results 

I’d appreciate any help! 
 

many thanks in advance 

Bryan

single frame Ha

master ha flat

 

results 

 

Edited October 23, 2022 by assouptro

[Craig a]

The reversed vignette in the resulting image suggests that your flats are too bright and looking at the histogram on the flat itself looks like it’s too bright, it’s clipped on the right hand side or am I looking at this wrong? 
whst duration we’re your flats? Did you take any dark flats or use bias to calibrate the flats?

 

Edited October 21, 2022 by Craig a

[assouptro]

54 minutes ago, Craig a said:

The reversed vignette in the resulting image suggests that your flats are too bright and looking at the histogram on the flat itself looks like it’s too bright, it’s clipped on the right hand side or am I looking at this wrong? 
whst duration we’re your flats? Did you take any dark flats or use bias to calibrate the flats?

 

Hi Craig a

Thanks for the reply

 

The integration time for the flat was 1.19s and there was a corresponding dark flat 

It’s a Samyang 135mm lens at f2.8 

The flat panel was barely on, almost so dim I couldn’t see it 

the flat darks are really smooth by comparison, no banding at all to my eye 

Cheers 

Bryan

[assouptro]

Hi again

I have tried to do research on this and haven’t found much info other than I am possibly not getting above the noise threshold in mode 0 at gain 26? 
I am totally new to cmos and as usual with this hobby there is a learning curve 

As the camera is second hand, I’m more concerned there may be a problem with it? 

I’ve tried calibrating with dark flats and bias but the results are the same but without flats pictures look good? 

No flats:

 

flats and dark flats:

flats and bias :

Any help would be greatly appreciated 

Cheers 

Bryan

[newbie alert]

I'm no CMOS guru but should you be using bias?

Never used the flat wizard but does it give you a Adu option

In Apt it gives you a Abu rage to aim for with a 5% tollerance, and if it can't adjust the exposure length it flags up a error to say your flat panel is too bright.. it simply runs through the filters and saves the result

Also what scope are you using with the 268? 

 

 

[assouptro]

13 minutes ago, newbie alert said:

I'm no CMOS guru but should you be using bias?

Never used the flat wizard but does it give you a Adu option

In Apt it gives you a Abu rage to aim for with a 5% tollerance, and if it can't adjust the exposure length it flags up a error to say your flat panel is too bright.. it simply runs through the filters and saves the result

Also what scope are you using with the 268? 

 

 

Thanks for the reply 

“scope” is a Samyang 135 lens 

there is an adu setting percentage and I aimed for 35% which was around 22000adu, this was achieved by adjusting the image time and adjusting the brightness of the panel (I have a dimmer on it) tolerance is 10% by default 

Regarding Bias, I originally didn’t bother but I have since read that bias can reduce the cameras fixed pattern noise so I thought I’d try!

Cheers 

[newbie alert]

 I've read that it's a big no no..

https://www.astropixelprocessor.com/community/main-forum/generation-of-calibration-frames-and-use-of-bias-frames-for-cmos/

[newbie alert]

Calibration frames will add noise if not used correctly 

[assouptro]

42 minutes ago, newbie alert said:

 I've read that it's a big no no..

https://www.astropixelprocessor.com/community/main-forum/generation-of-calibration-frames-and-use-of-bias-frames-for-cmos/

Thanks, that’s useful but it doesn’t solve my initial issue of the flat frames 

My first attempt didn’t include bias frames and I wasn’t intending taking and using bias, just flats and dark flats 

The results were badly affected by the flats though which is when I posed my question here and started doing my own research where I learned that bias frames can be useful with cmos and even though I got a slightly better result it was still the flat frames that caused the issue 

I just need to find out how to take an effective flat with this camera? It takes a long time to process the images compared to my ccd due to the file size and that’s why I’m reaching out for help to avoid the time wasted in calibration attempts! 
I don’t know whether I should be using different settings (mode 0 or 1, gain, offset etc) 

I took some 5 min subs in mode 1 but they were over saturated so settled on mode 0 to avoid this 

Cheers 

Bryan 

[ONIKKINEN]

For things specific to the QHY version of the 571 sensor camera, check out this thread:

But my 2 cents as someone who uses the same sensor but a different brand of camera, bias frames are ok provided that you take a whole lot of them and you have no offset issues = no pixels at 0 in any of the frames.

I dont know how to interpret your histogram in the screnshot as it appears to be in log mode and maybe stretched. But it looks like your left side is completely clipped. I think you would also get heavy readout pattern banding visible on the frames if you had too low an offset where the dark bands are clipped to 0 (= will not be calibrateable)

Can you post 1 of each calibration frame and a matching light fame in .FITS format for us to have a look?

[assouptro]

43 minutes ago, ONIKKINEN said:

For things specific to the QHY version of the 571 sensor camera, check out this thread:

But my 2 cents as someone who uses the same sensor but a different brand of camera, bias frames are ok provided that you take a whole lot of them and you have no offset issues = no pixels at 0 in any of the frames.

I dont know how to interpret your histogram in the screnshot as it appears to be in log mode and maybe stretched. But it looks like your left side is completely clipped. I think you would also get heavy readout pattern banding visible on the frames if you had too low an offset where the dark bands are clipped to 0 (= will not be calibrateable)

Can you post 1 of each calibration frame and a matching light fame in .FITS format for us to have a look?

Thanks 

Here are some files 

I know the flat is at a different temperature but it was taken indoors with a flat panel

(my first flats were 50% adu so I retook them at 35% adu in the hope that would help[)

Cheers 

BryanBias.fitdark_60.fitha_flat.fitha_lights.fit

 

[newbie alert]

1 hour ago, assouptro said:

Thanks, that’s useful but it doesn’t solve my initial issue of the flat frames 

My first attempt didn’t include bias frames and I wasn’t intending taking and using bias, just flats and dark flats 

The results were badly affected by the flats though which is when I posed my question here and started doing my own research where I learned that bias frames can be useful with cmos and even though I got a slightly better result it was still the flat frames that caused the issue 

I just need to find out how to take an effective flat with this camera? It takes a long time to process the images compared to my ccd due to the file size and that’s why I’m reaching out for help to avoid the time wasted in calibration attempts! 
I don’t know whether I should be using different settings (mode 0 or 1, gain, offset etc) 

I took some 5 min subs in mode 1 but they were over saturated so settled on mode 0 to avoid this 

Cheers 

Bryan 

Yes of course the flats are the intended topic and agree that there's an over correction issue there. But you have vertical over correcting lines which I wouldn't think is usual in a flat

[ONIKKINEN]

A number of issues emerge from these, firstly your bias frame has a median ADU value of 30 190? Not sure whats up with that, did you have a light panel on the sensor while taking those? These will completely ruin any hope of calibration if you used them to calibrate your flats.

Your light frame has a median ADU value of 305, and a minimum value of 0. This means your offset is way too low and some pixels are clipped to black. 0 value pixels are uncalibrateable since they cannot be multiplied or divided with, this leads to some (probably minor) calibration issues with those particular clipped pixels.

Your dark frame has a median ADU value of 341, which is higher than your light frame so these will definitely not work. Also a min value of 0 here, offset issues. I am guessing you took the darks with the camera not properly plugged from ambient light? Only way a darkframe can have a higher median ADU value than the light frame is if you had light leaks somewhere. Take the camera off the lens, plug it properly and place it in a dark place like a completely dark room, your fridge (helps with cooling), under some blankets or something. Just a couple of photons creeping through somewhere will ruin your darks like this.

Your flats also have 0 value pixels and appear to be extremely noisy, probably best to try and increase exposure time a bit by reducing your flat panel brightness. Not sure if thats good advice as i do not do narrowband with mine, but seems like it would be a good thing to do. The median value is 4935 while the mean value is 21674. Was this an OSC camera or the mono one? Looks like i can debayer the file so at least Siril thinks its an OSC camera. That would lead to weird pixel values because the greens and blues will not be exposed as much as the reds through the Ha filter. *

TLDR: Increase offset until you dont get 0 value pixels in bias frames taken in complete darkness anymore, fix light leaks with darks and retake them. Flats are weird, not sure about those.

*That was a Siril thing, ignore that.

Edited October 23, 2022 by ONIKKINEN
Fixed something

[assouptro]

38 minutes ago, ONIKKINEN said:

A number of issues emerge from these, firstly your bias frame has a median ADU value of 30 190? Not sure whats up with that, did you have a light panel on the sensor while taking those? These will completely ruin any hope of calibration if you used them to calibrate your flats.

Your light frame has a median ADU value of 305, and a minimum value of 0. This means your offset is way too low and some pixels are clipped to black. 0 value pixels are uncalibrateable since they cannot be multiplied or divided with, this leads to some (probably minor) calibration issues with those particular clipped pixels.

Your dark frame has a median ADU value of 341, which is higher than your light frame so these will definitely not work. Also a min value of 0 here, offset issues. I am guessing you took the darks with the camera not properly plugged from ambient light? Only way a darkframe can have a higher median ADU value than the light frame is if you had light leaks somewhere. Take the camera off the lens, plug it properly and place it in a dark place like a completely dark room, your fridge (helps with cooling), under some blankets or something. Just a couple of photons creeping through somewhere will ruin your darks like this.

Your flats also have 0 value pixels and appear to be extremely noisy, probably best to try and increase exposure time a bit by reducing your flat panel brightness. Not sure if thats good advice as i do not do narrowband with mine, but seems like it would be a good thing to do. The median value is 4935 while the mean value is 21674. Was this an OSC camera or the mono one? Looks like i can debayer the file so at least Siril thinks its an OSC camera. That would lead to weird pixel values because the greens and blues will not be exposed as much as the reds through the Ha filter. *

TLDR: Increase offset until you dont get 0 value pixels in bias frames taken in complete darkness anymore, fix light leaks with darks and retake them. Flats are weird, not sure about those.

*That was a Siril thing, ignore that.

A lot to digest there! 
I really am going back to school with cmos! 
I’m so used to ccd it’s a completely different beast! 
I take everything you say on board it’s just a shame the flats (which are the one component messing up my integration) are the one thing you cannot help me with 😢

I will get to the bottom of this somehow 

Thank you sincerely for your help 

Bryan 😊

[assouptro]

43 minutes ago, newbie alert said:

Yes of course the flats are the intended topic and agree that there's an over correction issue there. But you have vertical over correcting lines which I wouldn't think is usual in a flat

Thanks 

The vertical lines are in the flats. 
Thats the issue! 
I don’t know what I’m doing wrong yet as the flats are taken the same way I’ve been taking them successfully for years with ccd. 
I will overcome this…. Somehow! 😊

Thanks again 

Bryan

[ONIKKINEN]

2 minutes ago, assouptro said:

A lot to digest there! 
I really am going back to school with cmos! 
I’m so used to ccd it’s a completely different beast! 
I take everything you say on board it’s just a shame the flats (which are the one component messing up my integration) are the one thing you cannot help me with 😢

I will get to the bottom of this somehow 

Thank you sincerely for your help 

Bryan 😊

Since you calibrated the flats with the weird bias frames, it could just be that those were the issue and the flats are ok. Give retaking your darks and darkflats a go and see if the flats will work?

[assouptro]

Just now, ONIKKINEN said:

Since you calibrated the flats with the weird bias frames, it could just be that those were the issue and the flats are ok. Give retaking your darks and darkflats a go and see if the flats will work?

I didn’t use bias originally and the effect was worse! 
Bias improved but didn’t eliminate the vertical lines 

the results of integration without the flats are pretty good and I will be able to produce a final image 

I have compared my flats to others taken by this camera and Nina and I haven’t seen anything like the interference produced by my efforts 

[Craig a]

Stab in the dark but Is there a chance there is electrical interference? 
Edit, just read the above findings from your darks and flat so ignore this post them findings will most certainly be the cause of the issue 

Edited October 23, 2022 by Craig a

[assouptro]

2 minutes ago, Craig a said:

Stab in the dark but Is there a chance there is electrical interference? 

Maybe? 
I have tried 2 different power supplies just in case, 

I’m  using a 13.8v 15 amp bench power supply atm 

Unless the flat panel is emitting some emf? 
cheers 

Bryan

[ollypenrice]

If you are using the Samyang at a fast F ratio your dust bunnies may be blurred into insignificance, meaning only your vignetting needs correcting. Gradient removal software is very good at dealing with that. By all means get to the bottom of your flats issue but bear in mind that doing so might not bring a vast benefit.

Olly

[assouptro]

8 minutes ago, ollypenrice said:

If you are using the Samyang at a fast F ratio your dust bunnies may be blurred into insignificance, meaning only your vignetting needs correcting. Gradient removal software is very good at dealing with that. By all means get to the bottom of your flats issue but bear in mind that doing so might not bring a vast benefit.

Olly

Thanks Olly 

appreciate the input! 
I understand what you are saying, and maybe I will have to wait until I use the camera with one of my slower scopes to achieve a successful flat 

Thanks 

Bryan 

[bottletopburly]

Im no expert but i had a banding issue with my led panel(refresh rate was the issue ) the cause was flat exposure wasnt long enough  you need between 3-5 sec  with cmos  have a ok at adam blocks 3 videos ,in  which adam confirms this , and bias is definetley needed , you may need to supreess panel brightness until you find the sweet spot to get 3-5 sec with your  required ADU.

 

[assouptro]

30 minutes ago, bottletopburly said:

Im no expert but i had a banding issue with my led panel(refresh rate was the issue ) the cause was flat exposure wasnt long enough  you need between 3-5 sec  with cmos  have a ok at adam blocks 3 videos ,in  which adam confirms this , and bias is definetley needed , you may need to supreess panel brightness until you find the sweet spot to get 3-5 sec with your  required ADU.

 

Thanks 

I have always used 5s flats with my Atik 383 due to the shutter I intended using longer flats originally but the Samyang lens at f2.8 with the QHY 268m I cannot get the panel brightness low enough! 
I will however pursue this 

Thank you 

Bryan

[bottletopburly]

12 minutes ago, assouptro said:

Thanks 

I have always used 5s flats with my Atik 383 due to the shutter I intended using longer flats originally but the Samyang lens at f2.8 with the QHY 268m I cannot get the panel brightness low enough! 
I will however pursue this 

Thank you 

Bryan

i use a sheet of white grease proof baking paper in front of panel use two if required ,works a treat .

[ONIKKINEN]

Re: Flats

They did seem to have little vignette and no obvious dust, so any smart background removal tool (even the one in Siril) will fix that quite easily. If calibration wont work with this dataset it will still produce a pretty image in the end so definitely not lost this dataset yet.