CMOS OSC Flats Look Wrong

[kirkster501]

Hi all,

Been doing some captures with my OSC CMOS QHY268C.  This is my first CMOS deep sky and I am impressed with it.   All going well apart from one problem. 

My flats look bonkers....

You can see the ADU is clearly 27000.  So why is the screen completely white and not a "flat" ???  The ADU is 27000 on all the individual flat files.  Why is SGP showing the capture at only 27000 totally whited out?

The resultant master flat is not calibrating out the vignetting properly on the lights.  Look at the master flat above.  It is incorrect representation of the true amount of vignetting

The only way round this is to De CFA the master flat, linear fit all components and remerge the four channels again.  When I do that I get this, which looks much more like a flat and it works much better.  However having to do this rules out the use of the batch pre-processing script.

Any thoughts as to what is going on please folks?

Regards, Steve

[Davey-T]

Hi Steve, glad I'm not the only one, also trying to get to grips with OSC processing and having flats problems same value as yours and they're overcorrecting giving me light edges and dark middles.

Just about to take some more flats, also can't get my head around this gain setting, using the ASIAIR set it to medium gain but subs are showing zero gain

Maybe just stretched as long as average value is right it it should work

Dave

Edited October 12, 2020 by Davey-T

[ollypenrice]

Regardng the whited-out flat in SGP, is it being given a screen stretch here? That would explain its having 27K ADU while looking white. If you save it as a TIFF and open it unstretched in PI, what does it look like?

Did you shoot dedicated darks for flats? The CCD trick of using a master bias as a universal flat dark won't work, but I think you know this. Asking just in case.

Olly

Edited October 12, 2020 by ollypenrice
Typo

[kirkster501]

This is a display issue I think with SGP and CFA images with a brighter object.  Obviously a real image is very much fainter than a flats panel.  I'm looking into it.

The individual flats are grey - as should be expected - but they look wrong and not like a flat.  Unless I am mistaken and the vignetting is less than I think.  Scope is a FSQ85 without reducer so does have a wide, flst field.... ?

[kirkster501]

.... they don't look "enough" like a flat although there is a fractional amount of the vignet towards the corners.  But I expected a lot more.  Yes Olly, already doing a stretch but a very low one.

We'll work it out Dave! , It is indeed somewhat different preprocessing CMOS than CCD.

 

[ollypenrice]

2 minutes ago, kirkster501 said:

This is a display issue I think with SGP and CFA images with a brighter object.  Obviously a real image is very much fainter than a flats panel.  I'm looking into it.

The individual flats are grey - as should be expected - but they look wrong and not like a flat.  Unless I am mistaken and the vignetting is less than I think.  Scope is a FSQ85 without reducer so does have a wide, flst field.... ?

Don't expect to be able to judge by eye the fall-off in a linear flat. It's really not a big fall-off. The most extreme vignetting I deal with involves a fall off of 23%. Something like 10% is far more common and doesn't look like much until its stretched. The best thing would be to measure the ADU values in the corners of the flats and in the middle to get an idea of the real fall-off in the corners.

Olly

[kirkster501]

Thanks Olly. Yes, maybe I am expecting to see too much of a fall off.  I have measured it and it is about 20% even though it does not look like that here.

Also, should not stretch the linear flat at all for display purposes.  The histogram is not as the left anyway with a flat frame!  My bad.

 

[Davey-T]

Just had a look at mine in Maxim and they're black but showing 24654 average so just the way its stretched by default, not that mine work anyway 😂

Dave

Just the phone giving the lighter centre actually jet black all over.

 

[ollypenrice]

8 minutes ago, kirkster501 said:

Thanks Olly. Yes, maybe I am expecting to see too much of a fall off.  I have measured it and it is about 20% even though it does not look like that here.

Also, should not stretch the linear flat at all for display purposes.  The histogram is not as the left anyway with a flat frame!  My bad.

 

Yup, I just measured a screen grab of your flat in Ps (Curves, sampling 5x5 average) and got this, which agrees with you quite well.

On the face of it that should be a perfect flat but if it's over or under correcting then it's not right. Can we check that you made your master flats using dedicated darks for flats? If not you'd expect over-correcting.

Olly

Edited October 12, 2020 by ollypenrice

[kirkster501]

I am going to try again Olly but finished my lunch now so back at work    I will look again later on.  Thanks mate.  Are the numbers above with the first flat or the "CFA flat" ?

 

[ollypenrice]

27 minutes ago, kirkster501 said:

This is a display issue I think with SGP and CFA images with a brighter object.  Obviously a real image is very much fainter than a flats panel.  I'm looking into it.

The individual flats are grey - as should be expected - but they look wrong and not like a flat.  Unless I am mistaken and the vignetting is less than I think.  Scope is a FSQ85 without reducer so does have a wide, flst field.... ?

This one, (I think!)

Olly

[tomato]

And as flats go I am extremely impressed with the cleanliness of your optical train, not a dust mote in sight.

[Laurin Dave]

Steve, Dave..  have you looked at the ADUs by channel and are they similar?  reason for asking is that with my LED panel and DSLR the channels are very different, with Blue at 30k or so Green would be 20k and Red about 5k (if I remember correctly)  also the curvature of the flats from vignetting is very different between the channels and they don't flatten each other (whether they should I don't know...  but my LRGB ones do) ..   they don't work and produce reverse vignetting..  Although not sure I attributed this to the large variation in ADU between channels and stopped using flats, cropping instead.    A friend also had the same issue with his ASI294 and an LED panel...  Following the thread as I'd like to know the answer, maybe @Allinthehead can shed some light.

Dave

[kirkster501]

I used the SplitCFA process to split the master flat into the four channels, linear fitted them to the weakest channel then reassembled them in MergeCFA.  It gives a more "flatlike" flat I found as per my top post because this then equalises the intensity of the different types of light from the flat panel. 

Edited October 12, 2020 by kirkster501

[kirkster501]

....I'm very much learning CMOS OSC processing but making progress, little step by little step.

[Davey-T]

1 hour ago, Laurin Dave said:

Steve, Dave..  have you looked at the ADUs by channel and are they similar?

Not yet, this is flat opened in CCD Stack as it does a more sensible job of displaying than Maxim

And stacked image from APP, ignore the eggy stars wasn't guiding

 

 

Edited October 12, 2020 by Davey-T

[symmetal]

When exposing for flats you need to display the full histogram, especially with OSC, so that you see the 3 flat 'bumps' for the three colours and that none of them are near the edges. I just aim to get them centred in the middle of the histogram. With SGP only the unstretched (top) histogram is relevant. The stretched histogram, and the corresponding stretched preview image just seems to display the brightest bump only so doesn't mean much. Though the image does show the top bump stretched correctly. Not sure why kirkster's is all white though as the image statistics don't indicate any clipping.

I use Astroart for calibrating (as Olly recommended it and it's just so easy to use 😀) and when I first tried the ASI071 OSC, I calibrated just the same as a mono image and then selected the RGB demosaic option from the Color menu to get the colour image. This produced an awful result with severe colour patches over the image and a residual grid of lines from the bayer mask. 

I then noticed there was a debayer option in the calibration setup panel, and selecting that and the corresponding bayer pattern (best done first with trial and error on a daytime image, as the Astroart debayer 1 of 4 pattern selection icons don't seem to correpond with reality 😁) I thankfully ended up with a colour image that looked good with no funny colour patches or superimposed grid. Not sure what the difference is between the two methods but it works.

Alan

Edit: Thinking about it the first method registers and stacks the frames before debayering which would certainly cause problems. 😁

Edited October 12, 2020 by symmetal

[Davey-T]

51 minutes ago, symmetal said:

When exposing for flats you need to display the full histogram, especially with OSC, so that you see the 3 flat 'bumps' for the three colours and that none of them are near the edges.

Get this in CCD Stack so not good I guess.

Dave

[symmetal]

That's not as bad as it looks Dave. You have a double peak for each colour and a shallow slope to the left which implies that the vignetting is not very symmetrical about the centre of the image for some reason. The peaks for the two rightmost colours are closer together so their individual double peaks have merged together. The whole histogram is well away from being clipped at the edges so it should calibrate correctly.

This assumes that your light source is even over it's whole area. You could rotate your light box by 90 and 180 degrees and/or shift it left or right so the camera is viewing a different area, and see if the flats still have the same shape histogram. If they don't then the flat box illumination isn't flat so will cause calibration problems with coloured patches on your images. 

Alan

[kirkster501]

Very quick and dirty process.

I did all the preprocessing manually and a quick stretch and colour boost.  But it looks OK and no vignetting.

[kirkster501]

..so as Olly said, the flats were OK.  I was expecting them to show much more vignetting than they do.  Just shows how wide the flat field is on the FSQ85.

Edited October 12, 2020 by kirkster501

[ollypenrice]

25 minutes ago, kirkster501 said:

..so as Olly said, the flats were OK.  I was expecting them to show much more vignetting that they do.  Just shows how wide the flat field is on the FSQ85.

Beware, this isn't showing how flat the field is, it's showing how even is the illumination. These are not at all the same thing. You may still find considerable field distortion on full frame with the 85. I certainly did on mine, as did a friend with his. (Elongated, comet-like stars at the long ends of the chip.) Nor do I think that a twenty-ish percent dropoff in illumination is all that good, really. It's fixable with flats but less would be nice. Compare this with full frame illumination on your TEC with flattener. Different world, but the TEC/flattener was designed to cover medium format film so 35mm is child's play.

Olly

[Allinthehead]

1 hour ago, kirkster501 said:

Very quick and dirty process.

I did all the preprocessing manually and a quick stretch and colour boost.  But it looks OK and no vignetting.

Nice, I did find with my Asi2600 that I needed the flats to be taken at the same temperature as the lights and flat darks. I never needed to do this with my other cameras.

[kirkster501]

I've not finished the pic as yet, it was just a quick blast at a process in light of my earlier issues.

Acquiring this data has proved extremely frustrating.  At the moment it is a total of about 45 x 180s subs, acquired in five or six sub separate occasions through the clouds.  It just keep clouding over all the time.  It's out again capturing now to add to this 45 count hopefully before it either clouds over (again) or disappears over my roof at about 11 O'clock.

[kirkster501]

...clouded over after four subs this time 🙄  Boy you need patience to be an astronomer in the UK......