ASI294MC Flats help. I'm stumped!

[Jamgood]

Hi all.

I got a ZWO ASI294MC Pro recently and I've come to a bit of a stumbling block with it. Using APT, everything connects great, good light frames, plates solves, etc.

Flat frames!!!!! Using APT CCD Flats Aid to create a plan of 0.31937s flats at 25000ADU, (histogram below) I cannot seem to get them to work in a stacked imaged. I'm left with terrible vignetting even after a couple of small stretches of the DSS stacked file. I take flats using a tablet with a white screen. This worked well for my old DSLR but no matter what method I try with it (Uncovered, White T-Shirt, Paper) with the 294MC, the flats seem to under/over compensate, I'm not sure which.

Imaging train is - 294MC Pro > TS-Optics NEWTON Coma Corrector 1.0x GPU > Optolong L-Enhance > Skywatcher 130PDS

The light frames have a little vignetting, as usual, but I can't seem to get a flat frame setting to calibrate it out. Everything I've tried makes it worse.

Stretched Light Frame

Stretched Master Flat 

Stacked Image with a couple of Level stretches.

Am I just being absolutely stupid and missing something obvious or is there something going on that is above my level of knowledge?

All help and advice welcome.

👍

Edited March 25, 2021 by Jamgood

[vineyard]

I'm sure someone will come along & help soon!  That flat does look a bit strange to me - I have the same camera & this is what one of my master flats looks like (stretched STF on PI).

(EDIT PS: weird, same square pattern when I post it as JPG so maybe its not the flat - I don't use APT so can't be of much help sorry!)

Edited March 25, 2021 by vineyard

[david_taurus83]

The square pattern looks like the bayer matrix. It's obvious on the flat but not so on the light. How are you calibrating?

Edited March 25, 2021 by david_taurus83
Spelling

[Jamgood]

1 hour ago, vineyard said:

I'm sure someone will come along & help soon!  That flat does look a bit strange to me - I have the same camera & this is what one of my master flats looks like (stretched STF on PI).

(EDIT PS: weird, same square pattern when I post it as JPG so maybe its not the flat - I don't use APT so can't be of much help sorry!)

Yeah I thought mine looked funny on here but it doesn't look like that on the PC.

1 hour ago, david_taurus83 said:

The square pattern looks like the bayer matrix. It's obvious on the flat but not so on the light. How are you calibrating?

I'm currently in the process of building my dark library as the camera is new but the image above was.....

Deep Sky Stacker > 12 Lights 300s > 10 Darks > 15 Flats > 15 Darks Flats - Fits files stacked with the generic Bayer pattern RGGB.

 

[david_taurus83]

Is that light frame straight out of the camera? It looks like it's been debayered as I can see colour in the brighter stars. The flat looks like it's still bayered. All your dark and flat calibrating should be done on bayered light frames. Once that's done, then you would debayer before moving onto aligning and stacking. 

[Jamgood]

@david_taurus83 Yeah, sadly that's not the problem. The reason the single image is debayered is because I had to run it through DSS before I could open it with Photoshop. I've not found a decent .fits viewer that will save to .jpg for sharing on here yet. GIMP should do it but I can't get it to work at all.

[david_taurus83]

Not sure then. I have had issues with DSS and calibrating DSLR flats. DSS sometimes over/under corrects but if I run them through Pixinsight manually they come out OK. It must be a setting in DSS somewhere. Do you want to Dropbox a set of flats, darks and lights and I cam run them through PI and see what happens?

[vlaiv]

Flat seems to be over correcting.

Since it is light / flat and value is higher than it should be - either light is higher than it should be or flat is lower than it should be - maybe both are happening.

How do you take your darks? Both darks and flat darks?

My guess is that you took flat darks while camera was on scope and front of the scope covered and regular darks you took with camera off the scope.

If that is the case - you might have a light leak. This happens on the back of the scope if there is light near the end of the scope. People deal with this in various ways, for example:

In any case - lights were taken on scope (obviously) and darks off scope - (lights - darks) will have higher value because of additional light from back of the scope.

Flats and flat darks were taken with camera on the scope - here flat darks are higher in value (flats might be also - if so the cancel out) - so resulting value will be smaller.

Hope it makes sense - moral of this story is check if you have light leak on the back of your scope.

Another possibility is IR light that passes thru plastic cover on front of the scope - but that is unlikely if you have UV/IR cut filter.

[Jamgood]

19 minutes ago, david_taurus83 said:

Not sure then. I have had issues with DSS and calibrating DSLR flats. DSS sometimes over/under corrects but if I run them through Pixinsight manually they come out OK. It must be a setting in DSS somewhere. Do you want to Dropbox a set of flats, darks and lights and I cam run them through PI and see what happens?

I'll dropbox then to you later, cheers 👍

[Jamgood]

11 minutes ago, vlaiv said:

Flat seems to be over correcting.

Since it is light / flat and value is higher than it should be - either light is higher than it should be or flat is lower than it should be - maybe both are happening.

How do you take your darks? Both darks and flat darks?

My guess is that you took flat darks while camera was on scope and front of the scope covered and regular darks you took with camera off the scope.

If that is the case - you might have a light leak. This happens on the back of the scope if there is light near the end of the scope. People deal with this in various ways, for example:

In any case - lights were taken on scope (obviously) and darks off scope - (lights - darks) will have higher value because of additional light from back of the scope.

Flats and flat darks were taken with camera on the scope - here flat darks are higher in value (flats might be also - if so the cancel out) - so resulting value will be smaller.

Hope it makes sense - moral of this story is check if you have light leak on the back of your scope.

Another possibility is IR light that passes thru plastic cover on front of the scope - but that is unlikely if you have UV/IR cut filter.

Hi Vlaiv. 

Everything was taken as you said.  I do use a black shower cap over the rear of the scope. Although I'm in Bortle 8 and the moon was half in the sky, I've never had vignetting so bad before. 

Other than the camera, the only things that are new to me are the GPU CC and the L-Enhance filter. I'm wondering if the CC is too close to the end of the focuser near the secondary and too much light is hitting the filter causing reflections. I have another focuser attachment that will pull the focus point out around 15mm so I can try that and see.

The other thing in question is the camera drivers. I originally read it was best to use the ASCOM drivers which I did. Next time it's clear I'll try with the ZWO Native driver and see if there's any difference. 

[vlaiv]

28 minutes ago, Jamgood said:

Hi Vlaiv. 

Everything was taken as you said.  I do use a black shower cap over the rear of the scope. Although I'm in Bortle 8 and the moon was half in the sky, I've never had vignetting so bad before. 

Other than the camera, the only things that are new to me are the GPU CC and the L-Enhance filter. I'm wondering if the CC is too close to the end of the focuser near the secondary and too much light is hitting the filter causing reflections. I have another focuser attachment that will pull the focus point out around 15mm so I can try that and see.

The other thing in question is the camera drivers. I originally read it was best to use the ASCOM drivers which I did. Next time it's clear I'll try with the ZWO Native driver and see if there's any difference. 

Ascom drivers are the way to go really, so I would not change that. Reflection off filter could be the reason - but I think that it would affect both flats and lights so it would sort of cancel out. You should however make sure your light path is clear of any reflections so do try that other attachment.

What was your offset set to? It could be issue with offset as well.

Out of the 4 types of the subs - only flat darks would be affected by low offset in a bad way since all others have additional signal (even darks in long exposure have some dark current, and flat darks tend to be short).

If offset is set too low - that would make flat darks higher in value and in turn master flat lower in value.

 

[Jamgood]

23 minutes ago, vlaiv said:

Ascom drivers are the way to go really, so I would not change that. Reflection off filter could be the reason - but I think that it would affect both flats and lights so it would sort of cancel out. You should however make sure your light path is clear of any reflections so do try that other attachment.

What was your offset set to? It could be issue with offset as well.

Out of the 4 types of the subs - only flat darks would be affected by low offset in a bad way since all others have additional signal (even darks in long exposure have some dark current, and flat darks tend to be short).

If offset is set too low - that would make flat darks higher in value and in turn master flat lower in value.

 

That was my first proper outing with the camera, (I had tested it the night before on the moon, between clouds) using the ASCOM driver and everything at default. 16bit, Unity Gain 120, Offset 30.

With the CC and Filter being near the end of the focuser tube, light could go directly into the focuser tube as well as reflected light but I would think that would make the flats look uneven? Where as they look like decent flats to me. I have a Baader CC which will be nowhere near the end of the focuser so I can also use that for testing when it's clear next.

Ideally though I want to avoid the Baader as in the past I've had nothing but problems with spacing. The GPU I recently acquired gives nice round stars across the image.

Edited March 26, 2021 by Jamgood

[vlaiv]

Those settings should be fine indeed.

Unfocused light will be like DC offset - I don't think it will create significant pattern.

In any case, I'll see if I can find something meaningful when you post your subs. I'll inspect them in ImageJ to see if I can find any clues as what has happened.

[Jamgood]

Cheers, I appreciate it. I'll get them up when I get back home.

👍

Edited March 26, 2021 by Jamgood

[The Admiral]

I've read something about flats issues with the 294. Whether this is a similar thing I wouldn't like to say, but I suggest you Goggle "flats and ASI294MC Pro", and you'll find threads which might be pertinent. Apart from flat non-uniformities with filters, flat exposure time seems to be important too I believe.

It's just a thought, but I'd be interested in the outcome. Good luck.

Ian

[Jamgood]

39 minutes ago, The Admiral said:

I've read something about flats issues with the 294. Whether this is a similar thing I wouldn't like to say, but I suggest you Goggle "flats and ASI294MC Pro", and you'll find threads which might be pertinent. Apart from flat non-uniformities with filters, flat exposure time seems to be important too I believe.

It's just a thought, but I'd be interested in the outcome. Good luck.

Ian

Believe me, I've read everything there is to read. Posting here and asking for help was my last line of call. 

I read about long exposed flats, short flats, t-shirt sky methods, multi t-shirts, paper, light boxes, etc. I've tried all different ways and the flats always come out looking like decent flats whether they are 0.03s or 12s long. They just don't seem to work or work too well and overcompensate. 

 

 

 

Edited March 26, 2021 by Jamgood

[Jamgood]

This is how far the CC and L-Enhance are away from the end of the focuser. Maybe a problem? 

I currently have a William Optics Compression Ring on the focuser. I put this on on the first night with the camera as I didn't have enough front focus to get a sharp image. This being bigger gave me about an extra 18mm. The next day I re-collimated the scope and pushed the Primary up a bit to give me that extra wiggle room.

I had this small compression ring on first which wouldn't focus but now after collimation, should have enough front focus and the focuser tube will be much further out (18mm +/-) and less protrusion into the light path.

This is my set up but I do use a black shower cap on the end of the scope.

 

I'm just trying to work out how to Share a folder on Dropbox. Completely alien to me. Might be WeTransfer if I can't solve it.

Edited March 26, 2021 by Jamgood

[The Admiral]

7 minutes ago, Jamgood said:

Believe me, I've read everything there is to read. Posting here and asking for help was my last line of call. 

Sorry to hear that.

Ian

[Jamgood]

@vlaiv @david_taurus83 Hopefully this Dropbox link works.

https://www.dropbox.com/sh/zr40vbbipmajaw4/AADj4ouLzIZSeG99b81STd6Na?dl=0

I somehow deleted the original Darks that I used so I had to make some new ones today. In DSS, however, these make no difference. Same outcome for me.

There's an hour subs each of M13 and M101. (Darks, Flats & Dark Flats) I was just testing and getting to grips with the camera with APT which is why they're so short. Obviously being my first "real" night using the camera, I wanted to see how it all worked and troubleshoot anything that needed it. Didn't bank on this though!

Thanks again for any help.

👍

Edited March 26, 2021 by Jamgood

[Jamgood]

7 minutes ago, The Admiral said:

Sorry to hear that.

Ian

Cheers. I'm usually pretty good at problem solving things and google is always a great help but this has got me stumped. I'm interested to see what others make of the files with different software as that would rule out many things.

[iapa]

3 hours ago, Jamgood said:

This is how far the CC and L-Enhance are away from the end of the focuser. Maybe a problem? 

I currently have a William Optics Compression Ring on the focuser. I put this on on the first night with the camera as I didn't have enough front focus to get a sharp image. This being bigger gave me about an extra 18mm. The next day I re-collimated the scope and pushed the Primary up a bit to give me that extra wiggle room.

I had this small compression ring on first which wouldn't focus but now after collimation, should have enough front focus and the focuser tube will be much further out (18mm +/-) and less protrusion into the light path.

This is my set up but I do use a black shower cap on the end of the scope.

 

I'm just trying to work out how to Share a folder on Dropbox. Completely alien to me. Might be WeTransfer if I can't solve it.

Ahh,

 

I think I see part of the problem.

the lighting off to the right of the OTA

LOL

<GD&R>

[vlaiv]

I think I found a solution to this, but we should really call in author of APT in this discussion as I suspect that there is a bug in APT auto flat routine. @Yoddha

This is not the first time I've come across this and before I suggested people to ditch auto flat routine and shoot flats as regular exposures - and set exposure time manually.

Now I have confirmation that there is probably something wrong with APT.

You mentioned that you had your flat auto routine set to 25000 ADU mark, right? Look what happens to calibration of single sub when I add DC offset to master flat (making it larger - as we have established that it is too low and over correcting because of that):

Above is normal calibration - and it is obviously over correcting. Now I started adding DC offset to master flat

When I add 100ADU it looks like there is slight improvement.

when I add 10000ADU - there is visible improvement.

when I add 25000ADU - it looks like flats are working properly (this is much harder stretch than two above - yet background appears flat).

If I add 35000 ADU - then flats are starting to under correct.

To me it looks like software is doing something wrong. Actual ADU of flat is indeed ~25000 - but I'm suspecting that it is actually setting exposure time so that average ADU is 50000 - twice what is set as target and then subtracting 25000 from that.

I might be completely wrong on this - but that is what it looks like.

In any case - to solve these images - add 25000 ADU dc offset to master flat and in future - don't use APT auto flats feature but rather set your flat exposure manually - like when shooting lights. In fact - it is best if you don't mention flats to APT at all (just say you are shooting lights of a different target called flat panel ).

 

 

[vlaiv]

By the way  - I also noticed that flat exposures are 0.32938s and flat darks are 0.48406s.

Is there any particular reason these two are mismatched?

[Jamgood]

49 minutes ago, vlaiv said:

I think I found a solution to this, but we should really call in author of APT in this discussion as I suspect that there is a bug in APT auto flat routine. @Yoddha

This is not the first time I've come across this and before I suggested people to ditch auto flat routine and shoot flats as regular exposures - and set exposure time manually.

Now I have confirmation that there is probably something wrong with APT.

You mentioned that you had your flat auto routine set to 25000 ADU mark, right? Look what happens to calibration of single sub when I add DC offset to master flat (making it larger - as we have established that it is too low and over correcting because of that):

Above is normal calibration - and it is obviously over correcting. Now I started adding DC offset to master flat

When I add 100ADU it looks like there is slight improvement.

when I add 10000ADU - there is visible improvement.

when I add 25000ADU - it looks like flats are working properly (this is much harder stretch than two above - yet background appears flat).

If I add 35000 ADU - then flats are starting to under correct.

To me it looks like software is doing something wrong. Actual ADU of flat is indeed ~25000 - but I'm suspecting that it is actually setting exposure time so that average ADU is 50000 - twice what is set as target and then subtracting 25000 from that.

I might be completely wrong on this - but that is what it looks like.

In any case - to solve these images - add 25000 ADU dc offset to master flat and in future - don't use APT auto flats feature but rather set your flat exposure manually - like when shooting lights. In fact - it is best if you don't mention flats to APT at all (just say you are shooting lights of a different target called flat panel ).

 

That's awesome! Put my mind at rest that it's not the camera, cc, filter, focuser or stacking software.

I did start with 25000ADU which is what these flats were taken at with just a white iPad screen. I tried 30000, 20000, 15000 &10000. All the flats looked the same but with varying exposure times and ever so slightly different histograms. And then there were the different ways I did them. iPad screen, iPad screen with paper, iPad screen with t-shirts, sky with tshirts.

I have absolutely no idea how to add 25000 ADU dc offset to master flat because as I said in the first post, I think this is something way beyond my level of knowledge but doing normal flats and inputting manual times under a light frame setting in APT, I can do. 👍

48 minutes ago, vlaiv said:

By the way  - I also noticed that flat exposures are 0.32938s and flat darks are 0.48406s.

Is there any particular reason these two are mismatched?

I didn't notice that, sorry. I did flat darks for the second set of flats after I thought the first batch failed. Tried with flats and flat darks and that didn't work so went back to the original flats after countless other attempts. Got in a bit of a muddle, file wise.

Next clear night, I'll try again and see what happens.

Thank you for your time @vlaiv I really appreciate it.

Edited March 27, 2021 by Jamgood

[Jamgood]

@vlaiv Could the error with the Auto Flats Aid be anything to do with the use of the ASCOM driver as opposed to using the native ZWO driver?

@Budgie1 uses the same camera with APT and the Auto Flats Aid works well for him but he uses the Native Zwo driver at 120 gain, 8 offset.

I'm intrigued. It's going to be clear on Monday night so I can do some testing with manual flats. 

Edited March 27, 2021 by Jamgood