Flats problem - again

[MKR]

Hi,

I've been trying to calibrate my M101 imaged with ASI1600MM, but am struggling with flats calibration. If I use flats, the centre is darker than the edges. If I don't use flats centre is lighter than the edged. It looks like flats are overcorrecting... I don't think I'm doing anything silly. Trying to stack it in DSS with pretty much default settings. I attach light, dark, flats and dark flats. But even in a simple case - just lights + flats still isn't working

I'm just focusing on the RED channel only to start with. My flats are taken on the same filter in Ekos with a target 25,000 ADU. I'm attaching the files I'm using. I'm hopeless... If anyone can take a look at this, I'd greatly appreciate. Many thanks.

-- M.

 

M_101_Light_Red_120_secs_2022-04-21T23-46-56_012.fits MasterDark_Gain139_120s.tif MasterFlat_Gain139.tif MasterDarkFlat_Gain139_0s.tif

[vlaiv]

Well, something is wrong with the files and no wonder you are getting very strange calibration results.

First off - do you have cooled camera? Is it ASI1600MM-Pro or regular ASI1600MM without cooling?

I'm asking because your master dark is very very noisy. Much noisier than one would expect from cooled camera.

left is your dark stack (not sure how many subs you stacked, but it does not matter) - right is only one dark sub from my cooled ASI1600. Both are stretched to the point amp glow is starting to show.

Your stack is much more noisy.

Second - I'm not sure what software are you using to capture and process data, but you have very odd dark flat there.

I measured both your master dark and master flat dark and here are the results:

First - it is very odd that mean value of master dark of 120s, cooled or not be that much lower than master flat dark as master flat dark is much lower in exposure time.

Second - column StdDev is confirming my assumption on not having cooled camera - noise is so much worse in master dark than in master flat dark - and that can be only due to thermal noise in those 120 seconds. With cooling, there is some difference, but difference is small - not x17 more noise.

Third very strange thing is that master flat dark has much higher min value than master dark. In fact - that low minimum value in stack suggests that offset is not good for camera.

In the end I conclude two things:

1. You don't have set point temperature / cooled camera so it is very hard to get matching darks

2. You probably shot master flat darks at very different settings to darks, or you have light leak

In any case - this is the cause of failed flat calibration.

[MKR]

Thanks. That's not very good, because I do have a cooled camera - ASI1600MM Pro. Every single frame taken was on a set temperature 0C - reported by Ekos. Gain 139 on every light/flat/dark. Offset 56, although I think I forgot to set offset for the darks (flat darks have offset 56).

I'm attaching a single Dark and a single flat dark as well, maybe I did something wrong with stacking them.

Is my camera broken? 😕 Ekos reports temperature which oscilates around 0C (from -0.5C to + 0.5C). Not sure how to check if it actually is this temperature though.

Dark_120_secs_2022-04-19T23-09-38_010.fits red__Dark_0.052_secs_2022-04-22T01-52-16_008.fits

[MKR]

Btw, i stacked 20 of each - flats, darks and dark flats

[vlaiv]

3 minutes ago, MKR said:

Offset 56, although I think I forgot to set offset for the darks (flat darks have offset 56).

This could be issue.

Try doing one more set of darks (luckily they can be done at this time as well) - but this time setting all parameters the same.

Yes, just checked fits headers - dark is at offset 0. Better redo those at offset 56 as well.

[vlaiv]

Issue with noisy master dark could be due to histogram clipping to the left due to too low offset.

See how it is "glued" to the left? No nice bell shaped curve for that dark.

This is histogram of flat dark

Much better looking.

 

[MKR]

Will do today. Thanks. I did try to stack only my master flat with a single sub and it didn't work either. It was noise as expected, but also did not correct the vingneting correctly.

[vlaiv]

4 minutes ago, MKR said:

It was noise as expected, but also did not correct the vingneting correctly.

In order for flats to work properly, both lights and flats must contain only light signal - offset and dark current must be removed from both (that is why we take darks and flat darks).

If we don't remove either of those two properly - flat correction will fail as it will "operate" on or by signal that is not subject to vignetting.

[MKR]

Thank you so much. Redoing my darks now. 120s, 139 gain, 56 offset, fixed 0C. According to the FITS viewer from Ekos/Kstars  each of them has parameters which worry me.

Stddev ~187, Mean 965, min 192, Max 65504. Is this expected for individual darks? Should I expect the outliers to go away once stacked ~30 of them which would reduce std Dev?

Thanks.

[alacant]

2 hours ago, MKR said:

I'm attaching the files I'm using

Hi

Perfect? No, But acceptable. Are you sure it's not just a light pollution gradient bottom up?

 

 

Edited April 22, 2022 by alacant

[vlaiv]

11 minutes ago, MKR said:

Stddev ~187, Mean 965, min 192, Max 65504. Is this expected for individual darks? Should I expect the outliers to go away once stacked ~30 of them which would reduce std Dev?

I guess for two minutes and 0C that is ok.

Stack them and apply them to see what sort of result you will get.

[MKR]

From some reason I've decided to use 0C always. I've read your other posts here (you are really helpful :)) and my conclusion was that fixed temp is the most important, whether it's 0C or -15C, it doesn't make that much difference actually. I thought that I don't want to stretch the cooling too much and want my camera to last for a long time. Don't know if it's reasonable or not...

[MKR]

Just now, alacant said:

Hi

Perfect? No, But acceptable. Are you sure it's not just a light pollution gradient bottom up?

Don't think so, gradient is in the same place all the time I think. Will confirm once my darks are fixed 🤞

[MKR]

OK, I think it's a lot better now. Still not great calibration, but better. Not sure about the noise level. @vlaiv what software do you use?

MasterDarkFlat_Gain139_0s.tif?MasterDark_Gain139_120s.tif 

MasterFlat_Gain139.tif Red_calibrated.tif

[vlaiv]

9 minutes ago, MKR said:

Not sure about the noise level. @vlaiv what software do you use?

What for?

For capture I use SGP. For calibration / stacking / preprocessing - mostly Fiji/ImageJ, Siril (this is new, it can align / interpolate with Lanczos) and Gimp for post processing.

[MKR]

Just now, vlaiv said:

What for?

For capture I use SGP. For calibration / stacking / preprocessing - mostly Fiji/ImageJ, Siril (this is new, it can align / interpolate with Lanczos) and Gimp for post processing.

For the stats? FITS viewer from Ekos isn't very handy as it's on my Astroberry. Where are those screenshots from? Many thanks for help. I think I'm getting somewhere.

[vlaiv]

1 minute ago, MKR said:

For the stats? FITS viewer from Ekos isn't very handy as it's on my Astroberry. Where are those screenshots from? Many thanks for help. I think I'm getting somewhere.

ImageJ/Fiji

Btw, there is quite a bit of issue with flats.

Not sure what it is.

Here are stats for dark and flat dark

So these are actually x16 lower because of 16bit padding.

That is ~59.9e and ~53.95, so about 6e of dark current.

Let see if we can match that to dark current data for ASI1600

Dark current is about 0.006e-/s at -20C

You have 6e over 120s so that is 0.05e/s and that is at 0C?

Doubling temperature is about 6C

If we start at -20 - we have three times to double until we get 0C (-20 + 6C +6C+6C = -2C)

0.006 * 8 = 0.048e/s

That sort of matches ok. I don't think there is light leak in darks. How about flats? Or lights? Is there any chance you have a light leak there? What is your setup like?

 

[MKR]

Not sure I understand the calculation above. Do you think my darks are ok-ish, given I took them at 0C (Just thinking if my camera works as expected)?

Maybe next time, I'll go to -10C to see how it improves things.

Now that you asked about a light leak in flats & lights - I have covered the primary with tin foil on my SW 200P - it was leaking a lot. There's a little bit of a leak from the focuser still, I don't know if it's significant. I can see it when I flash with flash light around the focuser's screws. Could this cause my issue with flats? Regardless of the camera used, I could never properly calibrate it with flats - I had been using a DSLR before

[alacant]

20 hours ago, MKR said:

gradient is in the same place all the time

Mmm. If I calibrate with your dark-flat, it doesn't work very well. If I simply subtract a constant bias value, it works better.  Certainly good enough to go on and process, especially as I just guessed a value.

@lock042 -who knows about these things- confirms that flat-dark frames are a waste of time. You don't need them-. Have a look here.

18 hours ago, MKR said:

covered the primary with tin foil on my SW 200P - it was leaking a lot

Ah, OK. A 200p. That could explain the unevenness. A black shower cap over the end of the tube would be a better idea than shiny reflective foil;) Have a look too at the focuser base curvature; it quite often does not match the radius of the tube. A long -at  least 30cm- dew shield helps and finally, a felt covering over the focuser barrel where it enters the focuser mount should get rid of the last reflections. Is the inside of the tube evenly matt? Grease marks? Nuts which protrude inside blackened? Secondary mirrior edges black? Spider matt black?

I'm assuming you've already eradicated any cc issues.

Cheers and HTH

Edited April 23, 2022 by alacant
spell foil

[vlaiv]

9 hours ago, MKR said:

Not sure I understand the calculation above. Do you think my darks are ok-ish, given I took them at 0C (Just thinking if my camera works as expected)?

Yes it works as expected.

Here is explanation for calculation.

I subtracted two darks under following premise: flat_dark is very short and is taken with same settings as dark. This means that it contains almost no dark current signal, but does contain bias signal.

Subtracting the two leaves only dark signal that accumulated over 120s. We average values of subs, divide it with 16 as 12bit camera data is padded with four zeros when written as 16 bit format (equivalent of multiplying with 16 - same as 123 written as 12300 is same as multiplying with 100 - only former was done in binary) and subtract the two.

Then we divide result with 120s to get dark current per second at 0C - in your case this turned out to be ~0.05e/s

Value that I've found for dark current of ASI1600 online is 0.006e/s at -20C. At lower temperature dark current is smaller, so how do we convert this value to 0C dark current to compare to what your dark shows? It turns out that there is a rule - each N degrees C increase in temperature dark current value doubles - this is called dark current doubling temperature and for most sensors it is somewhere around 6C

This simply means that at

-20C - dark current is 0.006e/s

-14C - dark current is double of above so 0.012e/s

-8C - dark current is double of above so 0.024e/s

-2C - dark current is double of above so 0.048e/s

-2C is close enough to 0C and you have 0.05e/s versus 0.048e/s - that is within measurement error, so I don't really think you have significant light leak with darks.

On the other hand if you have light leak on your setup for flats and lights - that will also skew flat calibration.

Any signal that is present - that did not come from objective lens of telescope and thus is subject to vignetting / dust / etc - that is not removed will mess up flat calibration.

This includes wrong offset, improper dark removal, light leak - anything that changes pixel values.

Pixel values are expected to be with certain ratio - say you have 1000e of signal and you have 100% and 80% illumination (center and edge) - you will measure 1000e and 800e respective.

1000/1.0 and 800/0.8 = 1000, 1000 - nice flat background equal everywhere.

Try adding some offset to either flat or lights - for example, light leak when shooting lights - say you have additional 100e that illuminates sensor

Now you have 1100 / 1.0 and 900 / 0.8 = 1100, 1125 - suddenly you have over correction as corner is brighter then center

Maybe you have light leak in flats? Now flats will have higher values then they should

1000 / 1.1, 800 / 0.9 = ~909.091, ~888.889

Huh, now you have under correction - center is still brighter than corners.

You have over correction - which means that lights are "stronger" than they should be, or flats are "weaker" than they should be.

Stronger lights means light leak when shooting lights

Weaker flats can mean stronger flat darks (as you remove them so if they are larger - result will be smaller in value) - which means light leak when shooting flat darks.

In any case - you should look to minimize light leak if possible in order to get flat calibration to work properly.

[MKR]

Thanks guys. This is a great explanation @vlaiv. I'm shooting again, covered all (I hope) of the wholes. Hopefully it will make a difference. If not, I think I'm giving up on 200P - will upgrade to a refractor.