Binning flats

[Ken82]

Is setting the binning to the same as the light frames important in taking flats ?

thanks 

[vlaiv]

You are better off doing it that way if you already bin your subs in hardware. In principle you don't need to do same binning of flats, but you should bin them afterwards in software. Most software that does calibration will complain if you give it mismatched size subs for calibration.

[Nikolas74]

How about the target ADU value ? Is it the same like bin 1x1 or should be less ?

[vlaiv]

7 minutes ago, Nikolas74 said:

How about the target ADU value ? Is it the same like bin 1x1 or should be less ?

I would say the same - but that means you'll need 1/4 of exposure to get there - which might be an issue with mechanical shutters, in that case dim light source a bit.

[Ken82]

6 hours ago, vlaiv said:

I would say the same - but that means you'll need 1/4 of exposure to get there - which might be an issue with mechanical shutters, in that case dim light source a bit.

Yea I realised this today. APT flats test was getting an ADU of 64000 with the simple white t shirt method. My flats panel is only 160mm and I’ve been trying to get some results with the edgehd at f7. 

I may go back to using a refractor for a while until I’m comfortable calibrating this 071 as I’m trying to cross a few bridges at once. 

Do use pixinsight Vlaiv ? What’s your process to calibrating ? I usually follow the light vortex tutorial which has worked wonders until now. 

[vlaiv]

50 minutes ago, Ken82 said:

Yea I realised this today. APT flats test was getting an ADU of 64000 with the simple white t shirt method. My flats panel is only 160mm and I’ve been trying to get some results with the edgehd at f7. 

I may go back to using a refractor for a while until I’m comfortable calibrating this 071 as I’m trying to cross a few bridges at once. 

Do use pixinsight Vlaiv ? What’s your process to calibrating ? I usually follow the light vortex tutorial which has worked wonders until now. 

I tried PI once - download a trial, but found it too complicated / unintuitive at a first glance, so did not use it much (and the fact that price was a bit steep at the time), which in hindsight was a good thing - I started thinking about different processing algorithms and implementing my own stuff.

I do my calibration in ImageJ at the moment, but work on a small software that will do calibration. Doing it in ImageJ is just a bit more involved, but works and it's not overly complicated. I'll outline steps and details briefly.

- I use darks / flats / flat darks as calibration frames (no bias since I use CMOS sensor and bias is not reliable on my camera - that being ASI1600 v2). I also get as much calibration frames as possible (usually around 256 of each). My flat panel is rather strong so it does not take a lot of time to get one sub (just a few milliseconds of exposure - it takes more time to download a sub in SGP than to shoot it). Flat darks are the same settings as flats (meaning gain, offset, exposure length, temperature), darks are at the same settings as lights (again gain, offset, exposure length, temperature). Actually I set my offset at 64 and leave it there (specific for ASI1600 - other cameras can either have offset control but use different values, or have it set at factory without possibility of change).

- I stack flats with simple average. I do the same with flat darks. I create master flat by simply subtracting flat darks stack from flat stack. If I plan on doing any sort of measurement I "scale" my flats to 0-1 range. I do this by examining histogram and finding value at the peak of histogram (or doing average of top few percent of pixel values), then divide all values with that value. I'm not overly bothered about where histogram peak should be, but I do keep it in right half of histogram (so over 50%, probably around 80% or so). This is because again I use ASI1600 at unity gain, so it effectively works in 0-4000e region which is far from saturation value of about 20000e and it is certainly in linear region. I just make sure there is no histogram clipping.

- I stack darks with sigma clip method (it removes any sort of funny cosmic ray hits, and there has been some in my darks).

- Calibration after that is simple calibrated = (light-master_dark)/master_flat

- All operations are performed in 32bit per pixel mode, so first step I do is to convert precision of every file I'm working with.

[Ken82]

Thanks for the detailed response Vlaiv ! Quite a bit different to calibrating a dslr in pixinsight although I do follow some similarities like using as many calibration frames as possible. 

I have read elsewhere some are having to use longer exposures for flat frames as these cameras are unreliable with short exposures. But you are using a few milliseconds. That would certainly make it easier to acquire flats with this scope. 

 

[Gina]

I have an EL panel for Flats and find the exposure (automatic in Ekos) is around 0.3s with my ASI1600MM-Cool with gain of 600 (60dB), 3nm filters and f2.8 lens.  These seem to be pretty consistent.

[Datalord]

On 29/09/2019 at 13:12, Ken82 said:

I have read elsewhere some are having to use longer exposures for flat frames as these cameras are unreliable with short exposures.

*raises hand*

I have a 12" RC combined with a mechanical shutter. It's a pestilence to acquire flats. I can't use a panel, because it blows out the ADU, because the manual shutter only allows me to go down to 0.5s exposures. I have a grand total of 40 min at dusk to gather flats, so last round took me a month.

Btw, I take them bin 1x1 and downsample in PI for other binnings. Works perfectly.

[Ken82]

1 hour ago, Datalord said:

*raises hand*

I have a 12" RC combined with a mechanical shutter. It's a pestilence to acquire flats. I can't use a panel, because it blows out the ADU, because the manual shutter only allows me to go down to 0.5s exposures. I have a grand total of 40 min at dusk to gather flats, so last round took me a month.

Btw, I take them bin 1x1 and downsample in PI for other binnings. Works perfectly.

I just acquired a few flats earlier at 1.2s ADU 20000. Hope that works . 

[Datalord]

12 hours ago, Ken82 said:

I just acquired a few flats earlier at 1.2s ADU 20000. Hope that works .

You can go higher, as long as you're not clipping in the top. I start my run at 30k-40k at 0.5s and stop when I can't get it above 10k with 5s exposures.

[Ken82]

2 hours ago, Datalord said:

You can go higher, as long as you're not clipping in the top. I start my run at 30k-40k at 0.5s and stop when I can't get it above 10k with 5s exposures.

Yea thanks, I’ve also taken Vlaiv’s advice and changed back to 1x1 binning. So should be at 0.5” per pixel but due to the pixels I think it’s now more like 1” per pixel . Or at least that’s what I understand 😀

[Datalord]

18 minutes ago, Ken82 said:

So should be at 0.5” per pixel but due to the pixels I think it’s now more like 1” per pixel . Or at least that’s what I understand 😀

The resolution doesn't matter.  They are used to calibrate the images and whether you are shooting at 0.1" or 15" is irrelevant. It's the resolution of the image itself, x*y, that needs to match, with the dust bunnies in the right place. 

[Ken82]

2 hours ago, Datalord said:

The resolution doesn't matter.  They are used to calibrate the images and whether you are shooting at 0.1" or 15" is irrelevant. It's the resolution of the image itself, x*y, that needs to match, with the dust bunnies in the right place. 

Yea sorry I didn’t explain very well. I had started to take lights and calibration frames at 2x binning as I was of the assumption this would give me 1” per pixel as 0.5” is oversampled. But Vlaiv has previously commented on how the pixels and Bayer matrix interact to give me this resolution without using 2x2 binning. I’ve now taken my flats at 1x binning the same as my other lights and calibration frames. 

[vlaiv]

8 minutes ago, Ken82 said:

I’ve now taken my flats at 1x binning the same as my other lights and calibration frames. 

That is something you should always do, regardless of any binning applied (before or after) - use exact same settings for calibration files.

[Datalord]

18 hours ago, vlaiv said:

That is something you should always do, regardless of any binning applied (before or after) - use exact same settings for calibration files.

yes, in an ideal world. My own experience tells me the downsampled flats work great.

[vlaiv]

1 hour ago, Datalord said:

yes, in an ideal world. My own experience tells me the downsampled flats work great.

It's not something that is hard to do and you need to take shortcuts because of that. Why would you do it differently if it is a proper way to do it?

It really depends on characteristics of the sensor. Flats don't correct for dust and vignetting only, they correct imperfections in QE on pixel level as well. For example, look at this flat (crop and stretch to show issue):

In left bottom corner there is a "small" dust doughnut here cropped to 1/4 of its size (just to explain why there is ring there and for size comparison), but important thing is checker board pattern in flat. That is pixel to pixel variation in QE due to manufacturing process - maybe electronics between pixels or shape of micro lens - it does not matter. What matters is that there is a bit of QE difference on pixel scale.

When you downsample such flat, unless you are very careful in the way you do it, you will introduce correlation between pixel values and you will no longer have true representation of pixel QE levels.

Difference between pixels is something like 30ADU per 1600 ADU, so it is ~1.9%, not much, but I would rather avoid additional noise that would come out of messing up per pixel QE with downsampling if I can - and in fact I can - just by following above rule (which again is not really any harder to do than downsampling flats).

 

[Datalord]

6 hours ago, vlaiv said:

which again is not really any harder to do than downsampling flats

it is considerably harder to do when you can't use a flat panel because of the combination of blowing out the ADU and a mechanical shutter. I only have 40 min of light suitable for flats, IF there are no clouds. Last time it took me 20 days to coordinate a full set for all filters...