Problems with flat subtraction

[Ovi]

Hello! Recently I succeeded in debayering a Canon 450D through polishing, which left me with a mono sensor. However, I've been having some issues with flat subtraction. Here's the stacked file:

And here is the master flat:

To me it kinda looks like the flat is being over-subtracted in certain areas. For context, here are an uncalibrated and a calibrated single light frame:

You can start seeing those patterns appear in the calibrated image, although they are more subtle than in the stacked image. Any idea how I could fix this?

[bottletopburly]

I’d say the debayer exercise wasn’t very successful tbh .

[scotty38]

Sorry to say and just my basic opinion but over subtracted flats is being massively optimistic here. That sensor looks knackered......

Edited March 10, 2022 by scotty38

[vlaiv]

Are you sure, you are doing it right?

Flats should not be subtracted - they should be divided with.

calibrated file = (light - dark) / flat

Ignore the above (I did not properly read the post).

Can you post single light, single flat and master flat cr2 files for inspection?

 

Edited March 10, 2022 by vlaiv

[Ovi]

11 hours ago, vlaiv said:

Can you post single light, single flat and master flat cr2 files for inspection?

 

Here they are. Unlike the ones I sent before, these are uncropped.

L_0002_orion_20-52-43__1C_ISO800_120s.CR2 IMG_7340.CR2 masterFlat.tif

[vlaiv]

4 hours ago, Ovi said:

Here they are. Unlike the ones I sent before, these are uncropped.

There seems to be something wrong with your master flat.

I took only single flat - which looks good and calibrated single sub with it. I had to bin it quite extensively to get good snr - but here is result after calibration, binning and background gradient removal:

I see absolutely no trace of issues with flat calibration.

Here it is again with black point adjusted to maximally show nebulosity vs background:

(this is linear stretch so nebula is blown completely - but point is to look at background around it).

But if we inspect single flat vs master flat - we get interesting thing:

Median and mean values differ by about ~3.1 (3.1 and 3.15) between single sub and master.

This should not happen - they should contain roughly the same average and median values.

Can you inspect all your flat subs for average value and see if there are significant differences? Maybe you should discard some of your flat files from master flat - those that are significantly different from the rest?

[ollypenrice]

I, too, think the sensor is badly damaged. The flat has patches of black which presumably have no signal, or very little, so there is little or no information there for the calibration to adjust. We often see this when there is dust on the sensor: there is nothing the flats can do to correct for that.

Olly

 

[Ovi]

On 13/03/2022 at 15:23, ollypenrice said:

I, too, think the sensor is badly damaged. The flat has patches of black which presumably have no signal, or very little, so there is little or no information there for the calibration to adjust. We often see this when there is dust on the sensor: there is nothing the flats can do to correct for that.

Olly

 

The black patches are actually around 60% of the brightness of the rest (they're patches where the cfa wasn't completely removed), it's just that the flat is very stretched. While I do not think the sensor is damaged, I'm starting to think I might not get flat calibration to work at all. I'll probably try again with another sensor since I wouldn't be able to sleep at night knowing I abandoned this project while this close to success.

[ollypenrice]

10 hours ago, Ovi said:

The black patches are actually around 60% of the brightness of the rest (they're patches where the cfa wasn't completely removed), it's just that the flat is very stretched. While I do not think the sensor is damaged, I'm starting to think I might not get flat calibration to work at all. I'll probably try again with another sensor since I wouldn't be able to sleep at night knowing I abandoned this project while this close to success.

A 40% fall-off in signal is a lot. However, I have a 23% vignetting fall-off on one setup and it does calibrate out.

What's still to be explained, tough, is why the flats don't calibrate out the patterns formed on the sensor by debayering.

I have to ask this: would you not be better off sticking with a standard sensor and using a dual or tri-band filter?

Olly

[Ovi]

5 hours ago, ollypenrice said:

A 40% fall-off in signal is a lot. However, I have a 23% vignetting fall-off on one setup and it does calibrate out.

What's still to be explained, tough, is why the flats don't calibrate out the patterns formed on the sensor by debayering.

I have to ask this: would you not be better off sticking with a standard sensor and using a dual or tri-band filter?

Olly

I prefer seeing it as the debayered area being 60% more sensitive

As for why not sticking with a color sensor, the allure of increased sensitivity and resolution are too strong to resist. If I can't the flats to work, I'll give this project one last chance and after that I'll be content with putting it down to sleep and going down the duo-band filter path.