PixInsight Cosmetic Correction & Darks Frames

[Northernlight]

Hi All,

I'm just working through Adam Blocks videos and I've just finished the section on cosmetic Correction, but it appear to gloss over the use of Darks with the Cosmetic Correction Tool, so just wonder if some of my Fellow PI Disciples could Help.

so assuming that i've been through the normal calibration process and my lights have been Bias, Dark & Flat subtracted :-

1) If you use a master dark with Cosmetic correction does it get scaled during cosmetic correction - i.e. can i use a 1200s Dark when correcting 600s Subs or does the dark need need to match the length of the light sub - i.e. 600s

2) Is using the master dark actually necessary ? - as if the light has already been dark subtracted  ?

The reason i ask the above question is that I noticed that in Adam Blocks video on BPP - he uses a generic Cosmetic-Correction Process icon, which does not use a Master Dark and only enables the Automatic Hot Pixel detection option. So just wanted to see what other people do, as I have seen other people create Cosmetic-Correction process Icons for each individual sub length, which use matching length master darks (300s, 600s, 900s) etc.

 

Cheers,

Rich.

[discardedastro]

19 hours ago, Northernlight said:

Hi All,

I'm just working through Adam Blocks videos and I've just finished the section on cosmetic Correction, but it appear to gloss over the use of Darks with the Cosmetic Correction Tool, so just wonder if some of my Fellow PI Disciples could Help.

so assuming that i've been through the normal calibration process and my lights have been Bias, Dark & Flat subtracted :-

1) If you use a master dark with Cosmetic correction does it get scaled during cosmetic correction - i.e. can i use a 1200s Dark when correcting 600s Subs or does the dark need need to match the length of the light sub - i.e. 600s

2) Is using the master dark actually necessary ? - as if the light has already been dark subtracted  ?

The reason i ask the above question is that I noticed that in Adam Blocks video on BPP - he uses a generic Cosmetic-Correction Process icon, which does not use a Master Dark and only enables the Automatic Hot Pixel detection option. So just wanted to see what other people do, as I have seen other people create Cosmetic-Correction process Icons for each individual sub length, which use matching length master darks (300s, 600s, 900s) etc.

 

Cheers,

Rich.

I'd always match darks to subs, since amp glow and other effects can grow nonlinearly with exposure time. You need the darks anyway for calibration so why not match them?

Using the master dark is always a good idea as calibration won't necessarily remove hot pixels but the master dark will act as an excellent reference for where to focus CC's attention. The CC algorithm can then make the required corrections with knowledge of the sensor's hot/cold pixels.

[Northernlight]

Hi Many thanks for your reply.  I traditionally would do CC as a separate step using Master Darks of Matching Sub Lengths, but Adam Blocks video seemed to suggest that only a single master dark is required as it will be scaled. In his Calibration Video, he doesn't even use a Master dark for CC and just selects the Auto options for Hot Pixels in CC and that's it. He then exports that as a generic Process icon to be used with BPP.

Just thought his approach seemed a little odd and Wondered if by using a master dark in CC you were somehow going to end up subtracting dark current twice. 

I left a message on his forum but 3 days later still no response yet 🙄

[MarkAR]

Have a look at Chris Woodhouse on YT. There's one where he shows using Cosmetic Correction.

[ngc1535]

HI all,

 

There may be some confusion here. Cosmetic Correction is a kernel filter that is run on data after they are calibrated with biases, darks, and flats. Cosmetic Correction (basically a hot pixel filter as well can have potential uses for column defects) is not connected to darks in the sense of calibration, scaling, etc. I do demonstrate it is possible to scale darks- which has some benefits regarding maintaining a dark library for a cooled CCD camera- but it comes with some pitfalls that deal with hotpixels and occasional scaling errors through dark frame optimization. It is almost always better to subtract a dark of the same duration if you have a high quality master in hand. 

This is a subtle connect between dark frames and Cosmetic Correction in that one of the methods of using Cosmetic Correction is to use a dark frame to generate a hotpixel map. But again, this isn't related to the calibration process. 

Feel free to connect with me through my website at AdamBlockStudios.com if this particular issue is confusing. 

Thanks,

Adam

 

[MarkAR]

Hi Adam, if I understand you correctly then Cosmetic Calibration is applied to the calibrated frames (if necessary) before they are integrated.

Also wish to thank you for all the PI videos, they are very helpful.

[Northernlight]

No I think That Adam is saying that we do the usual calibration then cosmetic correction can be performed in a number of ways:-

1)  You can use a master dark with cosmetic correction as one method of removing the HotPixels

2) The other method is not to use a dark and just use just use the auto detection option for removing the HotPixels, as that what adam does in his Cosmetic correction Videos as far as i remember when i looked at it last week which allows you to use a single process icon for BPP/WBPP

 

Cheers,

Rich.

Edited May 10, 2020 by Northernlight

[ngc1535]

10 hours ago, MarkAR said:

Hi Adam, if I understand you correctly then Cosmetic Calibration is applied to the calibrated frames (if necessary) before they are integrated.

Also wish to thank you for all the PI videos, they are very helpful.

Yes! 

This is something of pitfall of "automation" through scripts- it isn't always obvious what the order of operations is. 

Great job on working through the videos and I am very glad they are helpful!

 

Sincerely,

Adam Block

[ngc1535]

9 hours ago, Northernlight said:

No I think That Adam is saying that we do the usual calibration then cosmetic correction can be performed in a number of ways:-

1)  You can use a master dark with cosmetic correction as one method of removing the HotPixels

2) The other method is not to use a dark and just use just use the auto detection option for removing the HotPixels, as that what adam does in his Cosmetic correction Videos as far as i remember when i looked at it last week which allows you to use a single process icon for BPP/WBPP

 

Cheers,

Rich.

It is true that after you have calibrated data in hand, you can apply Cosmetic Correction. There are THREE different methods of using Cosmetic Correction. I use the Sigma method (which is relatively automatic and quite robust. You can be particular about it and determine hot pixels somewhat empirically by measuring one of the dark frames. (All of this is explained in my lessons.  ) I don't like "1)" above as much because it doesn't work in every case.

Thanks!

-adam

[Northernlight]

Hi Adam,

Just slowly working through your video's when time permits and the kids aren't constantly harassing me.  I ordered a new high quality newt today, which will be ready in around 10 weeks - so it's motivation to keep working through the tutorials to get my PI skills in order.

For me, i've always found the automatic sigma method works very well.

 

[ngc1535]

9 hours ago, Northernlight said:

Hi Adam,

Just slowly working through your video's when time permits and the kids aren't constantly harassing me.  I ordered a new high quality newt today, which will be ready in around 10 weeks - so it's motivation to keep working through the tutorials to get my PI skills in order.

For me, i've always found the automatic sigma method works very well.

 

I agree. I use Cosmetic Correction (auto) as an incremental step. Anything that it does not take care of- is likely to be handled during rejection in ImageIntegration.

-adam

[Northernlight]

Adam, as separate note - will you be releasing a video on WBPP at some point as part of the fundamentals license ?

[ngc1535]

Not only am I releasing videos on WBPP... but I am making them public. 

https://www.youtube.com/playlist?list=PLAzMa9eIVQkDnrwzRCDLYB3JGoJz7tlxF

There are still a few more sections to go... and the section I hope to release tomorrow is the most important one I think.

 

-Adam Block

[kirkster501]

I use darks and auto-detect as a CC process that WBPP links to.

I set the darks to 5 sigmas and autodetect to 3 sigmas.

I find it gives me a very good result.