Stacking multiple choice fun special 🧐

[Jezphil]

I've got two sets of Ha subs of the same DSO from different nights. Ignoring how closely they are matched in framing terms for the purposes of this question, which of the following would be correct? 

1 The best result would come from stacking the first and second days's subs separately and then combining them.
2. Best result is to stack all the subs from both nights in one go.

3 Doesn't make any difference.

My (beginner) feeling is that it is best to treat them as two sets and stack separately so that each can benefit from its own dark flats to eradicate hot pixels which might vary from one night to the next. If so, another question arises...if you did this,  would you then combine the unstretched masters and stretch them as one, or stretch them individually? I'm thinking most likely the latter.

Can't believe how quickly astrophotography  has turned me into someone who loves nerdy questions like this 🤪 but I'm very interested to hear what you clever folk here think. Thanks in advance.

Edited October 25, 2020 by Jezphil

[Adam1234]

Hi, it shouldn't make a difference if you stack all the subs in one go, I've taken images over several nights and stacked all together. 

If you have specific calibration frames for each night, make sure you apply these to the appropriate subs before then stacking the calibrated subs together. You can do this easily in Deep Sky Stacker using the Groups tab I.e. put night 1 subs + calibration frames in one group, then night 2 subs + calibration frames in another group.

FYI any calibration frames in the Main Group tab get applied to all groups, so I put a random sub that is not being used in the Main Group and uncheck it so that it is not used, then put night 1 in group 1, night 2 in group 2.

[Jezphil]

Thanks. This is really useful. I use DSS so will try this. Need to look at the software to figure out that last bit though. So do you put respective dark frames in with each group and you need an unticked frame for it to know that there the different dark frames apply to different groups? 

[Adam1234]

No problem. Yes if you put the respective dark frames (and any other calibration frames you have, if any, that are specific to each night) in with each group. 

The purpose of the unticked frame in the Main Group is so that you get the subsequent groups - you can't get Group 1 without putting something in the main group, hence putting in an image that you're not stacking and unticking it.

Of course, if you have a set of calibration frames which are not specific to each night and want to apply to both sets of data, say bias frames, you can put these in the Main Group instead and DSS will apply these to all groups.

[johngm]

What was your Ha subs taken with ? DSLR or dedicated Cooled Astro Cam ?

john

Edited October 25, 2020 by johngm
Incorrect info

[Jezphil]

40 minutes ago, Adam1234 said:

No problem. Yes if you put the respective dark frames (and any other calibration frames you have, if any, that are specific to each night) in with each group. 

The purpose of the unticked frame in the Main Group is so that you get the subsequent groups - you can't get Group 1 without putting something in the main group, hence putting in an image that you're not stacking and unticking it.

Of course, if you have a set of calibration frames which are not specific to each night and want to apply to both sets of data, say bias frames, you can put these in the Main Group instead and DSS will apply these to all groups.

Okay, got it. Thanks

[Jezphil]

30 minutes ago, johngm said:

What was your Ha subs taken with ? DSLR or dedicated Cooled Astro Cam ?

john

Cooled CDD Astro cam. 

[johngm]

18 minutes ago, Jezphil said:

Cooled CDD Astro cam. 

Then you will be able to use the same darks, so long as they are at the same exposure length and temperature, and stack them all together . Another tip when using DSS, i only stack the best 90% of the images, and even down to 80% if seeing conditions vary greatly. DSS will exclude your worse subs in stacking.

John

[Adam1234]

22 minutes ago, Jezphil said:

Cooled CDD Astro cam. 

In which case if you had the camera set to the same temperature for both nights, you can use your same set of darks for both nights (provided the same exposure, gain etc were used)

[Jezphil]

Thanks. Don’t the darks get rid of hot pixels though? Might these not be in different places on subsequent nights? 

[Adam1234]

Hot pixels are an artefact of the camera sensor, and as far as I'm aware don't change on a day to day basis, hence why most people using cooled cameras reuse darks, and maybe redo them a couple times a year perhaps. 

@vlaiv could probably provide you with a better understanding of hot pixels and dark frames.

[Adam1234]

6 minutes ago, Jezphil said:

 Might these not be in different places on subsequent nights? 

You could be thinking of flat frames which reduces vignetting and shadows caused by dust bunnies (which could well be in different places each night if you remove your camera after each session).

[ollypenrice]

Let's assume you can use the same darks and flats for both runs. To some extent the choice between 'stacking the two stacks' and stacking the lot in one go depends on which stacking algorithm you're going to choose. If you're going to use a Sigma Reject algorithm, which is very powerful, then the more subs in the stack the better it will work. It's looking for rogue pixels, outliers from the norm, so the more subs you have, the more successfully the algorithm can establish a norm, and so identify the outliers (which will be awarded the average value of the norm.) If you have a sat trail in an otherwise good sub, for instance, and you have a lot of subs in the stack, the rogue values of the trail will be overwritten by the average values of the norm in a Sigma routine. The trail will vanish. The same is true for random residual noise after dark subtraction. Since it's random it isn't part of the norm and will be over-ruled by the norm. So if using Sigma it is certainly best to restack from the start. (If you use a simple averaging algorithm the trail will be 'diluted' by the averaging but not excluded.)

If not using Sigma it still has to be mathematically better to stack from the start as well, but less so.  You may not notice much real-world difference. Do stack from linear in all cases, though.

I would always restack from scratch and use a Sigma routine if the calibration files were applicable and the images from the same scope.

Olly

Edited October 25, 2020 by ollypenrice

[Jezphil]

57 minutes ago, Jezphil said:

Okay, got it. Thanks

Just checked this out in Deep Sky Stacker. Makes complete sense now. Hadn’t used that ‘groups’ function before.

Edited October 25, 2020 by Jezphil

[vlaiv]

Hot pixels should not change day to day. They are due to manufacturing artifacts and current leak in sensor substrate (material sensor is built from and where those pathways for current are printed). There is no perfect insulator and every insulator will "fail" at some point. Current will leak into particular pixel and that will make it hot pixel.

Since there are physical stresses on substrate from both cooling and electrical forces - hot pixels can change during life time of camera, but they don't do it over night. It is however advisable to change your master darks once or twice a year because of this change in physical properties of sensor "material".

As for answer to original question:

2. stack as individual subs - after their respective calibration (each set with their calibration frames if they are different).

 

[Jezphil]

6 minutes ago, ollypenrice said:

Let's assume you can use the same darks and flats for both runs. To some extent the choice between 'stacking the two stacks' and stacking the lot in one go depends on which stacking algorithm you're going to choose. If you're going to use a Sigma Reject algorithm, which is very powerful, then the more subs in the stack the better it will work. It's looking for rogue pixels, outliers from the norm, so the more subs you have, the more successfully the algorithm can establish a norm, and so identify the outliers (which will be awarded the average value of the norm.) If you have a sat trail in an otherwise good sub, for instance, and you have a lot of subs in the stack, the rogue values of the trail will be overwritten by the average values of the norm in a Sigma routine. The trail will vanish. The same is true for random residual noise after dark subtraction. Since it's random it isn't part of the norm and will be over-ruled by the norm. So if using Sigma it is certainly best to restack from the start. (If you use a simple averaging algorithm the trail will be 'diluted' by the averaging but not excluded.)

If not using Sigma it still has to be mathematically better to stack from the start as well, but less so.  You may not notice much real-world difference. Do stack from linear in all cases, though.

I would always restack from scratch and use a Sigma routine if the calibration files were applicable and the images from the same scope.

Olly

Thanks Olly. I’m using Deep Sky Stacker for this. Will try the ‘groups’ function and allocate darks/flats to each group and of course maximise my lights. I don’t know where that leaves me with the algorithm.

[Jezphil]

27 minutes ago, Adam1234 said:

Hot pixels are an artefact of the camera sensor, and as far as I'm aware don't change on a day to day basis, hence why most people using cooled cameras reuse darks, and maybe redo them a couple times a year perhaps. 

@vlaiv could probably provide you with a better understanding of hot pixels and dark frames.

Thanks 👍

[ollypenrice]

26 minutes ago, Jezphil said:

Thanks Olly. I’m using Deep Sky Stacker for this. Will try the ‘groups’ function and allocate darks/flats to each group and of course maximise my lights. I don’t know where that leaves me with the algorithm.

I don't have the groups facility in my stacking programme but it is a good idea and should let the Sigma routine work as if all subs were from the same run.

Olly

Edit: For anyone else without a 'groups' facility you have the option of calibrating all the subs with the right flats and darks without combining them and then stacking all the calibrated subs without further calibration. That will have the same effect.

Edited October 25, 2020 by ollypenrice

[Jezphil]

1 hour ago, ollypenrice said:

I don't have the groups facility in my stacking programme but it is a good idea and should let the Sigma routine work as if all subs were from the same run.

Olly

Edit: For anyone else without a 'groups' facility you have the option of calibrating all the subs with the right flats and darks without combining them and then stacking all the calibrated subs without further calibration. That will have the same effect.

Thanks Olly 👍