Integrating data from multiple nights

[Vic L S]

Hi all, 

So far I have only calibrated and processed data taken from a single night, but now I'd like to try and improve my images by taking more subs over multiple nights. 

My question is-- when integrating data from different nights, what is the usual process? 

Do you calibrate each set of data from each night with their specific darks, flats and dark flats, then combine the master lights together? 

Or do you calibrate all data together with one set of calibration frames from one of those nights? 

Thank you in advance for your help and advice! 

[Elp]

1. You can calibrate and stack it in all one go with each session having their respective calibration data though it could be a logistical nightmare, this generally gives the best result,

2. If you're very very lucky and you keep your camera settings the same, temps, focus position, orientation etc you might be able to calibrate it all with one set of calibration files,

3. Or do what I do, stack the master stacks. At least this way you can review whether there were any issues during the sessions for corrective action next time.

[Fegato]

I use flats, darks and bias. The darks and bias are only updated very occasionally, so these remain the same anyway.

With flats, it depends on your situation to some extent.  I don't have an observatory, but I don't build and tear down every time, I have the rig on wheels and wheel it outside a short distance. I also don't normally rotate my camera. So if I'm happy that nothing has changed in my image train, I will re-use flats. Any fiddling with anything, or an extended time period, and I'll shoot new flats.

If you need to take new flats for your second session (& flat darks if using), then yes - calibrate separately, and then integrate the two sets together after calibration.

[Vroobel]

I can recommend ASTAP if you are interested in free software. It can stack your all multi-night data with flats if you do them after each session. Be aware that flats allow shadow removal (related to dust on your lens/mirror or filter if you use any), so I can recommend doing them every time. Also, ASTAP offers plate solving and gives quite realistic colours.

Edited April 30 by Vroobel

[Vic L S]

4 hours ago, Vroobel said:

I can recommend ASTAP if you are interested in free software. It can stack your all multi-night data with flats if you do them after each session. Be aware that flats allow shadow removal (related to dust on your lens/mirror or filter if you use any), so I can recommend doing them every time. Also, ASTAP offers plate solving and gives quite realistic colours.

I actually use PixInsight, and WBPP to integrate my files, I should have mentioned that in my post. 

[Hughsie]

12 minutes ago, Vic L S said:

I actually use PixInsight, and WBPP to integrate my files, I should have mentioned that in my post. 

When doing multi-night projects I save all my data by night. In each folder will be the lights, flats and dark flats captured for that session. If the project has the same camera settings such as exposure time, gain and temp then there will only be a single master dark otherwise there will be different darks to reflect those camera settings as well.

In WBPP you can view a calibration diagram for each set of lights to ensure the correct flats and darks are being used. It helps to be methodical otherwise it can become confusing. If WBPP selects the wrong calibration files then you can override it. 

Another setting to be aware of is the exposure tolerance. This defaults to 2 seconds. WBPP will integrate all calibrated lights with the same exposure time but you may have used different exposure times during the project and still want the data integrated into a single master light. For instance, one night you may have collected subs at 300 s exposure and a second night at 180 s exposure. By default WBPP will produce two master lights one representing 300 s data and the other for 180 s. There may be good reason why you want this eg where you want data not blowing out cores on globular clusters. If you don’t need them to be separate then you can adjust the exposure tolerance from 2 to 120 s (300-180) and WBPP will combine the lights data post calibration for you.

Hope that helps.

John

[Vroobel]

Thanks John for the tip regarding tolerance. I learn every day. 😊

There is also another way to add files with e.g. different exposure time by adding custom files. You can cheat a bit and declare that the files have other parameters then in fact they have. 

[Fegato]

4 hours ago, Vic L S said:

I actually use PixInsight, and WBPP to integrate my files, I should have mentioned that in my post. 

Just in case you're not using them, note that WBPP has Grouping Keywords as well. These are really useful.

So - I also organise different sessions into sub-directories (flats and lights, or a flat master if I'm re-using). I name them either Session 1, Session 2...  or Panel 1, Panel 2....  and I have them set up to calibrate separately, and then Session is set up to integrate to a single master, and Panel to separate masters. You just use the Grouping Keywords table to the right hand side.

re: Darks - I have all my dark masters permanently loaded into WBPP. It just picks the right exposure, so I never even have to think about them.

[Vroobel]

4 minutes ago, Fegato said:

Darks - I have all my dark masters permanently loaded into WBPP. It just picks the right exposure, so I never even have to think about them.

 

How to do it? Can I do it with masterBias files as well? For different cameras? 🙂

[Fegato]

Just add them all in and don't take them out again. They just stay there. I have darks for two cameras in there. WBPP just recognises which one to use for which lights (probably using the sensor size). I'm sure it will work with multiple bias too.

[Vic L S]

Amazing! Learn something new everyday. Thanks everyone, this has been really helpful.