The joys of sum stacking with LodestarLive

[Martin Meredith]

In my most recent Lodestar Live session last Friday I started to experiment with sum stacking. Up to this point I've not used the sum stacking option much because of a perceived need to constantly shift the black and white points as each sub comes in. But I've seen some great images on this forum recently resulting from sum stacking (thanks Don and others) so this time I just left it stacking away before pausing and adjusting. 

I've also tended to think of sum stacking as a way of detecting very faint stuff. For example, here's 8 x 30s sum of the faint mag 13.7 planetary nebula NGC7094 in Pegasus (mid top left -- ignore the dust shadow near the centre…). For comparison, M27 is more than 6.5 magnitudes brighter. With mean stacking I could barely make this out.

But sum stacking helps for brighter objects too. Here's a comparison of the wonderful barred spiral NGC7479 in Pegasus, a truly magnificent sight on the page when I planned my session and fascinating to see it appear on the screen. The right panel is a stack of 16 x 30s means (I gave it longer to give the features a chance to appear through the noise). The left panel is a stack of 6 x 30s summed. In spite of the noisier image, the dual nature of the spiral arms to the right of the galaxy is much clearer via the sum stacking approach.

Finally, here's the same treatment applied to M33 -- an object I wouldn't have thought of as benefitting from sum stacking. Top-left is a single 30s sub, top-right 4 x sum, bottom left 10 x sum, bottom right 18 x sum. The faint outer detail is much clearer in the longer runs.

Something else I noticed is that sum stacking appears to widen the important 'data' part of the histogram, making further tweaks in gamma/contrast less critical and helping in manual colour balancing.

All in all a great learning experience for me. Thanks Paul for including this feature 

cheers

Martin

[Dom543]

Hi Martin,

I particularly like your NGC7479.

I don't know how you are using sum stacking. In my experiments with LL sum stacks were getting brighter and brighter with just one having the right balance. One had to catch that one and stop the process. If one went too far, the image got washed out an one had to start again.

In his note about stacking, Paul also mentions hybrid stacking. There you can set the number of frames that will be summed. If you have more frames, then cycles sums will be averaged. I believe that this is still in the planning stage for LL but would be nice to have it implemented.

Sum stacking is also useful, when one cannot have longer exposures due to wind or imperfect alignment. Right now, for example, I am working in New England for 4 mounts away from home. I can watch stars from a South facing fire escape but I don't see Polaris. So my alignments are just so so. 15 sec exposures are fine but 60 secs have stretched out stars. It would be nice to synthesize 60 secs from four 15 secs by summing.

Clear Skies!

--Dom

[Martin Meredith]

I don't know how you are using sum stacking. In my experiments with LL sum stacks were getting brighter and brighter with just one having the right balance. One had to catch that one and stop the process. If one went too far, the image got washed out an one had to start again.

Thanks Dom. I hadn't seen NGC7479 before which is surprising since it is well-placed and a really beautiful object.

You LL experience with sum stacking mirrors mine prior to this session. For me, a typical session would involve turning the gamma up early on and leaving it at a mid-high value for the rest of the session. If you do this and then try sum stacking it is true that the images quickly wash out (and the bulk of the histogram reaches the right-hand edge very quickly -- same thing). 

The solution I realised is to turn down the gamma or brightness to ensure that the main body of the histogram remains more or less in the middle of the range. I'm not sure if this will work with all objects though, but for M33 as you can see I was able to reach 18 x 30s without burning anything much out (and I only stopped at 18 subs due to the primary starting to dew up in 99% humidity). Perhaps I was lucky in choosing an object with a relatively dim core and little in the way of stellar interference. More testing will be done on the next clear night!

Martin

[HiloDon]

Great results, Martin. I like the sum stacking feature as well. I try to follow the histogram with the black and white levels as it moves to the right so I can see the object develop. You can also drop the last stack if it's too much. Your NGC7479 is beautiful. You show the benefits of both types of stacking. I think that's why Paul wants to develop a hybrid one. I ran the same experiment as you did with M33 on the Bubble the other night and saw the same thing. The longer exposure brings out more detail and the level of brightness can be controlled better. I pushed it to 10x15s, but probably could have used more. It's great to see everyone experimenting and getting better results. I think my next experiment is to try a narrow band Ha filter on some nebulae. Astrogate Chris has done some impressive work with one and his mono Mallincam. I would like to see how it works with the Lodestar X2. Unfortunately the cheapest filter is $150US and the one I would like to get is $330US.

Thanks for sharing your results, Martin.

Don

[Paul81]

Great write-up and comparison Martin! Much better to visually see the differences compared to my technobabble in the tech notes. With your permission, could I use some of these in the next iteration of the notes (full credits of course)?

The sum stacking is great for the really faint stuff, I tend to use it quite a lot myself. Most of the time the extra noise doesn't bother me too much as it reveals some previously hidden details that I can look at and so my brain tends to filter out the noise.

The hybrid modes are still in the development frame, but taken a slight back seat for the colour correction. Once I am happy with that I intend to get back onto the case.

It is also really great so see the community coming together, helping each other out and doing experiments like these to generate tips and tricks to get the best out of the precious few clear skies we seem to get! 

[Martin Meredith]

Great results, Martin. I like the sum stacking feature as well. I try to follow the histogram with the black and white levels as it moves to the right so I can see the object develop. You can also drop the last stack if it's too much. Your NGC7479 is beautiful. You show the benefits of both types of stacking. I think that's why Paul wants to develop a hybrid one. I ran the same experiment as you did with M33 on the Bubble the other night and saw the same thing. The longer exposure brings out more detail and the level of brightness can be controlled better. I pushed it to 10x15s, but probably could have used more. It's great to see everyone experimenting and getting better results. I think my next experiment is to try a narrow band Ha filter on some nebulae. Astrogate Chris has done some impressive work with one and his mono Mallincam. I would like to see how it works with the Lodestar X2. Unfortunately the cheapest filter is $150US and the one I would like to get is $330US.

Thanks for sharing your results, Martin.

Don

Thanks Don. I look forward to your results in Ha if and when you obtain a filter.

Great observing site by the way! To follow the big professional scopes is a good plan if you can do it. I'd love to eventually retire in Tenerife, the Hawaii of Europe...

Martin

[Martin Meredith]

Great write-up and comparison Martin! Much better to visually see the differences compared to my technobabble in the tech notes. With your permission, could I use some of these in the next iteration of the notes (full credits of course)?

The sum stacking is great for the really faint stuff, I tend to use it quite a lot myself. Most of the time the extra noise doesn't bother me too much as it reveals some previously hidden details that I can look at and so my brain tends to filter out the noise.

The hybrid modes are still in the development frame, but taken a slight back seat for the colour correction. Once I am happy with that I intend to get back onto the case.

Cheers Paul. Sure, use whatever you like.

In theory SNR is the same for sum and mean stacking, leaving aside any clipping, but quantisation noise after sum stacking ought to be lower than for mean stacking. That's my understanding of where the benefits of sum stacking come from -- I could easily be completely wrong  .  It would be interesting to do an SNR/quantisation-noise analysis to predict the benefits of the hybrid modes too.

Martin

[Paul81]

Cheers Martin!

I'd have to puzzle that one through in my head regarding the SNR!   Visually, I'd expect to see more noise in the sum stacked image as the noise is accumulated (as is the signal). Mean / Median should do a better job at controlling random noise so give a better visual image (and less noise?), but any signal close to the noise limit will be harder to distinguish (but is easier with sum stacking), thus why sum stacking often allows you see fainter details.

I'm sure things were easier when all we had was a selection of eyepieces! Haha!

Paul

[Martin Meredith]

I'd have to puzzle that one through in my head regarding the SNR!   Visually, I'd expect to see more noise in the sum stacked image as the noise is accumulated (as is the signal). Mean / Median should do a better job at controlling random noise so give a better visual image (and less noise?), but any signal close to the noise limit will be harder to distinguish (but is easier with sum stacking), thus why sum stacking often allows you see fainter details.

My reasoning is that since sum is just number of subs*mean, then, in the absence of any other factors the SNR must be the same. Other factors are rounding, clipping, mapping from 16 bits to the display, … so if there is any benefit of sum stacking it must lie in those factors. The effect of these other factors will be changed/magnified by transformations such as gamma, contrast enhancement etc.

Reduction of quantisation noise strikes me as a possible cause but I'm not certain. If the signal of interest is weak and therefore occupies a low range of values, summing prior to the necessary rounding (for display purposes) ought to increase the effective number of bits; every 1 bit increase is typically worth a reduction of 6 dB in quantisation noise (at least for audio, and I assume also for images).

I will think some more about it….

Martin

[Martin Meredith]

Last night during a long session (mainly focused on Hicksons in Pegasus, Pisces and Piscis Austrinus) I had a chance to look further at sum vs mean stacking for NGC7479 and some other objects. The image above wasn't a fair comparison as the sum version was a stack of only 6 subs while the mean was a stack of 16. Below are zoomed in versions for stacks of 13 x 30s (top) and 20 x 30s (bottom) for sum stacking (left column) and mean stacking (right). I wouldn't normally stack so many but I wanted to see how long I could reasonably push sum stacking by reducing brightness to keep the histogram in check… the answer is quite a lot! In all cases I tried to make the comparison valid by spending quite a bit of time tweaking the controls for each case.

By my reckoning, we're seeing both more faint detail and less noise in the sum stacked version, especially for the condensations in the right-hand spiral arm and the two spiral arms on the left. By the way, the orientation has changed since my earlier post.

I also compared sum vs mean on two Hickson groups. First, HCG96 in Pegasus, with sum stacked on the left (8 x 30s) and mean stacked on the right.

The faintest galaxy (D) is mag 16.6.  I don't see much difference here, perhaps because most of the content is stellar rather than faint wispy stuff.

Similarly, here's HCG97 in Pisces:

Galaxy E is mag 16.3. Galaxies A and B appear to have a larger extent with sum stacking.

One down side of sum stacking is the difficulty in rejecting subs, since the display settings for the stacked version need to be changed to evaluate the most recent sub.

cheers

Martin

[Dom543]

My simplistic understanding is that the signal/noise ratio gets improved by the same degree by both sum and mean stacking. The fundamental reason for the improvement is that signal is appearing always at the same location, while random noise appears at different random locations on the frames to be stacked.

Assume that we stack 10 frames. In the case of sum stacking, at the signal locations the brightness get increased by 10 times as we add up the frames. On the other hand noise appears at all different pixels on the ten frames, hence its intensity will not be increased at any of those locations. So the signal/noise ratio improves 10:1.

In the case of mean stacking we add up 10 signal brightnesses at the signal locations and then divide the sum by ten. So signal level remains the same. Noise doesn't get added up as it appears at different locations on every frame, its level remains 1 but then gets divided by 10. So the S/N ratio becomes 1:0.1, which is the same as 10:1.

I know that this is over simplified but believe that still covers the core of the issue.

--Dom

[Martin Meredith]

Another image from the end of the last session: the Helix Nebula. Stacking (whether sum or mean I don't know) seems essential to get anything out of this very low surface brightness object. The panels below are based on sum-stacking with a pause after each 4 subs to try to get the best possible image (colour variations show that I didn't really succeed in this).

The top left panel is a single sub. This is the first time I've tried the Helix so I don't know if the poor quality speaks of the difficulty of the object, the 99% humidity and primary dewing at the point of capturing it, the relatively low altitude (25 deg) into the main direction of light pollution, or just my incompetence… some mix of them all no doubt. Anyway, the effects of stacking are very clear on the remaining panels (stacks of 4,8,12,16 and 20). Stacking also shows up the need to clean my sensor  . (I just noticed that the 12 x 30s panel is from a slightly earlier run that I restarted when I noticed the dew/light shield had slipped).

If you are searching for this visually or otherwise note that these are not zooms -- it really is a large object!

cheers

Martin

[HiloDon]

Hi Martin and All,

I had a good session last night and my last object before the rain came just happened to be the Helix. It was the first time I viewed it with the Lodestar, so I didn't know what to expect. I also wanted to test the comparison of sum v median stacking as Martin has been doing. I first did a median stack of 12 15 second exposures. The I did a sum stack of 12 15 second exposures. The final one is a sum stack of 20 15 second exposures.

I think this supports Martin's earlier finding that the sum stack shows less noise. I used median stacking to compare and probably should have tried mean as well. Don't know if there would be any difference.

Martin, I think you were running into some moisture problems with your Helix captures. I do think it needs quite a bit of exposure. I think my 20x sum stack was a little better than the 12x, but you be the judge. Let me know what you think.

After looking at the preview, I believe the noise difference is much more obvious when looking at the original on the computer. Must be something in the upload process.

Don

[Martin Meredith]

Hi Don

Excellent Helices! In yours it actually looks like a helix. The 20x is definitely the winner, but a lot comes down to making the right adjustments at the time I guess. 

Your median looks to be suffering a little graininess in the Helix itself compared to the sum version, which I suppose is to be expected.

I did a few simulations of sum vs mean stacking by measuring the SNR of a weak 1-D signal (a tone) in noise while varying quantisation, but the only way I could see a gain for sum stacking was under the assumption that the image is held internally at display resolution rather than at the full 16 bits. I will continue in odd spare moments.

I think humidity was part of my Helix problem. It's amazing how quickly it hits. I was observing again last night (having a great time in Draco and UMi and other points north) and 2 hours into the session it was still quite reasonable, at 85%; but then 15 minutes later it was up to 99% (I don't think the meter goes to 100% :-) That is generally my cue to pack in for the night (that and the laptop battery).

cheers

Martin

[Paul81]

Hi Gents,

I am enjoying reading about and looking at the stacking mode image comparisons - I have to admit I am learning a thing or two on best practice in the use of the various modes! 

Paul