More data is better

[pyrasanth]

The image below shows 4 hours of HA data on the left and 12 hours on the right. This clearly illustrates why more data is better. Notice how the 12 hours of data is going to be far easier to reprocess as the background is not as grainy. Both images have had similar processing run on them so more is better.

This is a useful evaluation to see how your image is progressing. The 12 hours of data will now be reprocessed into the final layer for the new image of M101 being worked on.

 

[wimvb]

Very informative comparison. Longer integration time just means less noise, so more information can be pulled out. Is the image you're working on going to be colour?

[laudropb]

You have certainly shown just what extra data can achieve.

[pyrasanth]

1 hour ago, wimvb said:

Very informative comparison. Longer integration time just means less noise, so more information can be pulled out. Is the image you're working on going to be colour?

Yes I plan to begin shooting the L data on the next clear moonless night. I am planning a minimum of 20 hours in the L channel & 5-6 hours in each of the RGB channels. The L will be at 1x1 bin & the RGB at 2x2 bin. The total image integration time will be about 60 hours of data in total including the already captured HA but might not get finished until the autumn.

[The Admiral]

Interesting comparison pyrasanth. Did these have the same sub length, or did you increase both sub length and sub number?

Ian

[pyrasanth]

1 hour ago, The Admiral said:

Interesting comparison pyrasanth. Did these have the same sub length, or did you increase both sub length and sub number?

Ian

Hi Ian- The subs were all the same at 30 minutes each. There are a couple of 20 minute subs but these were more for initial testing. I left these in the stack so the left image has 6 x 30 mins + 2 x 20 mins & the right stack 19 x 30 + 2 x 20 minutes.

It was interesting to see how the middle of the galaxy filled out with data as I initially thought that the middle was just noisy but realised that I was actually capturing data later on.

[wimvb]

15 hours ago, pyrasanth said:

Yes I plan to begin shooting the L data on the next clear moonless night. [...]

Isn't that a contradiction in terms?

[Herzy]

I do see the difference, it's very obvious, but shouldn't 3x the amount of data be a VERY significant difference? Your 12 hour image certainly is much smoother and cleaner then the 4h one but I can't see that much more detail. :/ 

[pyrasanth]

On 5/4/2016 at 16:45, Herzy said:

I do see the difference, it's very obvious, but shouldn't 3x the amount of data be a VERY significant difference? Your 12 hour image certainly is much smoother and cleaner then the 4h one but I can't see that much more detail. :/ 

Sadly the differences are not incremental they are the square root of the number of frames for noise reduction. The images shown are very rough so should not be taken as the "bible" they were just to give an idea. 49 images have seven times less noise than 1 sub so it's easy to see why the law of diminishing returns quickly rears its ugly head. You would have to take 100 frames to reduce noise by a factor of 10 where only 49 frames reduce by a factor of 7. Detail will always be limited by seeing conditions and aperture so perhaps your seeing all the detail I could get at the time!

[wimvb]

I would say there is a very significant improvement for the longer integration time; the weak parts of the galaxy are much clearer. More data and (therefore) less noise allows for more separation of weak signal from the background.

[StuartJPP]

Diminishing returns does indeed come in to play with respect to SNR, but even then the difference between a factor of 7x and a factor of 10x is still a difference. Most of the exceptionally good images are in the many hours of exposure and to get them is painful, but to stop is to give in early. I will admit that once you get to 128 subs, going to 256 subs is a long slog and probably time to give up....In my mind it is better to get one exceptional image rather than many mediocre ones, not always easy in the UK of course but you might as well spend the whole night on a single target rather than many targets (unless it isn't possible due to the target's position).

Personally I'd try to aim for in the region of 60+ subs of whatever length I am capable of getting under the sky conditions. The images are much cleaner and hence easier to work with. That way the noise in the images is generally not the limiting factor.

The thing that isn't usually covered in all of this is the total exposure time. The more total exposure you also get, the more likely you are to capture those photons that arrive very infrequently, the ones that will normally be thrown away as noise or end up looking like noise.

It isn't just a simple calculation that one size fits all, otherwise this wouldn't be coming up again and again. There are MANY variables and the two I find king are the total exposure time and ensuring that there are enough subs so that statistical methods can be used to reduce the SNR of the combined stack. That all assumes using the same kit of course.

I.e. for my setup...(notice that the total exposure time is the same for all):

1 sub of 8 hours = bad
8 subs of 1 hour = bad but better
16 subs of 30 minutes = better
32 subs of 15 minutes = getting there
64 subs of 7.5 minutes = best.

I could be totally wrong of course, usually I am. But if the theory is correct why is this so heavily debated? Surely it should be plug in your values and out pops the answer.

[strutsinaction]

Coincidentally, Kayron Mercieca (Light Vortex Astronomy) did a similar experiment with M101, Ha 3nm subs and SNR:

http://www.lightvortexastronomy.com/blog/improving-snr-in-m101-pinwheel-galaxy-in-hydrogen-alpha-3nm

There's nothing surprising here but it's interesting to see the visual comparison (presented as an animated gif).

Regards
John

[StuartJPP]

1 hour ago, strutsinaction said:

Coincidentally, Kayron Mercieca (Light Vortex Astronomy) did a similar experiment with M101, Ha 3nm subs and SNR:

http://www.lightvortexastronomy.com/blog/improving-snr-in-m101-pinwheel-galaxy-in-hydrogen-alpha-3nm

There's nothing surprising here but it's interesting to see the visual comparison (presented as an animated gif).

Regards
John

The thing is that you aren't going to notice the difference (or it is very slight) between 45 and 50 subs, but you will notice the difference between 45 and 90 subs. This is the point. You basically have to double the number of exposures to get a reasonable increase in SNR (1.414x).

It is diminishing returns, but it is still a return, the increase in SNR never reaches zero (though may come close), if you want the cleanest image then you need to take enough exposures, period.

When using Lucky Imaging, why do some people use 1000's of subs? They don't stop at 50...but that brings us back on to total exposure time as well, they are all connected.

[wimvb]

50 minutes ago, StuartJPP said:

The thing is that you aren't going to notice the difference (or it is very slight) between 45 and 50 subs, but you will notice the difference between 45 and 90 subs. This is the point. You basically have to double the number of exposures to get a reasonable increase in SNR (1.414x).

It is diminishing returns, but it is still a return, the increase in SNR never reaches zero (though may come close), if you want the cleanest image then you need to take enough exposures, period.

When using Lucky Imaging, why do some people use 1000's of subs? They don't stop at 50...but that brings us back on to total exposure time as well, they are all connected.

Up to a point. Each individual sub also contributes with read noise. To decrease noise by taking many subs, you have to make sure that read noise is low. This is why lucky imaging demands cameras wiht low read noise.

Noise is not the only determining factor in lucky imaging. By keeping the exposure time short, you beat seeing conditions. During lucky imaging, the sharpest subs are kept, which will give you more detail.

 

[StuartJPP]

I agree fully that one factor alone is not the only determining factor in anything related to this genre everything is connected but if you don't expose for long enough, don't expect anything decent. Note that "long enough" depends on all the factors for your particular imaging rig.

My point is that if it wasn't always so cloudy, people would be exposing for a lot longer than they currently do rather than making up excuses for not doing so. We'd also know where the point of diminishing return was with empirical data rather than speculation, hyperbole, statistics and damn lies

 

[ollypenrice]

The majority of theorists I've read seem to think that after around 40 subs the diminishing returns syndrome sets in severely. Tony Hallas puts it lower than that. I haven't tried going beyond the mid thirties and even that would be exceptional for me. However, I'm a fan of the long sub. So rather than going for 60x15 minutes I'd go for 30x30 minutes. You need a dark enough sky, of course. I find that, going for faint stuff, 24x30 minute subs seems to find what is there to be found. It needed less than that to find the structured outer shell of M31.

On the other hand fast frame imaging does seem to benefit by going for much deeper stacks.

As ever, suck it and see.

Olly

[StuartJPP]

That's 12 hours Olly much more than some people's 64x5min subs...total exposure time still wins though as you have to have subs long enough to capture the infrequent photons and you also need enough of those subs to get something useful statistically.

[The Admiral]

Let me stick my neck out here. Notwithstanding all the discussion above, which I wouldn't want to diminish for a second, if I was to present you with this:

would the benefits of a 3x exposure duration be so immediately apparent? Those well versed in the art might well screw up their faces in horror, but I would contend that the differences now appear rather more subtle, certainly to the casual observer. To me, this represents ever more the problem of diminishing returns. The left-hand picture here is the same as the one in the first picture, but tweaked a bit in Lightroom (as if you couldn't guess). And don't forget, these are starting with low res. images. In the pursuit of excellence the more exposure the better, clearly, but for me, I'd take the 4h and run with it.

I'll get me coat

Ian