Does signal strength depend on exposure time and / or number of stacked subs?

[vlaiv]

This is actually almost a trick question - but I feel one that needs to be answered, as I believe many people doing imaging don't in fact know the proper answer to it.

In recent discussion with @Rodd about ins and outs of stacking and SNR - this came up and it occurred to me that many people would not be able to answer it properly. Here is question in a nut shell:

Given same scope, same camera, same target and sky conditions, with only signal in mind (disregard noise for the moment), how do signal levels compare in following cases:

1. Single one minute exposure

2. Average stack of 10 one minute exposures

3. Sum stack of 10 one minute exposures

4. One 10 minute exposure

What do you think?

 

[Merlin66]

1. Has the lowest target signal and low SNR

2. Has an improved target signal, but similar to 1. due to the averaging, but an improved SNR (1 sigma)

3. Has a much higher target signal due to sum but similar SNR to 2.

4. Similar to 3.

 

[vlaiv]

12 minutes ago, Merlin66 said:

1. Has the lowest target signal and low SNR

2. Has an improved target signal, but similar to 1. due to the averaging, but an improved SNR (1 sigma)

3. Has a much higher target signal due to sum but similar SNR to 2.

4. Similar to 3.

 

Here is a hint:

Let's leave noise part aside for a moment as it is related to measurement. We are not talking about measurement here - we are talking about signal strength regardless of measurement.

[Rodd]

I’ll be the judge 

Rodd

[AngryDonkey]

My gut feeling is that the signal level will stay the same, regardless of how long we record it or how it is stacked (as long as the signal is recorded in the given exposure time). So just considering signal level (and ignoring any noise or other technicalities of recording it) then I could 'stretch or amplify' the signal of one minute and it would theoretically be the same as the signal of ten minutes...

Yes/No? 😀

Would 'signal level' in this context be equivalent to photon flux?

Edited November 19, 2019 by AngryDonkey
added a question

[Merlin66]

If the sensor records say 10 photons in one minute, then over ten minutes it will receive and record 10 x 10 photons.

 

[AngryDonkey]

4 minutes ago, Merlin66 said:

If the sensor records say 10 photons in one minute, then over ten minutes it will receive and record 10 x 10 photons.

True you record more photons, but what I am saying is that if we only consider the signal (and this will stay the same) then I can just take my signal after one minute and multiply it by 10 and I will have the same result as recording it for 10 minutes.

Edited November 19, 2019 by AngryDonkey

[ollypenrice]

I'm watching with interest but removing the noise from the exercise reminds me of a joke about a statistical formula for betting and winning on the horses. The formula is perfectly correct... but only works for spherical horses racing in a vacuum. 

lly

[fireballxl5]

1 and 2 give the same signal level

3 and 4 give the same signal level

In each comparison there will be some spatial variation in signal intensity across the image plane, but peak signal levels should be roughly the same.

CS, Andy

[vlaiv]

4 hours ago, Merlin66 said:

If the sensor records say 10 photons in one minute, then over ten minutes it will receive and record 10 x 10 photons.

 

This is very good start for understanding

We have 10 photons per minute and we have 100 photons per 10 minutes. Are those two different?

The reason I said that we should leave noise for now is because noise is related to measurement. It is not "intrinsic" property of signal. Signal is what it is. Act of measurement introduces noise in resulting numerical value, and the way you measure will impact how much noise there will be.

If you measure above signal for one minute - you will conclude that it is in fact 0.1666 photons / second, but the noise associated with measurement will be related to how long you performed measurement for - SNR will be 3.1622776.... (if we imagine perfect system - no read noise and such). Once you measure for 10 minutes - you will also conclude that signal is in fact 0.166666 / second - but noise associated with measurement is less - SNR will be 10.

Same goes for stacking with average - measured signal will be the same (but noise will be reduced due to repeated measurement).

Point being - there is no sub that contains no signal - all subs contain signal and they contain exactly the same signal, even 0.0001s exposure sub will contain that signal. Difference is only in the noise associated with each measurement. We could argue that numerical value that we got from measurement is also different - but that is really not important - it's a bit like saying I'm "27 tall" (ok, but what units, can we compare that to someone else's height?), or maybe my speed is 27 kilometers (per minute? per hour? per day?)

I think above way of thinking is the key for understanding that it is all about SNR and that measured numerical value can be arbitrary large or small - it will not matter as long as we are happy with level of noise in our measurement.

[AngryDonkey]

Sorry for being thick but are you saying that what I said was true or not? 🤣

[vlaiv]

9 minutes ago, AngryDonkey said:

Sorry for being thick but are you saying that what I said was true or not? 🤣

Indeed it is - at least in the way I'm advocating for people to think about signal / image.

One might argue that we use word signal to describe different things, and indeed I'm guilty of that in many previous discussions on topic of signal and signal/noise ratios and such (sometimes we take shortcuts and don't express ourselves properly).

Problem with that approach is: If we think of signal as measured value and not intrinsic property of the source - we could be arriving to wrong conclusions. Namely following couple seems to be common:

- You can't record image / display image with low measured values, or rather image that has average numerical value of 1 will be somehow better quality than image that has average numerical value of 0.0001. This can stop one seeing that it is not absolute values that are important but rather ratios of measured values (one pixel being twice the value of another and three times value of third and fourth ...). Image is relative pixel values - regardless of absolute values that we assign to each of the pixels as long as we maintain their ratio (describe them in compatible units - be that photons per hour or per millisecond) - in fact to make image presentable we often give up their proper ratios and introduce non linear transform (but that is extension to this topic).

- Some subs contain no signal. Suppose that we have source that for our purposes emits one photon per hour and we take one minute subs. As many subs would end up with measured value of 0 - if we associate word signal with measured value as opposed with intrinsic property of source - we can conclude that many of those subs are in fact wasted - they did not capture anything - no "signal" is present on them - but we would be wrong in thinking so. If we follow that line of thought - nothing would prevent us from discarding such subs (as they are meaningless contain no signal) - however if one does that - they will end up with wrong image.

From previous point, extension (that can sometimes be heard) - you can't record target that is too dim for single exposure, or you can't record signal that is below noise level in single sub.

In any case - if we associate term signal - with intrinsic property of object (or shall I say object / setup relationship) and understand that each sub contains the same signal, it opens up possibility of easier understanding of the whole process of image acquisition and also histogram manipulation. It can help one understand what image really is (in data / pixel values sense and their relative ratios). It also helps understand signal/noise ratio - and how ultimately that is only important thing for a good image.

[Merlin66]

11 hours ago, vlaiv said:

- Some subs contain no signal. Suppose that we have source that for our purposes emits one photon per hour and we take one minute subs. As many subs would end up with measured value of 0 - if we associate word signal with measured value as opposed with intrinsic property of source - we can conclude that many of those subs are in fact wasted - they did not capture anything - no "signal" is present on them - but we would be wrong in thinking so. If we follow that line of thought - nothing would prevent us from discarding such subs (as they are meaningless contain no signal) - however if one does that - they will end up with wrong image.

Why would we have the "wrong image"??? 

[JamesF]

11 hours ago, vlaiv said:

From previous point, extension (that can sometimes be heard) - you can't record target that is too dim for single exposure, or you can't record signal that is below noise level in single sub.

I'm thinking, perhaps wrongly, that there ought to be a further condition on this, that the recorded signal must be greater than some property of the noise level (variance, perhaps), otherwise you can't distinguish it from the noise?

James

[Physopto]

10 photons in 1 minute, or 100 photons in 10 minutes. You cannot just multiply the 1 minute result to get a 10 minute sub. The photons are random in placement so do not all fall in the same positions when recorded. So a 10 minute sub is filling in the blanks so to speak, resulting in a smoother overall picture. Hence a 100 minute sub will produce a much more substantially smoothed image. Several then can be added to increase the overall image intensity. 

Derek

[Merlin66]

2 minutes ago, Physopto said:

The photons are random in placement so do not all fall in the same positions when recorded.

Derek,

Hmmm I accept the arrival of the photons is random (shot noise) but they don't vary in position....I believe, based on our spectroscopy work, a combination of subs with some signal give a increased intensity and improved SNR. How else can I record the spectra of faint stars???

 

[Physopto]

2 minutes ago, Merlin66 said:

Derek,

Hmmm I accept the arrival of the photons is random (shot noise) but they don't vary in position....I believe, based on our spectroscopy work, a combination of subs with some signal give a increased intensity and improved SNR. How else can I record the spectra of faint stars???

 

Photon are random in arrival and position. It takes time for a picture to become clear. It depends upon the signal intensity from the source. Physics 101. Look in any undergraduate physics book for the explanation. This is why our pictures of the stars and nebula can take so many hours to accomplish a final acceptable image. In daylight a photograph is possible in milliseconds because there are so many photons arriving.  🙂

Derek

[JamesF]

I believe vlaiv might just be considering the signal for a single photosite.  Or the signal from that part of the sky covered by a single photosite.

James

[andrew s]

Photons are Fermions and follow fermi dirac  statistics  which for optical wavelengths approximates to poisson statistics. The "arrival times" are correlated in that the if a photon is detected it increases tha probability another will be in a given time interval. This effect is significant at radio wavelengths. 

Given @vlaiv comments his question is unanswerable without a clear definition  of his terms. 

Regards Andrew 

Sorry bosons and Bose Einstein statistics 

Edited November 20, 2019 by andrew s
Mistake

[Merlin66]

Yeah,

I'm only doing spectroscopy....

I'm interested in the target star - not the whole field.

In the spectroscope slit, normally 3-4 pixel wide and covering 90% of the FWHM image, it's the photons I can collect which makes the difference.

I believe in this application, the "arrival times" are important.

 

Edited November 19, 2019 by Merlin66

[Physopto]

Er no, photons are not Fermions, they are Bosons. Fermions have half interger spins, photons have a hole number, normally! In experiments it can be shown that photons can be produced with half interger spins, I.e act like Fermions. This is caused by the half-quantisation of the total optical angular momentum..

As far as Merlin66 needs, a star is so bright, photon arrive very quickly in droves, so I can't see a problem, short exposures I would guess are fine. Never tried what you are doing, very interesting work. For faint objects it can take a long time for enough photons to arrive so long exposures to get enough photons into the ccd wells producing electron counts.

[andrew s]

40 minutes ago, Physopto said:

Er no, photons are not Fermions, they are Bosons. Fermions have half interger spins, photons have a hole number, normally! In experiments it can be shown that photons can be produced with half interger spins, I.e act like Fermions. This is caused by the half-quantisation of the total optical angular momentum..

As far as Merlin66 needs, a star is so bright, photon arrive very quickly in droves, so I can't see a problem, short exposures I would guess are fine. Never tried what you are doing, very interesting work. For faint objects it can take a long time for enough photons to arrive so long exposures to get enough photons into the ccd wells producing electron counts.

Opps quite right boson and Bose Einstein statistics. Not sure what I was thinking but still approximates to poisson statistics with clumping 😱

Regards Andrew 

[noah4x4]

By coincidence, this has recently been extensively debated in Cloudy Nights. What emerged was a conclusion that for every different rig there is an optimal signal to noise ratio/limit that can be reached by applying various combinations of multiple stacked exposures x the SAME aggregate time. That optimal ratio can however differ from evening to evening dependent on sky conditions. Whatever ratio(s) works one night might be less effective on another. 

I find this is particularly notable on Hyperstar that (say) converts an f/10 OTA to f/2 and is hence 25x faster. Frankly, I can't see any tangible difference in my results  between 60 x 5 second stacks when compared to 5 x 60 second stacks (both five minutes of total acquisition), except that the latter demands overcoming greater technical challenges to avoid the impact of field rotation, but that might also additionally be attributable to my Atik Horizon being exceptional low noise and my eyesight not being wonderful. But on another night, the total acquisition time to produce the same result(s) might be longer or shorter, dependent on local sky conditions. The good thing about being able to embrace extreme short exposures (say 10 seconds) is you have time to try a number of combinations in any single session.

Where this is material to me is that at f/2 and below 20 second exposures I can discard my wedge and I don't need polar alignment or autoguiding. So stacking more x shorter subs works best for me as it is not too technically challenging. At my age weak eyesight and sciatica means less knob twiddling is good.  I can also be set up in a few minutes between persistent UK clouds. Hyperstar has been a real game-changer for me and I can also see why RASA scopes are popular. Total acquisition time seems to be more important than the length of individual subs. But it seems that each rig will differ from another as to where the optimal ratio lies, and each night's conditions will require some adjustment.

[vlaiv]

I had a feeling that this sort of thinking will cause much confusion.

Idea was to try to introduce a sort of mathematical thinking rather than anecdotal when it comes what signal is, what measurement gives and above all importance of SNR and how it relates to imaging.

11 hours ago, Merlin66 said:

Why would we have the "wrong image"??? 

You are right - I stand corrected. Excluding some subs with measured 0 value will not change image - it will only change SNR of resulting stack - same as if we choose to discard any sub regardless what the measured value is.

It will however skew our measurement if we want to determine absolute value (not important for image but it is important if one tries to measure photon flux).

Average of 0,0,0,1,1 = 2/5 is not the same as average of 1,1 = 2/2 = 1. Discarding some measurements just because they are zero introduces bias error.

11 hours ago, JamesF said:

I'm thinking, perhaps wrongly, that there ought to be a further condition on this, that the recorded signal must be greater than some property of the noise level (variance, perhaps), otherwise you can't distinguish it from the noise?

James

I was referring to the statement that I've heard couple times before, and it goes:

"You can't record something that is fainter than background noise."

That is in fact not true. Suppose that we have some galaxy that is faint and signal from it is less than read noise of the camera. In fact it is so faint that there are some subs that contain no photons from that galaxy.

Regardless of above, signal of the galaxy is recorded on each frame - with certain error. Stack enough such subs and you will be able to get the noise down so that SNR is enough to see that galaxy.

It is not out of reach by virtue of being below noise in single sub.

Let's say that SNR in single sub is something like 0.1 on average - galaxy is 10 times fainter than noise in that sub. You need something like 3-5 SNR to be able to detect galaxy. Stack 10000 such subs and resulting SNR for that galaxy will be 10 - you will be able to see it clearly.

11 hours ago, Physopto said:

10 photons in 1 minute, or 100 photons in 10 minutes. You cannot just multiply the 1 minute result to get a 10 minute sub. The photons are random in placement so do not all fall in the same positions when recorded. So a 10 minute sub is filling in the blanks so to speak, resulting in a smoother overall picture. Hence a 100 minute sub will produce a much more substantially smoothed image. Several then can be added to increase the overall image intensity. 

Derek

No one said that you can take 1 minute sub and multiply it with 10 and get 10 minute sub.

You can take measured signal from 1 minute sub and multiply it with 10 and you will get measured signal in 10 minute sub (roughly there will be uncertainty in measured value). Problem is that there is noise associated with each measurement and when you multiply your 1 minute sub - you will multiply measured values - which are not pure, but polluted with measurement noise - so you will end up with same measured value of signal but different noise value - both in sense it will be random but also in numerical value.

100 minute sub will produce smoother result not because signal suddenly became stronger just because we've chosen to do measurement differently. Noise level is associated with measurement, and the way we measure changes noise level - this is why result is smoother image.

For image intensity - you can choose to add measured values together or you can choose to multiply single sub with some constant. Both will change intensity of the image. What we are interested in is SNR. That is why we stack - not to increase image intensity but to boost SNR.

2 hours ago, andrew s said:

Opps quite right boson and Bose Einstein statistics. Not sure what I was thinking but still approximates to poisson statistics with clumping 😱

Regards Andrew 

Ok, I'm so confused with this one.

Most of the light that we are dealing with when taking images of astronomical objects is in fact thermal light (star light), but according to sources, Bose Einstein distribution has SNR of:

This means that it is always less than 1, regardless of how much exposure time we use?

[andrew s]

@vlaiv photon detection is a fundamentally quantum process. You can't think of a star emitting photons which travel to earth and are then detected on earth. This mixes the QM and classical views.

In QM there is a probabilty that photons are created at the star and given that a probability the will be detected (annihilated) here on earth. This is process is fundamentally probabilistic. Measurements over time will converge to the average  but will always retain some uncertainty for any real example. 

Regards Andrew 

Edited November 20, 2019 by andrew s