Exposures to get unsaturated stars

[wimvb]

28 minutes ago, iantaylor2uk said:

If you use DSS for stacking, it rates the quality of each sub and you can tell DSS to stack the best 80% (or 70% or 90%, whatever you specify).

How does dss determine image quality? 

[pete_81]

Will have a quick look at that too @iantaylor2uk, thanks.
In the meantime, here's a quick edit with APP using the less cloudy images. Agreed @wimvb that it works much better (or just easier with my present workflow)
Thanks again to all. Processing workflow to be optimised now!

[wimvb]

4 hours ago, pete_81 said:

Good idea Wim,
Very quickly, can I ask again for advice here - here's a screenshot of the subs used

I'm going to assume for selecting the ones without cloud to use the darker ones - eg the range 014-042,then 056-080? Opening 025 in Affinity and 090 have very different histograms, unsurprisingly.

Out of interest on that also, is the histogram on the Left (image 025 in the above folder screenshot) OK position wise or should it be more to the left or right (is exposure OK or not) for future reference of where it should be placed?

(Taking the very bright images (looks like 30) still leaves 64 similar ones in the folder to process)
 

Take the darkest images. Clouds reflect terrestrial light, ie light pollution. The histogram shows that your light pollution is mostly in the red channel, with some also in green. Red + green = orange, which is the colour of sodium and mercury lamps.

The histogram on the left still has a fair gap on the left side. You can experiment with shorter exposures, but compensate by taking more. Remember, if you have more light coming in, you also get more noise. The light from light pollution will be subtracted during processing,  but the associated noise remains. Increasing the number of subs is always the best form of noise reduction you can apply.

[pete_81]

Hi @wimvb - So the histogram(s) in the previous post (1/3 into DR) were opening the individual subs in APP. If I open the stacked, "dimmer" images, the histogram shifts to the left and other than the brightest stars, there's not much else visible as the following screenshots show:

In light of the comments before, are the subs still too far to the right and should appear similar to the above (stacked may be even further left)? Why after stacking images that look the same are they darker? Is it purely the 14-bit output of the camera (where DSLR-Astro is quoting DR=10.3 "stops") being (4x ?) lower when DSS converts to 16-bit format (2^14 < 2^16)?
I stress again that through this, I'm seeing the benefit of shorter exposures - more can be taken during imaging session and clearly the data is there once processed, so my conclusion to starting out is still "take more shorter subs, ignore darks and dither". Bringing me back to comments before - I noted you commented this earlier, and also back in 2016 make reference to the same in my linked SL post:

On 22/10/2021 at 06:06, wimvb said:

That is the recommended amount of dithering. I believe that it was first suggested by Tony Hallas in a presentation he gave years ago. The presentation is still available on youtube: Tony Hallas astrophotography.

Using EKOS, the max dithering is I could set on Monday night was ≤10px. You commented back in 2016 that you are setting at 15px - are you using PHD2 to get this number or have I missed a setting in KStars/EKOS on the RPi?! (Any way to set this in EKOS or is 10 the limit?)

@The Lazy Astronomer and  @iantaylor2uk mentioned about 30-60sec subs and that AP is completely different to my regular photos (ETTR etc!), so is it fair to say that this is really the first (maybe most important?!) revelation to unlocking success in the hobby?

Thanks again guys

Edited October 27, 2021 by pete_81

[wimvb]

Those subs look good, with the histogram to the left, but a small gap so you don't black clip any data. The subs from my cooled camera look like yours, with only the brightest stars visible.

[wimvb]

6 hours ago, pete_81 said:

Using EKOS, the max dithering is I could set on Monday night was ≤10px. You commented back in 2016 that you are setting at 15px - are you using PHD2 to get this number or have I missed a setting in KStars/EKOS on the RPi?! (Any way to set this in EKOS or is 10 the limit?)

Back in 2016 I dithered manually, because I couldn't connect my dslr camera to a computer. One reason for the large dithering with non-cooled dslr cameras is that the images suffer from 'colour mottle' if not dithered enough. 'Colour mottle' is a term that Tony Hallas uses to describe large scale chominance noise. Cooled cmos cameras suffer less from this than dslr cameras, so there is no reason to use large dither steps. Now that I have a computer controlled cooled zwo camera, I set the dithering ta a larger than default value. It is not 15 pixels, but it works anyway, because my colour masters are clean and flat. Btw, if you set the dither scale in phd, it's the guide camera's image scale that is used. The dithering in the captured subs can be quite different.

[pete_81]

15 minutes ago, wimvb said:

Those subs look good, with the histogram to the left, but a small gap so you don't black clip any data. The subs from my cooled camera look like yours, with only the brightest stars visible.

This is the output from DSS - the individual subs were the previous histograms with the significant gap on LHS. Just to confirm:

Histogram from light subs with likely cloud reflection:

Histogram from light subs where there is (hopefully) no cloud reflection:

Histogram from the 16-bit stack from DSS (using only the light subs with the 2nd histogram, just above):

So query is, is the middle one OK or would you suggest it's too bright?

[wimvb]

2 hours ago, pete_81 said:

Histogram from the 16-bit stack from DSS (using only the light subs with the 2nd histogram, just above):

So query is, is the middle one OK or would you suggest it's too bright?

The middle one is ok. Personally, I would like to have it slightly more to the left, halving the gap between the diagram edge and the left side of the histogram, but that's a minor adjustment. If you do colour correction in the stacked image and in essence subtract the red glow, you subtract an equal amount from burnt out stars. They then end up with cyan star cores, irrespective of their natural colour. This is one of the problems with light pollution.

You need to have an exposure time that burries the read noise (read pattern). If you can see horizontal or vertical lines in your subs, that is read noise. But otherwise you can keep the exposure time short to avoid burnt out stars. With low read noise cooled cameras, you can have shorter exposure times and keep better star colour, just because of this.

[The Lazy Astronomer]

On 26/10/2021 at 22:22, wimvb said:

How does dss determine image quality? 

There isn't really any documentation about it, or at least, nothing that immediately popped up with a quick Google (of course, I could just be using the wrong search term), but I've always assumed it assigned the score based on some combination of the number of detected stars and the FWHM.

[wimvb]

7 hours ago, The Lazy Astronomer said:

There isn't really any documentation about it, or at least, nothing that immediately popped up with a quick Google (of course, I could just be using the wrong search term), but I've always assumed it assigned the score based on some combination of the number of detected stars and the FWHM.

This lack of transparency bugged me when I used DSS a long time ago. If you have it set to accept x%, then on a bad night you get x% of garbage, and on an excellent night you get x% of gems, while 100% would still have given you an excellent image with better noise characteristics. My guess would be that signal to noise ratio or amount of noise is also one of the criteria.

[vlaiv]

3 hours ago, wimvb said:

This lack of transparency bugged me when I used DSS a long time ago. If you have it set to accept x%, then on a bad night you get x% of garbage, and on an excellent night you get x% of gems, while 100% would still have given you an excellent image with better noise characteristics. My guess would be that signal to noise ratio or amount of noise is also one of the criteria.

I don't think you can sensibly measure SNR in a single image.

Simplest criteria and one that I believe DSS is using is background brightness. In fact - you have that data in file list. If LP is stronger - more unwanted signal will get in the image (higher background value) - and as that signal is removed associated noise remains.

If you want very good measure - you would do "linear fit" against reference image in form of A*image + B.

A is multiplicative constant that means sky transparency versus reference

B is additive constant that means LP levels versus reference image.

Higher A and lower B means better image in terms of SNR.

[pete_81]

Posted again on another thread about my experiences with getting new mount (following on from the GEM1)- this got me reading more into SNR calcs and using the site Jerry mentioned in the topic opening (SNR Calculator). Jerry also mentions that @vlaiv that you may have a spreadsheet to do some calculations? Care to share?
Clearly the greatest thing from what Jerry posts is dark(er) skies - I've a large factory about 100m south of home, which has a new LED floodlight lit all blooming night! That's clearly not going to do much for my imaging, but with visual, it's not too bad (I've a home-made light shield using photographic light stands and damp-proof which keeps light from the floodlight out of my eyes so I can at least think about 'dark adapting' even when just outside the back door).

I just wanted to query if I've entered things correctly (ish) - browsing online, I've managed to find stats for the T2i at QE40%, ReadNoise=2.5e and DarkCurrent=0.5e/s. Do these numbers sound sensible? The QE is fine, but querying the others. Also, filling in some targets, suggested that using 60sec subs and aiming for SNR of ≥10 (Brian's recommendations on the SNR calculator homepage), I'd need 10min of subs for M27 with mag7.5 vs 17hrs for NGC7000 with mag4??? So my original queries about number of subs/length of exposure really has no relation to magnitude? Or am I missing the point? I know @The Lazy Astronomer said that AP is completely different to daytime, but is it really that much?! Where there's no 'sense' between magnitude and number of subs?!

TIA

 

[vlaiv]

41 minutes ago, pete_81 said:

Jerry also mentions that @vlaiv that you may have a spreadsheet to do some calculations? Care to share?

Sure. Here it is:

SNRCalc-english.ods

Use LibreOffice to view it (maybe Excel works as well? Haven't tried it).

43 minutes ago, pete_81 said:

Where there's no 'sense' between magnitude and number of subs?!

There are two different calculations usually performed, and I'm not sure which one you are interested in. I'll briefly go over both and point out important bits.

1. Exposure length needed to swamp read noise

2. Total exposure needed to reach SNR (this is what above spreadsheet roughly calculates).

Let's explain first one.

Only difference between one long exposure and stack of shorter exposures that add up to same total time in terms of SNR is read noise. Or more precisely - how read noise relates to other noise sources in the image.

Other noise sources are - dark current noise, LP noise and target shot noise.

Usually dark current noise and target shot noise are quite low (cooled cameras and faint targets), but can be significant if target is rather bright (you don't really care in this case as SNR will be good with bright target) and if you are using DSLR camera as it does not have cooling. LP can also be significant and is major factor most of the time.

In any case - you want your read noise to be 3 to 5 lower than any other noise in your sub.

Both LP noise and dark current noise depend on respective signal - LP signal and dark current signal. Noise is equal to square root of signal (in electrons).

Here is an example of how to calculate for dark current. Your camera has 2.5e of read noise and Dark Current of 0.5e/s/px.

2.5e x 3 = 7.5

We want our dark current noise to be 7.5e or larger. This corresponds to dark current signal of 56.25e (square of noise). Dark current builds up 0.5e every second so we need 112.5s for it to build up to 56.25e

Based on dark current - I'd expose for max of 2 minutes per single exposure.

Similar thing goes for LP signal, but you need to either calculate it - using above spreadsheet - just input SQM value for your sky (can be measured with SQM meter or found online from lightpollutionmap.info or similar resource), or directly measure it from your images.

Later is more precise but requires you to know e/ADU for particular ISO setting. You simply take your image and measure background ADU units in single exposure (in channel of interest) and convert to electrons by multiplying with e/ADU - then take square root and you have your background noise. Just make sure your sub is properly calibrated (with darks to remove dark current signal).

Now on to second point / calculation

Time to reach target SNR. Very important point to understand is that there is no single SNR value in the image. There is no single SNR value for the target either - every pixel has its own SNR value. When we say - time to reach SNR - we need to make sure we take representative signal strength for the image we want to make.

You mentioned M27 and NGC7000 and gave mag7.5 and mag4 values. Those mag values are integrated magnitude values - total amount of light. We want surface brightness. Simple conversion would be to divide total light with surface of object - but that does not give you accurate results. That technique assumes that you have uniform distribution of light - that is never the case.

When talking about surface brightness - you need to estimate it - or have it measured precisely. I'll give you an example how things differ with respect to this.

Let's take "A Justifiable Replacement for M51 Whirlpool Galaxy" as an example (just saying what it says 😞

It says Mag 8.1 as total brightness. Then it says 21.45 as surface brightness. Second one is derived from the first one by dividing with rough size of object. Is that good estimate?

Not at all. Galaxies have really bright cores - in this case brighter than mag15 and depending on type of galaxy - brightness can fall of as fourth power of distance from the center. That means that outer parts of the galaxy are very very faint.

I'll give you two different measurements of M51 brightness:

This was taken by Roger Clark - and it gives sqm in red, green and blue. It goes down to just sqm23 due to SNR of the image - and it only reaches spiral arms and not outer parts of galaxy.

This one was actually done by me. You need sqm at least 24 for spiral parts and down to sqm26-27 if you hope to capture tidal tail of the galaxy. This is luminance brightness by the way.

So you see - you can falsely believe that Mag 21.45 is brightness that you should use - but in reality, you want to use Mag27 as your guide when imaging M51 - if you want to capture tidal tails and mag 24 if only wanting spiral structure.

Using integrated brightness - and dividing with surface will give you wrong results - as you've seen yourself.

Another important point is - what SNR you want to achieve. Aim at SNR >=5 if you want detail (like in spiral arms) and SNR>=3 if you just want to capture nebulosity without much detail.

[vlaiv]

Oh, I love this

Found another, more serious measurement of M51:

Central part is up to Mag24.6, then brighter part of tidal tails is 24.6-26.5 and outer parts are fainter than 26.5 (marked on the image as three different zones of black-red-yellow-white gradients).