M101 Processing Help

[Herzy]

I feel pretty discouraged because I spent 8 hours on this galaxy and the images are all still noisy and ugly. I thought I might be able to pull out some nice detail with 8 hours, but I guess not. It would mean a lot if someone with better processing skills could give this a try. That way, I might know if it's my data that is lacking or if it's my processing ability that's lacking. This was several hundred 2 minute exposures at ISO 1600. The galaxy was just barely visible (only the core) in the single frames.

https://www.dropbox.com/s/696q1vophzzj0cg/M101 Combined Stack.tiff?dl=0

Thanks,

Hayden

[frugal]

Doing a very quick crop and stretch in PixInsight, there seems to be a lot of streaking lines from upper right to lower left. To me they look regular noise. What calibration of the individual frames was done before the stacking?

[Oddsocks]

On 03/04/2017 at 01:19, Herzy said:

spent 8 hours on this galaxy and the images are all still noisy and ugly.

Hi Hayden

As with Frugal's post above.

What program did you use to stack the frames and what settings were used to combine the subs?

Have you used calibration frames? at the very least you should be creating an artificial Master Dark from a minimum of approx' 80 zero-time (minimum exposure time at same ISO as lights) Bias frames, using "Average" combination, use this artificial "Master Dark" in place of conventional dark frames which are rather pointless with an uncooled DSLR.

If you used DSS then check under Settings > Stacking Parameters, the default for DSS is that "Average" combination is used but this is not the best for DSLR frames that contain lots of noise. Try a stacking combination parameter that ignores pixels that fall outside the mean and replaces them with Sigma clipped or a weighted average. Try a stacking combination using Kappa-Sigma clipping as the default method, you will need to play around with the Kappa value and number of iterations to find what works best for you but begin with the default setting of Kappa = 2.00 and number of iterations = 5, be aware that these routines add greatly to the processing time so if you are using a low end laptop or PC that these operations will take a while. If the result is still a very noisy and streaked background then try the Median Kappa-Sigma clipping option or Auto Adaptive Weighted Average to determine which option works best with your data.

Looking at your image I suspect that the RA axis is aligned with the direction the streaks rather than having the horizontal axis of the camera aligned perfectly with the RA axis of the mount and that the streaks are created by slight differences between frames in the RA direction. Zooming into the brighter stars (and ignoring the coma) shows all the stars have a slight oval deformation and split colour artefact on either side of the star that is consistent with the direction of the streaking.

M101 is a very hard target for an uncooled camera, the galaxy has a very low surface brightness and taking lots of short subs with a DSLR will still struggle to bring the galaxy out of the camera's intrinsic background noise so you have to help by using calibration frames and choose an appropriate combination routine. Once the background noise is removed then you will be able to stretch and colour balance the result of the combination routine much more easily.

 

 

[Herzy]

I will try these. My first stack was only a weak reject algorithm. I'm a little disappointed because I thought the point of dithering was to remove the diagonal streaks, but I dithered all night and it still didn't work. I took flats, but no darks (my camera battery died the first night. I'll try to stack in DSS this time.

[Oddsocks]

3 minutes ago, Herzy said:

I thought the point of dithering was to remove the diagonal streaks

Dithering is used to remove streaks and artefacts caused by hot pixels and only works with some kind of Sigma clipping applied, dithering won't remove noise, you can only do this with calibration frames.

For a DSLR don't bother collecting darks as they vary too much with sensor temperature over which you have little control, use a master-bias-as-dark, just shoot off a load of dark bias frames at minimum exposure time using the same ISO you used for the lights, combine these using an averaging combination routine, call the resulting image "Master Dark" and use this as a replacement for a conventional dark.

In your image there are several groups of dithered hot pixels that have not been removed so whatever reject routine you used it was not sufficient.

[wimvb]

I'm processing your image as I write.

The tiff you shared is a typical example of underexposure (been there, done that). Taking more subs will never increase the signal, it can only help reduce the noise. However, if the noise is read noise or fixed pattern noise, it won't be removed completely, no matter how many images you stack.

I wonder how much you dithered. The minimum dither distance should be 12 - 15 pixels on BOTH axes. If you dither manually, that is easy to achieve. But if you dither through software, and you set a random step length with a maximum of 12 pixels, you dither on average only 6 pixels. Backlash will also diminish the effect of dithering.

As for hot pixel removal in stacking. DSS has a setting called "cosmetic correction" which can remove hot pixels in the calibrated subs prior to stacking. If your dark frames are not enough to remove hot pixels, you can apply the correction to remove residual hot pixels. Also lower the rejection limits in the clipping routine (= more pixels rejected). This can reduce the streaks as well. I wouldn't rule out darks completely. If you take them under the same conditions (time and temperature) as the light frames, you may get them to work. An alternative is to create a bad pixel map that the stacking software can use to remove hot pixels. (I don't know if DSS can use these)

For future images: increase the exposure time (and decrease the ISO to decrease noise). If your setup allows, aim for 4 - 5 minutes exposures per sub.

I will post the processed image here later.

Cheers,

[wimvb]

A rather quick process:

DBE

colour calibration

noise reduction (TGV on Luminance, MMT on colour)

Stretch

Colour saturation with mask protecting the background

Colour desaturation with inverted mask, protecting the galaxies and stars

Local contrast enhancement in the main galaxy

Curves tweaking

Star reduction

All in PixInsight

[Herzy]

This is the best I could do. I tried stacking in DSS. The streaks are all gone, but the for some reason I can't pull any data out of that stack. Wim, you managed to make the galaxy very bright, which is great. I can't seem to stretch it much without the noise being overwhelming.

M101 Final.tif

[Herzy]

3 hours ago, wimvb said:

I'm processing your image as I write.

The tiff you shared is a typical example of underexposure (been there, done that). Taking more subs will never increase the signal, it can only help reduce the noise. However, if the noise is read noise or fixed pattern noise, it won't be removed completely, no matter how many images you stack.

I wonder how much you dithered. The minimum dither distance should be 12 - 15 pixels on BOTH axes. If you dither manually, that is easy to achieve. But if you dither through software, and you set a random step length with a maximum of 12 pixels, you dither on average only 6 pixels. Backlash will also diminish the effect of dithering.

As for hot pixel removal in stacking. DSS has a setting called "cosmetic correction" which can remove hot pixels in the calibrated subs prior to stacking. If your dark frames are not enough to remove hot pixels, you can apply the correction to remove residual hot pixels. Also lower the rejection limits in the clipping routine (= more pixels rejected). This can reduce the streaks as well. I wouldn't rule out darks completely. If you take them under the same conditions (time and temperature) as the light frames, you may get them to work. An alternative is to create a bad pixel map that the stacking software can use to remove hot pixels. (I don't know if DSS can use these)

For future images: increase the exposure time (and decrease the ISO to decrease noise). If your setup allows, aim for 4 - 5 minutes exposures per sub.

I will post the processed image here later.

Cheers,

That is an interesting read. My single frames were already very bright. (2m at ISO 1600). My mount can't do any longer unguided. When I eventually start guiding, I might be able to do 5 minutes at ISO 400. I don't see why that would help though. I might be capturing more signal, but I am also capturing more light pollution. The effects should cancel each-other out? 

[wimvb]

The optical system can't differentiate between signal and light pollution or sky glow. But the camera will most likely produce less noise at lower iso. If frames are underexposed, camera noise becomes more evident. That's why you'd want exposures that are sky limited, and not camera limited. Experiment with different settings, until you find the sweet spot. Unfortunately, skies are getting lighter the closer we get to summer, an sky glow will increase.

I probably overstretched your original image a bit. Processing software should come with a stop sign. 

[Oddsocks]

4 hours ago, Herzy said:

My single frames were already very bright. (2m at ISO 1600). My mount can't do any longer unguided. When I eventually start guiding, I might be able to do 5 minutes at ISO 400. I don't see why that would help though.

This is an interesting subject that comes up over and over in imaging forums.

My understanding, and this may be wrong, is that in DSLR's ISO is not the same as ISO in film.

In film, ISO is down to a difference in chemical sensitivities between film emulsions and it was always true that as ISO rating increased so sensitivity to light increased but at the same time "clumping" of the light sensitive crystals in the film emulsion occured during development that gave the final image a grainy or noisy appearance. Lower ISO was less sensitive to light but less "clumping" occured and the final image had less noise. In film, a long exposure time, low ISO image always looked less noisy than a short exposure time high ISO image.

In a DSLR, the CMOS sensor maintains the same basic sensitivity to light falling upon it and the ISO rating only changes the way that the post-exposure sensor-reading hardware and software applies a variable gain setting when reading out and storing the image for subsequent display. Increasing the ISO rating makes the captured image appear brighter by applying a strong gain and gamma during sensor reading but does not differentiate between background noise and wanted image so in a way similar to film the final image may appear brighter but noiser with a high ISO setting.

In Astrophotography we selectively stretch and curve any images we capture and we are able to manually choose the black level cut-off where stretch begins and in this sense using a high ISO setting is not so useful or desirable for us. At the time of capture a "bright" image is not needed or desirable except when finding and focussing our subject. When we post-process the image we want a low noise image, not necessarily a bright image.

Since all DSLR cameras differ in the ammount of noise versus ISO setting and this depends of camera temperature, speed of optics etc it is important to experiment with your own camera settings rather than follow by rote what may be generally stated as an "ideal" ISO to use. You already know the limit to exposure time defined by your mount so using that limit time and taking a series of exposures at differing ISO settings of the same object, ignore initial image brightness as a quality indicator but stretch and compare the results, look in the low contrast areas of the image and choose the ISO setting that gives the lowest noise then look in the brightest areas and check for saturation. If saturated keep the ISO setting that gives lowest noise and adjust the exposure time down, if not saturated, and even if the image appears dim do not be tempted to increase ISO, aim for minimum noise and use post proccessing to restore image brightness.

While writing this reply I noticed Wim's reply appear which is saying the same thing but more succinctly, I did think about deleting all this but may be there is still something useful here.

 

[wimvb]

There is, so thanks for not hitting the delete key.

The comparison between film and sensor is a usefull one.

A sensor has a completely different light response than film, of course. Still there are enough parallells to merit a comparison. At the same time the differences should also be highlighted. I may muse more on this later on, when I get home.

[alacant]

On 3/4/2017 at 16:48, Herzy said:

I dithered all night

I think you've done well to get this far with 2 minute frames only.

If you are not guiding, then maybe the mount is not moving sufficiently between exposures. You'd have to move it manually or by using a camera and mount control program such as APT.

I think the streaks may be caused by tracking errors streaking the noisy pixels. For dither to work, the mount must only be moved between exposures, not constantly during them.

+1 for Oddsocks' comment: you can remove a lot of noise by not using dark frames. Light, flat and bias only give good results with Canon sensors. HTH.