Calling all the background masters

[broadsword]

Hello All,

I've recently posted some pics in my first Gallery album, and one early comment I've had relating to a cropped image of M51 is that there is loads more detail in there waiting to come out:

(sorry if the link doesn't work, it should be easy to find in my gallery album).

Well, I kind of already know that. The problem I face is that as soon as I apply a stretch to bring out the faint wispy part the background starts to show all manner of a problems, even after doing Dynamic Background Extraction in Pixinsight. Hence in the image linked above I had to crop my way out of trouble. So, unless I'm missing something I seem to have backgrounds that are hard to suppress without ruining the faint object detail.

I would however invite anyone that fancies it to see what they can do with this image - and, if successful, could they take me through the process?

Attached are the original stacked image, plus an image arising from a Pixinsight run through based on Harry's tutorials, both jpegs I'm afraid. The strange black beastie on the right is a bit of fluff that got caught under the DSLR mirror, btw.

Thanks in advance...

[nmoushon]

One trick I learned is that you do as much as you can with your levels first, with very very minor intermitant tweaking of curves. When you get as far as you can with levels then you move on to curves. This will help control the noise and keep it down while you pull more detail out. Look up some tutorials on levels and curves. Its surprising how much the little details of just these two tools can really help your image.

Also the fact is you are shooting at f6.3 and only 120sec subs. Though 120sec subs are good and you can get good amount of detail with them (as your image has shown) f6.3 isn't helping you much. You will just need a lot more subs. You only have 80min total. Trying to aim for 3 or 4 hrs. Especially at f6.3. You will be able to pull a lot more out.

[RikM]

Try saving from DSS with settings embedded but not applied and then process from scratch in PI or PS.

I had a quick play with the DSS output but couldn't do any better than your PI image. One problem was that the background was already stretched in DSS. I normally start from a essentially a black square. There are many ways to process images. I do a lot with curves as I find the exact opposite of nmoushon. Levels affects the image linearly, so lifting the object lifts the noise by the same amount. I can be much more selective using curves. I also use layer masks a lot to further target where I make changes to the image. Try both and see what suits you best, there is no right or wrong way, only the way that works for You.

[broadsword]

Thanks for the comments guys.

I actually used to do a lot more with levels, before I started to use PI. I moved the black slider up to the start of the histogram, as per the tutorials, but I found it took out faint pixels in the wispy areas and left them blotchy. I agree, it needs more frames, but this inage came from an experimentation phase and I now take a minimum of 1 hr 20 mins.

As to the DSS points Rik, I'm intrigued by the black square description. What I get directly from DSS before I tweak it (as I did in the image above) is usually very light - in fact the unmodified DSS image is shown below. How do you get your 'black square'?

I've started to mess about with masks, but I'm not sure what to do. I've used a kind of radial mask and blend function in GIMP, with poor results. Otherwise I've used a version of the luminance mask method shown in the popular tutorials on M42, but only for replacing blown out cores, not for accessing faint detail.

[RikM]

How do you get your 'black square'?

When you get the final stacked image, save it as a 16bit tiff and select the option to 'embed adjustments in the saved image but do not apply them'. Then when you open the saved image in e.g. Photoshop (something without a screen stretch function) you see a very dark image. A black square with just a few of the brightest stars showing.

This is what I get out of DSS

This is after stretching it in Photoshop

[broadsword]

Rik, looks good, I'll give it a try. I only used the DSS stretches anyway as a guide to any detail and colour in the image prior to full post processing.

Any thoughts on the mask question?

Thanks

[nmoushon]

Try saving from DSS with settings embedded but not applied and then process from scratch in PI or PS.

I had a quick play with the DSS output but couldn't do any better than your PI image. One problem was that the background was already stretched in DSS. I normally start from a essentially a black square. There are many ways to process images. I do a lot with curves as I find the exact opposite of nmoushon. Levels affects the image linearly, so lifting the object lifts the noise by the same amount. I can be much more selective using curves. I also use layer masks a lot to further target where I make changes to the image. Try both and see what suits you best, there is no right or wrong way, only the way that works for You.

Maybe I should explain a bit more. I use curves a lot. Just not at the beginning. I watched a tutorial where he explained (I'm paraphrasing here so sorry for any rough translations) that levels pull out more data thats in your image without "stretching" it. You effect the levels in the image that are already in the data. Where curves "stretch" the data beyond whats normally there. So you are able to pull more out and stretch it to specific points, like Rik mentioned. Curves is a great tool but I just use it after I use levels to pull the data till its at a good plateau then use I switch to curves to select where I set my dark/light points to where I want. Which is what makes curves great because you can pick and choose what to stretch.

So basically what this guy is saying is that levels pulls data out of your image while increasing noise minimally while curves will stretch your data to pull even more out but at the cost of increasing noise more than levels does. So his thought is to work with levels till you reach the plateau where you have pulled as much data out as you can without increasing the noise. Then you switch to curves where you can be specific and detailed at pulling out even more data. So you've moved your starting point from that basically black image to a more developed image.

[RikM]

Maybe I should explain a bit more. I use curves a lot. Just not at the beginning. I watched a tutorial where he explained (I'm paraphrasing here so sorry for any rough translations) that levels pull out more data thats in your image without "stretching" it. You effect the levels in the image that are already in the data. Where curves "stretch" the data beyond whats normally there. So you are able to pull more out and stretch it to specific points, like Rik mentioned. Curves is a great tool but I just use it after I use levels to pull the data till its at a good plateau then use I switch to curves to select where I set my dark/light points to where I want. Which is what makes curves great because you can pick and choose what to stretch.

So basically what this guy is saying is that levels pulls data out of your image while increasing noise minimally while curves will stretch your data to pull even more out but at the cost of increasing noise more than levels does. So his thought is to work with levels till you reach the plateau where you have pulled as much data out as you can without increasing the noise. Then you switch to curves where you can be specific and detailed at pulling out even more data. So you've moved your starting point from that basically black image to a more developed image.

I see what you are saying, and I know that a lot of people do it this way and get great results, it just doesn't work for me. I use levels to set the black point (I normally set it at 27 using the target colour/dropper tool) then in curves drop a control point to hold the background and then apply a 'standard curve' stretch above this, lifting quickly then flattening off to the top. Then basically repeat this in stages looking at contrast along the way.

I am interested why curves should introduce more noise than levels? My understanding of both tools is that you have certain values in the discrete brightness levels and when you 'stretch' an image, you move some of those values into higher brightness values. By stretching using the mid-slider in levels you can't stretch the lowest values as much, but this also means you can't lift the faintest signal as much as in curves, where you can essentially type in the exact value you want to lift.

I understand that some people could find stretching with levels easier than curves, but I don't see that it can be technically 'better'. If you have a link to more detailed info on this, or could explain why I would be very keen to learn more about it

Sorry for sidetracking a bit from your question Neil

Masks!

I use Photoshop but I think Gimp has the same functionality. Mostly what I do is: make a duplicate layer, make an adjustment on that layer (perhaps brightening it with curves, or adjusting the contrast to highlight the spiral arms against dark dust lanes), this adjustment will probably screw-up the stars or bring noise into the background or something...so I apply a 'hide all' layer mask, that makes the adjustments invisible. Then I take a soft round brush tool set to white and normally a low opacity 5 - 30%, make the mask active and then paint on the mask only where you want the adjustments to show through. Layers and Masks Rock!!!

[broadsword]

I see what you are saying, and I know that a lot of people do it this way and get great results, it just doesn't work for me. I use levels to set the black point (I normally set it at 27 using the target colour/dropper tool) faintest signal as much as in curves, where you can essentially type in the exact value you want to lift.

Masks!

I use Photoshop but I think Gimp has the same functionality. Mostly what I do is: make a duplicate layer, make an adjustment on that layer (perhaps brightening it with curves, or adjusting the contrast to highlight the spiral arms against dark dust lanes), this adjustment will probably screw-up the stars or bring noise into the background or something...so I apply a 'hide all' layer mask, that makes the adjustments invisible. Then I take a soft round brush tool set to white and normally a low opacity 5 - 30%, make the mask active and then paint on the mask only where you want the adjustments to show through. Layers and Masks Rock!!!

Hi Rik,

I noted your point about setting the black point in levels, I used to do something like this but I got comments on SGL about my pictures having lost a lot of data through clipping.....

On the masks, this looks like just the kind of thing I've been trying to do, but with the wrong method. I used the radial mask function in GIMP, which seemed good at first but on high res monitors it left a very strong light background halo around the object....a painted on option looks a much better bet in terms of putting the mask where you need it.

However, I'd be very keen to see what can be achieved by an expert, so if you had a spare few minutes to try my stacked image.......as you can see. tact is not my greatest strength....

[RikM]

I'm no expert, that's for sure but if you could let me have a tiff without the DSS adjustments applied, I will give it a go. I had a quick look at your DSS stack from post #4 and with the jpg compression, I can't even get close to your Pixinsight processed image from your first post. I can keep the background under control, but there is no detail in the galaxy, I just made a pixelated mess I am afraid.

You have a PM

[broadsword]

Hi Rik,

The tif. is about 30 Mb I think....I've never found a way of uploading things this big onto posts, any ideas?

I'll try when I get home...what's a PM btw?

[AndyUK]

Hi Neil,

(PM = Private message )

I've never found a way of uploading things this big onto posts, any ideas?

One way of doing this is to open a dropbox account (It's free!) and then you upload your files to there.

Once uploaded you can choose the share option which then gives enables you to either invite (email) a specific person to access the file or folder or you can "get link" which copies the URL to your clipboard. You can then post the link in your thread and others can then use that to download the full 30Mb+ file(s)

[broadsword]

Thanks for the advice Andy. When you say Dropbox do you mean something like Astrobin (I'm already on that!)?

[AndyUK]

Not quite - Dropbox will allow you to copy any files to it that you want of "almost" whatever size - I think they give you 2Gb to begin with, but if you invite others you get an extra 500Mb / referral up to 8Gb. Here's the link. I find it really useful when wanting to share exposures for co-op projects...

[RikM]

Well...I didn't have much success I am afraid

I found it really difficult starting with an already bright image. Essentially working backwards to darken the background without loosing the brightness in the DSO. If anyone has some ideas why DSS wouldn't save the unstretched tiff it would be good to find out.

What I did was:

- Open the image in Photoshop CS3

- Crop to get rid of the black blob and vignetting in the corners.

- Save and make a duplicate file, convert to greyscale, then back to RGB to make a mono luminance image.

- In levels, set the grey point dropper in the upper right corner (darkest part of the image), double click on the black point dropper to set the target values to 27 and then click in the same place as for the greypoint. This sets the darkest values in your image to 27 and I find makes it easier to process and see what you are doing. I adjust it again properly at the end.

- Run two passes of gradient Xterminator, coarse / high, then fine / low.

- Reset black point to 27 (but I don't think it changed.)

- I ran the deep space noise reduction action from Noel's Astronomy Tools but it just turned the backgound very patchy / blocky even if reduced in opacity or masked, so I undid this.

For background noise reduction instead, I duplicated the layer, added a quickmask, copied the image onto the mask, inverted it, used levels on the mask to make the background white and the galaxy black, blurred the mask, then clicking back onto the actual image, applied a Gaussian blur at about 2.5 pixels radius. This worked better but still wasn't perfect and I backed off the opacity to about 60%.

- Then I ran several iterations of curves to brighten the galaxy and enhance contrast. Using the 'hide-all layer mask and paint on the mask to let the detail show through' technique outlined in my earlier post.

- I used the same masking technique to apply a low opacity high-pass filter at 40pix and a second pass at 6pix to try and sharpen the dust lane detail (but I think it added more noise than detail, so would probably look at that again with different settings)

- Save the luminance as a PSD document.

Open the original cropped colour image again.

- Drop a colour sampler point into the top right corner and in levels, adjust the black points of the individual colour channels to be all the same value (turned out to be about no lower than 55 or I would have clipped the tail of the peak and started loosing faint data)

- Increased the saturation by 30 points and limited it to the bright parts with a layer mask in the same was as for the noise reduction blur earlier, but this time without inverting it. Masks work on the white bits and not on the black bits.

- Using the same mask again, adjust curves to try and get the galaxy the right colour. In the red channel set a control point on the galaxy core and on an outer spiral arm, then lift the core point by 5 and lower the arm point by 5, do the same again in the blue channel but this time lift the arm point and drop the core point. - didn't really do much, but sometimes this works very well.

- applied a 3pix gaussian blur to de-noise.

Save the RGB as a PSD file.

open the luminance image again, select all, copy, then paste it into your RGB image as a new layer.

Set the layer blending mode to luminosity and drop the opacity to 50%

Flatten the image, increase the saturation by 60 points (be brave )

Then I did run Noel's deep space noise reduction as a new layer on top and backed off to 60%

Paste in another copy of the luminance image, again set to luminosity and this time reduced to about 75% opacity and flatten.

Finally in levels I set the black point to about 20 and adjusted the mid point to 1.1 (this is my attempt at a correction value because when I upload to SGL, my images always seem to get a mighty extra stretch)

Resize to 1000 pixels wide and use the save for web option to save a jpeg copy optimised to 200kb in size and upload it to this post.

Takes longer to write than to do!

Sorry it's not come out better but as I said...I'm no expert

Seeing how much colour and low noise there is in your original PI processed image, I feel sure someone with more skill and knowledge than me could improve it

[broadsword]

Hi Rik,

Just got back from the pub and my head is reeling at all of this. I've obviously got a heck of a lot to learn. Will report back tomorrow.

Many, many thanks for the hard work, and looks like my first go at Dropbox worked!! Thanks Andy for suggesting this...

Neil

[russellhq]

Neil, I've been working on this target recently (http://stargazerslounge.com/topic/184851-m51-first-light/) and will give your image a go.

It would help if you posted them as PNG files as these are uncompressed. Photoshop has a handy tool for this- "Save for web" under the file menu.

[broadsword]

Hi Russell,

That's a coincidence, I was looking at your M51 about 2 hours ago, very nice it is too.

Thanks for offering to have a go.

Here is the DSS stacked image, converted to a PNG with GIMP - hope it works OK.

[russellhq]

I've had a quick go but really struggling to get any more dust out of it!

I'll have another go in the morning as it's time for bed now.

[RikM]

Much better star colour in this one from the PNG but I still can't do much better I am afraid

[russellhq]

Ok, it's taken a bit of a battering but here's my effort.

[broadsword]

Hi Russell,

Ah, now, this is really interesting, as you've done an incredible job (way better than me)getting all if the faint wispy gas out whilst keeping the noise down....but now the background has blobs of whatever and some glow at the edges, which is where I came in on with the thread. So how to get the background flat and dark, and not lose that gas? I can only be clever masking, as Rik was describing.....

[russellhq]

Yeah, I created a mask using threshold to isolate the galaxy which allowed me to bring up the brightness while keeping the background low.

[lensman57]

Hello All,

I've recently posted some pics in my first Gallery album, and one early comment I've had relating to a cropped image of M51 is that there is loads more detail in there waiting to come out:

http://stargazerslou...galaxy-cropped/

(sorry if the link doesn't work, it should be easy to find in my gallery album).

Well, I kind of already know that. The problem I face is that as soon as I apply a stretch to bring out the faint wispy part the background starts to show all manner of a problems, even after doing Dynamic Background Extraction in Pixinsight. Hence in the image linked above I had to crop my way out of trouble. So, unless I'm missing something I seem to have backgrounds that are hard to suppress without ruining the faint object detail.

I would however invite anyone that fancies it to see what they can do with this image - and, if successful, could they take me through the process?

Attached are the original stacked image, plus an image arising from a Pixinsight run through based on Harry's tutorials, both jpegs I'm afraid. The strange black beastie on the right is a bit of fluff that got caught under the DSLR mirror, btw.

Thanks in advance...

Hi Neil,

I saw the post and couldn't resist having a little play with the data. living in Manchester doesn't give a lot of opportunity for imaging at the best of times so as a Newbie I am just practising on other peoples data, hope you don't mind.

Regards,

A.G

M51 DSS PNG ST2 CS2.tif

[broadsword]

Hi AG,

Absolutely, no problem at all, although I can't open the file yet on my iPad!

Hey, you're only living a few miles from where I took my pics from, is it light pollution that's the problem? I used to think LP ruined all my stuff, but I've persevered and there are many ways round it...unless you just mean the Manc weather of course...