Colour Preserving Stretch in Photoshop

[sharkmelley]

Some of you may know that I wrote the recently released Arcsinh Stretch function in PixInsight:  http://pixinsight.com/doc/tools/ArcsinhStretch/ArcsinhStretch.html

It has frustrated me that it has been very difficult to do something similar in Photoshop.  So with a bit of maths I've come up with a way of taking linear stacked data and performing a powerful stretch on it in Photoshop whilst preserving the original star colour.  I'll explain how to do this in Photoshop CS2 which I suspect is the most common version that folk are using.  There are minor differences for more recent versions of Photoshop

First I'm assuming that DeepSkyStacker (or your chosen stacking program) has produced stacked linear data (i.e. data with no stretches performed on it) and that you have applied white balance and background subtraction.  Here's an example image of the North America Nebula you can download and play with: 

https://drive.google.com/open?id=0B3Ky5pyZvsINdmswNG9NSmluX00    I'll use this image in the demo below.

1) Open the image in Photoshop

2) In the Layers menu right click and Duplicate Layer.  Name this new layer "Grey Multiply" as a helpful hint to the role it will play.

3) Click on the "Grey Multiply" layer and go to Image->Adjustments->ChannelMixer.  For each Output Channel (Red, Green, Blue) set the source channels to be 33% each then hit OK.  This converts the layer to grey.  Don't be tempted to click on "Monochrome" instead because in CS2 this does something subtly different and will cause later steps to fail.

4) Right click the "Grey Multiply" layer and Duplicate Layer.  Name this new layer "Grey Divide", again as a helpful hint to the role it will play.

5) For the "Grey Multiply" layer set the blend mode to "Multiply"

6) For the "Grey Divide" layer set the blend mode to "Divide".  But "Divide" doesn't exist as a blending mode in CS2 so we use the trick that mathematically Colour Dodge on an inverted image does the same thing as Divide (trust me!).  So in CS2 set the blend mode to "Colour Dodge" then select the "Grey Divide" layer and do Image->Adjustments->Invert

You should now have a layers menu that looks like this:

 

The main image window will not have changed.  All you can see is a few stars:

 

We've now set up the trick so it's time to perform the real magic.  We need to apply a powerful stretch to the "Grey Multiply" layer.  Whatever stretch we apply ends up being applied to the original image whilst preserving the colour and saturation.  The shape of the stretch curve is crucial and Arcsinh (Hyperbolic arcsine) is ideal.  Photoshop doesn't have this curve but it can apply a gamma curve, which is very similar.  We can then "tweak" it afterwards.  So on to step 7:

7) Select the "Grey Multiply" layer and goto Image->Adjustments->Exposure.  In recent versions of Photoshop you want to apply a gamma of around 2.5   However it is done backwards in CS2 so in CS2 you apply a gamma of 0.4 instead (which is the reciprocal of 2.5).  Whichever way round it is in your version of Photoshop you should end up with something like this:

Make sure you are viewing your image with a scale of 66% or 100% otherwise you'll see a complete posterised mess instead. I do wish Photoshop wouldn't behave like that.  We are now getting pretty close to the final result.  Just one more "Curves" tweak is required to kill the residual background and to increase the stretch.

8) Select the "Grey Multiply" layer and goto Image->Adjustments->Curves.  Look at the shape of the cure I've created. The flat section of curve at the bottom left removes the residual background then the steeply rising section gives the extra stretch required:

The final result I obtained by applying the above curve is here:

Note how well the faint detail has been stretched whilst preserving colour in all the stars.  Now you can apply any final steps such as increasing saturation, removing remaining background etc.

The steps to perform this colour preserving stretch might seem a bit awkward and non-intuitive at first but you'll soon get used to it.  Remember that you need to get the white balance more or less correct before you start and you must already have subtracted as much as possible of the sky fog background.

Finally don't be afraid to experiment with different values of gamma and various curve shapes in the "Grey Multiply" layer.  Different data is likely to respond well to different values.

Enjoy!

Mark

Edited October 12, 2017 by sharkmelley

[Davey-T]

Sounds  good Mark, I'm still resisting getting PI, could do without spending any more money on astro but I fear that ship has sailed 

Will bookmark this to study another day.

Dave

[wimvb]

Great write up. Glad I have PixInsight; only a few sliders and no layers.

Thanks for the script, Mark.

[carastro]

Brilliant Mark, it works and I don't have to buy Pixinsight.

Carole 

[sharkmelley]

5 hours ago, carastro said:

Brilliant Mark, it works and I don't have to buy Pixinsight.

Carole 

Hi Carole,

I knew you would like it

Mark

[swag72]

Interesting!! I'll give this a go soon  

[ollypenrice]

Thanks Mark, one to have a play with!

Olly

[newbie alert]

Thanks Mark.. something to play with at a later date ...

[StargeezerTim]

Thanks for sharing this. Will give it a go. Would it work on Ha stacks or extracted red channels?

[Xiga]

This looks great! I'm a PS guy so I can't wait to have a go at using this.

Quick question. The example you gave used just 1 gamma adjustment and 1 curve. Normally when i do my stretching I do each one in small to medium amounts, so it can take quite a few iterations before I'm done. How would the gamma adjustment work in such a case, should it always be in the region of 2.5 or would it need to be lowered significantly? 

I'll do some experimenting myself to be sure but would like to hear from the expert all the same ?

[sharkmelley]

12 hours ago, Xiga said:

This looks great! I'm a PS guy so I can't wait to have a go at using this.

Quick question. The example you gave used just 1 gamma adjustment and 1 curve. Normally when i do my stretching I do each one in small to medium amounts, so it can take quite a few iterations before I'm done. How would the gamma adjustment work in such a case, should it always be in the region of 2.5 or would it need to be lowered significantly? 

I'll do some experimenting myself to be sure but would like to hear from the expert all the same ?

That's a good question.  The whole point of applying gamma is that it is a very powerful curve.  It's difficult to reproduce anything like the gamma curve using manual curves functions.   With a gamma curve applied you simply won't need to perform a whole succession of small to medium iterative adjustments.  One or two curve adjustments is sufficient once the gamma has been applied.

Typically the images I work on are very deep, so a gamma of 2.5 is about right.  With images that are less deep and therefore require less stretch, apply less gamma.

In later versions of Photoshop gamma can be applied as a separate adjustment layer Layer->NewAdjustmentLayer->Exposure.  Here's an approach using Photoshop CC:

As before, the layers "Grey" and "Grey Divide" are monochrome duplicates of the original image.  The "Grey Divide" layer has its blend mode set to Divide and the "Multiply Group" layer has its blend mode set to Multiply.  All other layers have their blend mode set to Normal.  Because the "Grey" layer is put into a group with added Gamma layer and Curves layer(s) it is much easier to achieve a better level of control over the stretch because the gamma layer and the curve layer(s) can be adjusted independently .

The crucial factor behind this colour preserving stretch is that the original colour image is multiplied by a (stretched) grey image and divided by a grey image.  Therefore the colours of the original image (i.e. the ratios of red, green and blue) are completely preserved and do not become washed out or bleached during the stretch. 

By the way, I've just discovered what I was doing wrong in CS2 in step 3 - the conversion to monochrome.  Click the "Monochrome" box and set the RGB channels to 33% each or use a different mix.  CS2 defaulted to 100% Red which made a mess of later steps.

Mark

 

 

 

Edited October 15, 2017 by sharkmelley

[Xiga]

11 hours ago, sharkmelley said:

That's a good question.  The whole point of applying gamma is that it is a very powerful curve.  It's difficult to reproduce anything like the gamma curve using manual curves functions.   With a gamma curve applied you simply won't need to perform a whole succession of small to medium iterative adjustments.  One or two curve adjustments is sufficient once the gamma has been applied.

Typically the images I work on are very deep, so a gamma of 2.5 is about right.  With images that are less deep and therefore require less stretch, apply less gamma.

In later versions of Photoshop gamma can be applied as a separate adjustment layer Layer->NewAdjustmentLayer->Exposure.  Here's an approach using Photoshop CC:

As before, the layers "Grey" and "Grey Divide" are monochrome duplicates of the original image.  The "Grey Divide" layer has its blend mode set to Divide and the "Multiply Group" layer has its blend mode set to Multiply.  All other layers have their blend mode set to Normal.  Because the "Grey" layer is put into a group with added Gamma layer and Curves layer(s) it is much easier to achieve a better level of control over the stretch because the gamma layer and the curve layer(s) can be adjusted independently .

The crucial factor behind this colour preserving stretch is that the original colour image is multiplied by a (stretched) grey image and divided by a grey image.  Therefore the colours of the original image (i.e. the ratios of red, green and blue) are completely preserved and do not become washed out or bleached during the stretch. 

By the way, I've just discovered what I was doing wrong in CS2 in step 3 - the conversion to monochrome.  Click the "Monochrome" box and set the RGB channels to 33% each or use a different mix.  CS2 defaulted to 100% Red which made a mess of later steps.

Mark

 

 

 

Top stuff Mark. Thanks for explaining that, it makes perfect sense now. Hopefully I'll get a chance to test this out on some old M33 data tonight. If I do I'll try and post up a comparison to just doing a bog standard stretch. 

[Xiga]

On 15/10/2017 at 01:25, sharkmelley said:

That's a good question.  The whole point of applying gamma is that it is a very powerful curve.  It's difficult to reproduce anything like the gamma curve using manual curves functions.   With a gamma curve applied you simply won't need to perform a whole succession of small to medium iterative adjustments.  One or two curve adjustments is sufficient once the gamma has been applied.

Typically the images I work on are very deep, so a gamma of 2.5 is about right.  With images that are less deep and therefore require less stretch, apply less gamma.

In later versions of Photoshop gamma can be applied as a separate adjustment layer Layer->NewAdjustmentLayer->Exposure.  Here's an approach using Photoshop CC:

 

Hi Mark

I had a go at this last night but I couldn't get anywhere with it. I also have PS CC and my steps looked exactly as per your picture above, but all I could get was a massively blown out galaxy and very strange stars, everything was basically white. I forgot to take a screen grab of how it was looking, doh!

Do you have any ideas what was going wrong? Would it help if I posted a link to the stack?

[sharkmelley]

10 hours ago, Xiga said:

Hi Mark

I had a go at this last night but I couldn't get anywhere with it. I also have PS CC and my steps looked exactly as per your picture above, but all I could get was a massively blown out galaxy and very strange stars, everything was basically white. I forgot to take a screen grab of how it was looking, doh!

Do you have any ideas what was going wrong? Would it help if I posted a link to the stack?

Is it that because of that annoying Photoshop problem where it shows you a very badly posterised version of the image unless you set an image scale of 66% or 100% ?

If it's not that then post a link to the stacked TIF and I'll be very happy to take a look.

Mark

Edited October 16, 2017 by sharkmelley

[Xiga]

4 hours ago, sharkmelley said:

Is it that because of that annoying Photoshop problem where it shows you a very badly posterised version of the image unless you set an image scale of 66% or 100% ?

If it's not that then post a link to the stacked TIF and I'll be very happy to take a look.

Mark

Hi Mark

Okay i had another go and actually had some success this time. And WOW, what a difference this colour-preserved stretch makes! 

I'll cut straight to the chase and post 2 pics below. The 1st one uses a regular stretch using multiple iterations of curves, and the 2nd one uses your colour-preserved stretch method. I tried to get the histograms as close as possible on both. Although there was a very slight difference, it didn't affect the comparison. These are crops of an M33 stack i did last year, and the stack had already had gradient reduction and colour balance applied. 

The difference is nothing short of remarkable. Right throughout, the colour of everything has been maintained. I used a Gamma of 2.5 as you suggested, and i think i used 2 or 3 curves. What's more, i was expecting to see an increase in the background colour noise, but it's the exact opposite! The red blotches are significantly reduced and, for whatever reason, the green blotches actually look like they are now 'missing' from the image. Honestly, looking at the image, i would have guessed that HLVG had been ran on it, but it hasn't! Is this normal behaviour for a stretch like this?

I did find a quirk in getting the process to actually work, i'll write another post next to explain.

Amazing work Mark to figure all this out for us PS guys who haven't moved over to P.I yet (in my case due to cost). 

[Xiga]

4 hours ago, sharkmelley said:

Is it that because of that annoying Photoshop problem where it shows you a very badly posterised version of the image unless you set an image scale of 66% or 100% ?

If it's not that then post a link to the stacked TIF and I'll be very happy to take a look.

Mark

You were quite right Mark, PS was indeed doing the weird posterization thing on me. Although on my system it was a little different, it would show things correctly at any zoom level above 63%, not just at 66% and 100% specifically. I'm using PS 2014 CC. 

I tried lowering my cache down to Level 1 but, while it did indeed get rid of the posterization, i found the overall slowness unbearable tbh. And i'm using an SSD as well! In fact PS even crashed on me while i was using it on Level 1 cache. That's the 1st crash i've ever had in the 3 or so years i've been using it, so i had to go back to Level 8 myself. It wasn't that big of a deal though as all i had to do was keep the zoom level above 63%

As far as actually getting the stretch to work properly, the issue i was having originally was that i wasn't grouping the layers (Grey Multiply, Exposure, Curves) as you showed in your last picture. I was just changing the Blending Mode of the Grey Multiply Layer to 'Multiply' as you said in your original post, but it wouldn't work for me until i grouped them all together and set the overall Blending Mode to 'Multiply' (the default was Pass-Through and it also didn't work for me). Anyways, just thought i'd let you know, and thanks again for this, it will completely revolutionise how i process my images from now on!

ps - I'm just noticing some slightly pinkish star cores. Got any solutions how to fix that?

Edited October 16, 2017 by Xiga

[StargeezerTim]

It's not working for me I'm afraid. Whenever I apply any gamma it just goes far too white. I'm using CC and tried the original instructions and the CC update with no success. 

Heres a screengrab... The layer names may be different but the top layer is set to 'divide' and the layer group set to multiply with the one above the bottom on normal. No posterization on my computer so I don't know what is going on!

[Xiga]

20 minutes ago, StargeezerTim said:

It's not working for me I'm afraid. Whenever I apply any gamma it just goes far too white. I'm using CC and tried the original instructions and the CC update with no success. 

Heres a screengrab... The layer names may be different but the top layer is set to 'divide' and the layer group set to multiply with the one above the bottom on normal. No posterization on my computer so I don't know what is going on!

I'm not sure if this is the reason Tim, but it looks to me like you've done your Exposure and Curves the wrong way round. You want the Exposure Layer showing as being *below* the curve layers.

[sharkmelley]

5 hours ago, Xiga said:

 I was just changing the Blending Mode of the Grey Multiply Layer to 'Multiply' as you said in your original post, but it wouldn't work for me until i grouped them all together and set the overall Blending Mode to 'Multiply' (the default was Pass-Through and it also didn't work for me). Anyways, just thought i'd let you know, and thanks again for this, it will completely revolutionise how i process my images from now on!

ps - I'm just noticing some slightly pinkish star cores. Got any solutions how to fix that?

That's right - if you use a group for the "Multiply" then it's the overall blending mode of the group that must be set to multiply.

The pink star cores are where the star has saturated the sensor.  When the white balance is applied, it is usually the blue and red channels that are increased, making the saturated area pink.  The colour preserving stretch faithfully preserves this pink.  My approach to this is to perform a Levels operation on the white balanced image, reducing the white point until the star core becomes white.  Use this adjusted image as the starting point for the colour preserving stretch.

Mark

 

[StargeezerTim]

I'm still having no luck Mark. Where do you have the black point on the source image? I usually start off with the black point about 30 from the left. Does it matter for this? 

Also, for the Gamma curve,, do you adjust the 'exposure' slider or the 'Gamma correction' slider?

The only half reasonable results I'm getting is giving a pinkish rendition and is not as good as my normal curves approach. Also do you adjust the black point back on levels after the Gamma and curves applications?

I attach the file I am using (obtained in poor seeing). If you could confirm the data is suitable for your approach I'd be grateful!

Thanks... Tim. 

Autosave converted.tif

[sharkmelley]

9 hours ago, StargeezerTim said:

I'm still having no luck Mark. Where do you have the black point on the source image? I usually start off with the black point about 30 from the left. Does it matter for this? 

Also, for the Gamma curve,, do you adjust the 'exposure' slider or the 'Gamma correction' slider?

 

Hi Tim,

The black point is crucial - I should have explained that better.  In your example I did an Image->Adjustments->Levels and pushed the black slider to 62 before starting.  Generally speaking, push it as far as you can without clipping.  For the gamma curve, it is the Gamma Correction slider that you need.  Using a gamma of 2.5 and one Curves adjustment in the "Multiply" layer gave me this:

Regards,

Mark

 

[carastro]

I have posted this on another thread, but it seems sensible and relevant to post it here to show comparisons.

This is my M20 image processed in my normal way a few months ago.

This is the same image I reprocessed today using Mark's Arcsinh method.  I am well pleased with the improvement, I always struggled with star colours, but it's even produced a better overall colour in the image.

[StargeezerTim]

1 hour ago, sharkmelley said:

Hi Tim,

The black point is crucial - I should have explained that better.  In your example I did an Image->Adjustments->Levels and pushed the black slider to 62 before starting.  Generally speaking, push it as far as you can without clipping.  For the gamma curve, it is the Gamma Correction slider that you need.  Using a gamma of 2.5 and one Curves adjustment in the "Multiply" layer gave me this:

Regards,

Mark

 

Thanks Mark... do you tweak/reset the black point between the gamma and curve applications?

[sharkmelley]

2 hours ago, StargeezerTim said:

Thanks Mark... do you tweak/reset the black point between the gamma and curve applications?

That's right.  Just like I did in step 8 of the original post - take a look at the shape of the curve I used.  You could equally well perform a Levels operation to adjust the black point followed by a Curves operation.

Mark

Edited October 17, 2017 by sharkmelley

[Uplooker]

Mark,

I am no image processor, not even in your wildest dreams. I have just followed your instruction and using the data you provided. That is nothing short of magical. To be fair I did not really understand anything of what I was doing, just by rote. It really is fabulous. Thank you so much.

I loved showing the original image to my love and then doing the 2.5 Gamma bit and hearing her go "Wow"