Who wants to play with my data....

[chubster1302]

I've been watching loads of processing tutorials but I'm now at the stage where I'm just moving sliders and not really having a clue what I'm doing or trying to actually achieve. Does anyone want to have a play with this and show me what can, if anything, be brought out from this image.  At least then I will have a benchmark to try and aim for 

r_pp_light_stacked.fit

[symmetal]

Here's my quick go with PI and PS. 🙂 It looks like I wasn't careful enough with DBE as the bottom of the image has a darkish band running across it and the centre is a bit blue. I hope that gives you some idea of what's there.

Alan

[chubster1302]

That's amazing, I have no idea how you managed to bring out so much of the nebula...

I'm using siril to do the initial stacking and stretching, I can see you removed some of the stars, can I ask at what point in the process would you do this. I have starnet++ for Siril

[symmetal]

In PixInsight I use Star Exteminator (star removal) as the last step on the linear data, after dynamic background extraction, blur exterminator, noise exterminator and photo metric colour calibation (with background neutralization).

I then stretch the background with Generalised Hyperbolic Stretch, and stretch the separate stars image slightly in GHS with emphasis on preserving colour similar to an arcsinh stretch. These are saved as 16 bit tiff files to load into PS where the stars are added in to the background as a separate layer using 'Screen' blend mode.

Then final adjustments using curves and satutration in PS, separately on the two layers to get the final look before the layers are combined into a finished image. 🙂

To make DBE in PI easier, I do  star removal to give just the background so DBE points can be added quickly without having to worry about stars being in the points. After DBE has done its magic I add the stars back in before continuing with the rest of the linear processing.

I'm not sure if starnet++ will work on linear data, in which case a partial stretch may be required before using it.

Alan

Edited July 12, 2023 by symmetal

[symmetal]

I've given it another go to see how far it can be pushed. Probably gone too far here. 😁

The dark band near the bottom is on your original data and is likely the flats not quite matching.

Alan

[wimvb]

I played around in PI with your image, and came up with this.

Your image has a common DSLR signature: colour mottle. Tony Halas did a presentation on DSLR astrophotography almost ten years ago, where he discussed this. His remedy: dither aggressively, about 12 pixels at least. Here's the link to his presentation

https://www.youtube.com/watch?v=PZoCJBLAYEs&t=3s

 

Edited July 13, 2023 by wimvb

[Fegato]

I've had a quick go, nothing better than Alan has produced above. Background extraction is a bit of a pain with an image like this as there's so much nebulosity, so I didn't bother, and the end results shows there is a bit of a gradient reducing from right to left! Anyway, a couple of observations:

1. There's a fair amount of noise. You didn't describe your equipment (including filter(s)), exposures and sky conditions - would be interesting to know. Looking at the data, I would guess maybe the exposures were fairly long (stars a bit blown out here and there), but you do need more data?

2. As Alan says - there is a bit of a darkish band across the bottom. If you stretch very hard, there's maybe one across the middle too, but not totally sure. It's difficult to be sure when there is all that dark stuff following a similar path. If it is an artifact of some sort, I don't know what could cause this - assume you have no obstructions in the light path? Anyway, best take some other images and see if it repeats before getting too worried about it.

[Elp]

Here's my dream like rendition, took 1h 12m to do mostly manual in Siril (PP), GIMP and PS, as most have said the noise is an issue which has caused a lot of mottling:

[chubster1302]

2 hours ago, wimvb said:

I played around in PI with your image, and came up with this.

Your image has a common DSLR signature: colour mottle. Tony Halas did a presentation on DSLR astrophotography almost ten years ago, where he discussed this. His remedy: dither aggressively, about 12 pixels at least. Here's the link to his presentation

https://www.youtube.com/watch?v=PZoCJBLAYEs&t=3s

 

 

Interesting watch, thanks. I did dither every 3rd frame when acquiring the subs via NINA, is that the same thing ? 🤔

[chubster1302]

Cheers everyone for the advice and showing me what is there, I still cant get anywhere near the examples above but ill keep experimenting. 

The data for this was taken over two nights, the first session be really bad, just waiting for another clear night to add some better subs. 

[chubster1302]

1 hour ago, Fegato said:

I've had a quick go, nothing better than Alan has produced above. Background extraction is a bit of a pain with an image like this as there's so much nebulosity, so I didn't bother, and the end results shows there is a bit of a gradient reducing from right to left! Anyway, a couple of observations:

1. There's a fair amount of noise. You didn't describe your equipment (including filter(s)), exposures and sky conditions - would be interesting to know. Looking at the data, I would guess maybe the exposures were fairly long (stars a bit blown out here and there), but you do need more data?

2. As Alan says - there is a bit of a darkish band across the bottom. If you stretch very hard, there's maybe one across the middle too, but not totally sure. It's difficult to be sure when there is all that dark stuff following a similar path. If it is an artifact of some sort, I don't know what could cause this - assume you have no obstructions in the light path? Anyway, best take some other images and see if it repeats before getting too worried about it.

D5300 unmodded on a William Optics ZS 73iii, no filters. Guiding with PHD2. 3min subs at 800iso. I "thought" the sky conditions were OK but as this graph shows, that really wasn't the case. 🙈

[wimvb]

1 hour ago, chubster1302 said:

Interesting watch, thanks. I did dither every 3rd frame when acquiring the subs via NINA, is that the same thing ? 🤔

yes, but how many pixels did you dither? I believe that your image scale is 1.8 "/pixel. A 12 pixel dither would be about 22 arc seconds (1.8 x 12). I don't know how many pixels that is on your guide camera (22 / guide camera pixel scale)

Edited July 13, 2023 by wimvb

[wimvb]

1 hour ago, chubster1302 said:

D5300 unmodded on a William Optics ZS 73iii, no filters. Guiding with PHD2. 3min subs at 800iso. I "thought" the sky conditions were OK but as this graph shows, that really wasn't the case. 🙈

I think that's the main cause for your results. Unmodded cameras don't pick up much Ha, and the Elephant's trunk is mostly an emission nebula. I tried pushing the data harder, but ended up with that mottled background. I think you may get marginally better results with aggressive dithering and more data, but in all honesty if you want to capture emission nebulae, you need a camera that is more sensitive to Ha. Larger nebulae are most suited for your telescope's focal length of 440 mm (according to the fits header of your image).

[chubster1302]

4 hours ago, wimvb said:

I think that's the main cause for your results. Unmodded cameras don't pick up much Ha, and the Elephant's trunk is mostly an emission nebula. I tried pushing the data harder, but ended up with that mottled background. I think you may get marginally better results with aggressive dithering and more data, but in all honesty if you want to capture emission nebulae, you need a camera that is more sensitive to Ha. Larger nebulae are most suited for your telescope's focal length of 440 mm (according to the fits header of your image).

Yep...im sure i will get it modded at some time, and then obviously the next step (I hate this hobby 🙈) a dedicated astro cam but in the meantime, considering the first nights rubbish subs Im quite impressed by the results you have all achieved on this post. Gives me something to aim for

[ONIKKINEN]

Bit late to the party, but here is what i got out of the stack:

Processed in Siril and Photoshop with the following steps:

Siril first: Bin x2, background extraction with just a handful of samplers around the edges, Asinh stretch at 1000, histogram transformation by just moving the black and white point. Export as 16-bit TIFF for Photoshop.

Photoshop: StarXterminator first. Stretched the starless layer a little bit more and applied a boatload of noiseXterminator to fight the noisy nebulosity. Some saturation and manual colourbalance work and a little bit of curves and contrast (this "fiddling" part takes often most of the processing time, no guide for that im afraid). Left the stars rather bright, at least for a typical nebula image. I think the stars being at the forefront can act as distractions from the nebulosity behind which could use some more integration. Only mild work for the stars-only layer including: Smart sharpen, saturation. Added the starless back to the stars-only layer with the Linear dodge blend mode.

Its not bad data, just could use more of it.

[chubster1302]

1 hour ago, ONIKKINEN said:

Bit late to the party, but here is what i got out of the stack:

Processed in Siril and Photoshop with the following steps:

Siril first: Bin x2, background extraction with just a handful of samplers around the edges, Asinh stretch at 1000, histogram transformation by just moving the black and white point. Export as 16-bit TIFF for Photoshop.

Photoshop: StarXterminator first. Stretched the starless layer a little bit more and applied a boatload of noiseXterminator to fight the noisy nebulosity. Some saturation and manual colourbalance work and a little bit of curves and contrast (this "fiddling" part takes often most of the processing time, no guide for that im afraid). Left the stars rather bright, at least for a typical nebula image. I think the stars being at the forefront can act as distractions from the nebulosity behind which could use some more integration. Only mild work for the stars-only layer including: Smart sharpen, saturation. Added the starless back to the stars-only layer with the Linear dodge blend mode.

Its not bad data, just could use more of it.

That looks awesome, well, to my untrained eye anyways.

Interesting to read your comment regarding the background extraction samplers, this is where i think I might be going wrong early on, watching a few tutorials on it and I know you have to take the samples off the nebula but this whole image seems to be nebula so I am probably leaving way too many on. And when you say "Bin x2", where is that option in Siril ? 

 

[ONIKKINEN]

4 minutes ago, chubster1302 said:

That looks awesome, well, to my untrained eye anyways.

Interesting to read your comment regarding the background extraction samplers, this is where i think I might be going wrong early on, watching a few tutorials on it and I know you have to take the samples off the nebula but this whole image seems to be nebula so I am probably leaving way too many on. And when you say "Bin x2", where is that option in Siril ? 

 

Binning is in Geometry-> binning. It wasn't always there, i am using 1.2.0-rc1 and it is. I think it was in the previous version too but not sure if much further back than that so you may need to update if you are on 1.0 still.

Below is an example of the samplers i placed (or something similar)

They are a bit difficult to see but i just placed them on the areas that looked least worst in terms of nebulosity. Most of them are on dark nebulosity but in this case since those are the darkest spots in the image they can be used as a sample for a background. Images like this are very difficult to work with for this step because realistically there is no background in this field of view and it is corner to corner nebulosity (and basically all forms of gradient removal also tack onto and remove some nebulosity, just a matter of getting the best compromise). Not very good at this yet myself but i think in these cases its better to have fewer rather than more samplers.

[chubster1302]

4 minutes ago, ONIKKINEN said:

Binning is in Geometry-> binning. It wasn't always there, i am using 1.2.0-rc1 and it is. I think it was in the previous version too but not sure if much further back than that so you may need to update if you are on 1.0 still.

Below is an example of the samplers i placed (or something similar)

 

They are a bit difficult to see but i just placed them on the areas that looked least worst in terms of nebulosity. Most of them are on dark nebulosity but in this case since those are the darkest spots in the image they can be used as a sample for a background. Images like this are very difficult to work with for this step because realistically there is no background in this field of view and it is corner to corner nebulosity (and basically all forms of gradient removal also tack onto and remove some nebulosity, just a matter of getting the best compromise). Not very good at this yet myself but i think in these cases its better to have fewer rather than more samplers.

Ah, yes, found the binning option, many thanks ill give that a go. And yeah, I had like the whole background with samples on and just removed the ones around the obvious nebula in the middle. 🙈

[ollypenrice]

On 12/07/2023 at 19:28, chubster1302 said:

I've been watching loads of processing tutorials but I'm now at the stage where I'm just moving sliders and not really having a clue what I'm doing or trying to actually achieve. Does anyone want to have a play with this and show me what can, if anything, be brought out from this image.  At least then I will have a benchmark to try and aim for 

r_pp_light_stacked.fit 244.02 MB · 31 downloads

I can't donload a file that size but I think the most important thing is that you have pinpointed the problem in the the phrase above.  We live in a world in which people post u-tube 'tutorials' in which they actually say 'I just play with the sliders  till I like what I see,' as if this were a sane way to proceed.

In effect, you answer your own question in posing it: understand the sliders.  Actually, I think there's a degree of back and forth between two approaches to learning how to process.

1) Know the tools at your disposal and understand what they do. 

2) Learn to look ever more analytically at an image and say to yourself, 'What does it need?' Once you've decided on this, go to the tools you have. Just don't go to a tool till you know what you want to do with it.

What I would like to have tried with this image is go to Photoshop's Selective Colour and move the top slider in Reds to the left to lower the cyan in red. This usually boosts Ha signal.

Olly

[chubster1302]

1 hour ago, ollypenrice said:

I can't donload a file that size but I think the most important thing is that you have pinpointed the problem in the the phrase above.  We live in a world in which people post u-tube 'tutorials' in which they actually say 'I just play with the sliders  till I like what I see,' as if this were a sane way to proceed.

In effect, you answer your own question in posing it: understand the sliders.  Actually, I think there's a degree of back and forth between two approaches to learning how to process.

1) Know the tools at your disposal and understand what they do. 

2) Learn to look ever more analytically at an image and say to yourself, 'What does it need?' Once you've decided on this, go to the tools you have. Just don't go to a tool till you know what you want to do with it.

What I would like to have tried with this image is go to Photoshop's Selective Colour and move the top slider in Reds to the left to lower the cyan in red. This usually boosts Ha signal.

Olly

I have a rough understanding of what the "sliders" do, but the problem with following tutorials is that  9 times 10 whatever they are doing will not work, or give widly different results on your image. Interpretating how to apply the correct tools is the hard bit. I used to have a fine art pet photography business so have spent a fair bit of time in LR and PS, but with those "targets" I knew the dog was black, the grass was green etc....

The only saving grace is everything is digital, so if you "get it wrong" you can either undo, or start again 

[ollypenrice]

10 minutes ago, chubster1302 said:

I have a rough understanding of what the "sliders" do, but the problem with following tutorials is that  9 times 10 whatever they are doing will not work, or give widly different results on your image. Interpretating how to apply the correct tools is the hard bit. I used to have a fine art pet photography business so have spent a fair bit of time in LR and PS, but with those "targets" I knew the dog was black, the grass was green etc....

The only saving grace is everything is digital, so if you "get it wrong" you can either undo, or start again 

We do have certain hard information on astro colour. We can look up the spectral classes of a selection of stars on a planetarium and see what their colour is by referring to something like this. https://socratic.org/questions/what-does-the-color-of-a-star-indicate  Herschel's Garnet Star in your image is a very good indicator, for instance.

We also know that 'empty' background sky (where we can find it) should be a neutral dark grey with all colour channels at parity. We know that ionized hydrogen is a deep red. (It's in this that I think your image needs attention in processing because  the Ha reds seem reluctant to come out. I think Wim and Onikkinen have had the most success here.)

A useful trick in Ps is to use Curves to lift faint signal out of the background. Pin the curve at the level of the background, put a fixing point below that and lift the curve delicately above these.

Olly

[Elp]

The red was difficult to bring out even with selective colour changes. My issue was I used an all over background extraction with 9 points per line and did a green noise removal too which altered the overall pallet considerably.

Regarding tutorials, I think just watching or reading any daylight image editing source will help, the benefit is you can apply that knowledge to any image and not just astro processing. Even reading PSs help files gives useful insights into how it works.

[wimvb]

When we learn new things, we start by copying and repeating what others have done before us. This is the same for a child that is learning to write, math or mastering a musical instrument, as it is for us learning astrophotography and image processing. Over time we start to grasp the basics and we figure out the principles behind techniques. That's when we become creative. After enough practice, we get fluent, and the basics become second nature.

One problem with the self study of AP is that there are so many different software packages, and so many different teachers (youtube tutorials mainly) available. So where to start? Which teacher is best

Another problem is that when we start in AP, we usually start with poor data and no knowledge, while tutorials usually assume good quality data and basic knowledge. As a beginner, you're fighting an up hill battle. So, to learn AP processing, try to find good quality data for excercise. One repository is in this forum. Some time ago, FLO released high quality data from their IKI observatory. This is excellent material to learn image processing. What's more, people have published their results using various software packages, on this forum, so you can see what to expect.

https://stargazerslounge.com/forum/294-iki-observatory/

A third problem which many beginners make, is that they are too impatient. Rather than learning what individual steps in a workflow, or the individual processes in a software package do, and how to use those that are available, they jump from one software to another trying to find quick fixes. Hence the popularity of ever new software (EZ suite in PI, Topaz sharpening, graXpert). Choose one software, and learn that before you mix other software in your workflow.

While you learn the art of image processing (despite being highly technical in nature, it is an art), you also need to learn the other half of AP, data acquisition. For this, have a look at this resource. It is dated, but the principles still hold.

https://www.firstlightoptics.com/books/making-every-photon-count-steve-richards.html

The number one problem with beginner's data in my experience, is the lack of it. Beginners generally underestimate the amount of time it takes to gather good quality data. When I started in AP (and others have confirmed it was the same for them), I started out as a space tourist, shooting as many different targets as possible on a single night. (Like tourists visiting as many places as possible during their short vacation.) Now, I try to capture at least 10-15 hours of data per object, sometimes spending several weeks collecting that data.

People say that PixInsight has a steep learning curve. Anything has a learning curve. How steep you find that depends on how fast you are going, or want to go. Rather than racing for the peak, enjoy the climb.

[Rodd]

I can't download the data either.  For what its worth, I will say that the data looks very good.  And you have to keep this in mind...In my opinion, you are attempting one of the most unforgiving and difficult tasks in all of astrophotography...broadband nebulae....and with a DSLR.  Not an easy thing to pull off well. I don't know what your sky is like, but if its anything like mine, you need boat loads of data.  Also, for the lack of a better analogy, we all have internal sliders--expectations of how we think the image should look.  These internal sliders can get out of whack.  A perusal of my early images will reveal my unwilling internship at Disney, or Looney Tunes....I could not understand why I was creating cartoons, while the same accomplished imagers who have toyed with your data produced realistic "nebulous" looking images.  And THAT was with narrow band!, which I personally find much easier.   Even so, I have color mottle quite frequently with narrowband, which I attribute to LP--multiply problems (of any sort) by at least 10x with broadband.  So I think you are headed in a pretty good direction.

It is a good think to play with slider to see what they do.  Don't forget, it is often that how multiple sliders work in unison that is important (and how multiple sliders in different tools work in unison).  The best way to learn this IS through playing with them (with a eye on theory as well--documentation on what they represent).  I know it's not much help, but the following should be.  Great advice from Olly..."leave 10% of the image on the table" (It might have been 5%--can't recall).  This will be a natural check on the propensity to cross the threshold into cartoon land (which can happen all of a sudden.  Suddenly, the Rubicon is a mile behind you. Coupled with this gem is realizing that the slightest change in an image--the smallest tightening of edges, or the slimmest increase in dynamic range between background and nebula can radically change an image.  Combining very small incremental changes builds an image.  The goal is to render a nebula that appears to be floating in space, with a color palette that draws you in to the faintest details, as oppose to blinding you with chroma.  This can happen quickly, much like a giant slab of rock hanging by a thread from some precipice can be made to break free and plummet by the addition of a single grain of sand dropped from a bird's foot.

One last bit of advice.....WALK AWAY (Now!!! Run!!! And drop the camera in the nearest waste bin!).  Just kidding (sort of).  But do walk away...you will be amazed how different an image looks the next morning, or even an hour later.  Its akin to editing your own writing.  Nothing is better for an essay or story then to spend a day or two in the drawer.  I think its like IR that builds up in some cameras that need a pre-flash to clean out the signal (older technology) --Our brains need a break, a shift, to wash away the conceptions we build into them.  Not sure why, but every time I have come back to an image, I find I can improve it.

Finally--space is not black, stars are not like colorful diodes, brightness steals color (and curving down brightness enhances it), and nebula are rarely like a barrier reef community in vibrancy.  Somehow, the reef world does not look like a cartoon. Or maybe it does, but for some reason that is OK.  But an astro-photograph with the same definition and palette most likely would.

Rodd

[chubster1302]

Some amazing advice in here, thanks everyone. 

@wimvb 

I didnt know about the data repository, many thanks I will check that out. 

Quote

A third problem which many beginners make, is that they are too impatient. Rather than learning what individual steps in a workflow, or the individual processes in a software package do, and how to use those that are available, they jump from one software to another trying to find quick fixes. Hence the popularity of ever new software (EZ suite in PI, Topaz sharpening, graXpert). Choose one software, and learn that before you mix other software in your workflow.

While you learn the art of image processing (despite being highly technical in nature, it is an art), you also need to learn the other half of AP, data acquisition. For this, have a look at this resource. It is dated, but the principles still hold.

https://www.firstlightoptics.com/books/making-every-photon-count-steve-richards.html

I am currently using Siril + PS CC and I dont have any plans to change yet, I do have some experience in PS, Camera Raw and LR so at least im not starting from stracth. "Luckily" for me, my "brain" is as such, that it needs to know exactly how one thing works before moving onto the next, the frustrating bit is not really knowing what effect early editing decisions may have down the line, if at all, and there is always the, just because of the lack of knowledge...."what if I did that first" scenario. Yeah, I know this isnt life and death but it just bugs me 😀

I have the book you mention and that has helped me get to where I am currently, a good read 👍

Quote

The number one problem with beginner's data in my experience, is the lack of it. Beginners generally underestimate the amount of time it takes to gather good quality data. When I started in AP (and others have confirmed it was the same for them), I started out as a space tourist, shooting as many different targets as possible on a single night. (Like tourists visiting as many places as possible during their short vacation.) Now, I try to capture at least 10-15 hours of data per object, sometimes spending several weeks collecting that data.

Yep, whilst reading many posts and watching many vids that is one thing I knew from the start....lots of subs. Im currently only working on two targets