Help needed for a 1st timer with narrowband data.

[Ryan_86]

Hi guys, 

Can anybody help a  prevent a Welsh guy from throwing his laptop into one of Brecon Beacon reservoirs 😬

I have gathered my 1st set of narrowband images of the Heart Nebula (10x600sec through each Ha, Sii & Oiii). I would really appreciate some help with a basic workflow of my rgb image prior to applying my Ha stack as a luminance layer. All stacks are calibrated with bias, flats and a 3x3 median noise reduction as I'm using a SXVR-H18 mono camera. 

I've stacked using DSS & am using Photoshop 2019 to post process. I have Adam Blocks Every Pixel Counts video tutorials & Steve Richards Dark Art book, noel carbons action toolset, HSLVG & GradientXterminator plug-ins at my disposal. However I am struggling to determine what steps to take and when. I've decided to assign my stacked filter sets to the SHO Hubble palette. My rgb and Ha luminance layer are aligned and cropped accordingly 👍🏼

Here are both my  merged rgb Sii-red, Ha-green & Oiii-blue and my Ha luminance image originals.

Any help would be very much appreciated. 

Thanks guys

Ryan

 

16bit tiff Ha aligned to rgb.tif

16bit tiff RGB aligned to Ha lum.tif

Edited December 15, 2019 by Ryan_86

[vlaiv]

Hi, I just had a look at data you posted and am slightly confused about it.

What stage of processing is this? I'm asking because I expected data straight out of DSS after stacking, but attached files feel like something has been done to them and I don't know quite what. It might be as simple thing as doing slight stretch in DSS itself or maybe some sharpening or something.

Background does not feel right, stars don't feel right - something is odd. Can you explain how you did your integration? (did you drizzle or do something "fancy" to the data?).

To be precise of what I mean, here is a piece of red channel in RGB composition:

Grain is too big in the background - like image was enlarged or maybe I don't know, sharpened or denoised or combination of the two done.

Btw, to answer your initial question, here is what I would do to do SHO with Ha lum composition (not sure if you can do that in PS, maybe you can with some clever layer manipulation):

1. make sure data is nicely stacked and linear, it is in 32bit float format (instead of 16bit fixed point/int)

2. do background elimination at this stage (wipe)

3. I would do individual stretch on each of R, G and B channels until I get nice looking nebulosity in each and then combine result in RGB image. You can use quite heavy denoising at this stage - this is just color information and it is not as important to preserve it's full sharpness.

4. I do the same with Ha to create luminance layer - stretch it, do sharpening / denoising - full processing thing like you would do mono image until you are happy with result.

5. Take rgb image and create "RGB map" image out of it - which is just R = R/max(R,G,B), G = G/max(R,G,B), B = B/max(R,G,B) . That is tricky part to do in PS, maybe one could do it with layers (make image from 3 layers set to "max value" then make other images with layers set to "divide value" or something like that).

6. Take completed luminance and multiply with Rmap, Gmap and Bmap to get R, G and B channels of completed color image.

[knobby]

Hi Ryan, went through this myself when I went narrowband a few months back, it's more like art than science ... I didn't use a lum just Ha Oii and Sii.

Aligned in DSS usung best frame as reference to align the 3 colours, then put into Photoshop and created new RGB image from the 3 channels.

Then much stretching and balancing to my eye trying to match backgrounds as best as possible. Then colour tweaks as in this post - 

 

Good luck and keep at it !

 

[Ryan_86]

10 minutes ago, vlaiv said:

Hi, I just had a look at data you posted and am slightly confused about it.

What stage of processing is this? I'm asking because I expected data straight out of DSS after stacking, but attached files feel like something has been done to them and I don't know quite what. It might be as simple thing as doing slight stretch in DSS itself or maybe some sharpening or something.

Background does not feel right, stars don't feel right - something is odd. Can you explain how you did your integration? (did you drizzle or do something "fancy" to the data?).

To be precise of what I mean, here is a piece of red channel in RGB composition:

Grain is too big in the background - like image was enlarged or maybe I don't know, sharpened or denoised or combination of the two done.

Btw, to answer your initial question, here is what I would do to do SHO with Ha lum composition (not sure if you can do that in PS, maybe you can with some clever layer manipulation):

1. make sure data is nicely stacked and linear, it is in 32bit float format (instead of 16bit fixed point/int)

2. do background elimination at this stage (wipe)

3. I would do individual stretch on each of R, G and B channels until I get nice looking nebulosity in each and then combine result in RGB image. You can use quite heavy denoising at this stage - this is just color information and it is not as important to preserve it's full sharpness.

4. I do the same with Ha to create luminance layer - stretch it, do sharpening / denoising - full processing thing like you would do mono image until you are happy with result.

5. Take rgb image and create "RGB map" image out of it - which is just R = R/max(R,G,B), G = G/max(R,G,B), B = B/max(R,G,B) . That is tricky part to do in PS, maybe one could do it with layers (make image from 3 layers set to "max value" then make other images with layers set to "divide value" or something like that).

6. Take completed luminance and multiply with Rmap, Gmap and Bmap to get R, G and B channels of completed color image.

Hi Vlaiv, 

Thanks for your reply. 

During integration I didn't drizzle, will try and give a break down of my stack. 

Used the best Ha frame as a reference for all 3 stacks. Used a 3x3 median noise reduction as the H18 is really noisy. Lights, bias and flats are stacked via kappa sigma clipping and applied to the three stacks. In the cosmetic section I did apply a 3 pixel removal of both hot and cold pixels (will this effect the weired grainines)? 

Maybe I have gone wrong during the stacking process. 

Thanks again Vlaiv👍🏼

 

[vlaiv]

8 minutes ago, Ryan_86 said:

Hi Vlaiv, 

Thanks for your reply. 

During integration I didn't drizzle, will try and give a break down of my stack. 

Used the best Ha frame as a reference for all 3 stacks. Used a 3x3 median noise reduction as the H18 is really noisy. Lights, bias and flats are stacked via kappa sigma clipping and applied to the three stacks. In the cosmetic section I did apply a 3 pixel removal of both hot and cold pixels (will this effect the weired grainines)? 

Maybe I have gone wrong during the stacking process. 

Thanks again Vlaiv👍🏼

 

Ok, let's keep things simple to see if we can avoid some issues.

No median filter for noise, no cosmetic correction at this stage. Use regular average for calibration frames (which camera are you using btw, since you don't use darks?) and use kappa sigma clipping for final lights integration. Set sigma to something like 3 and kappa to something like 2-3 (sigma being standard deviation and kappa number of iterations). You can use Ha as reference frame, but tell DSS not to include it in stack (or even better - use groups, DSS should align them all to same reference frame, right?)

Save result as 32bit fits and post those

[Ryan_86]

2 minutes ago, vlaiv said:

Ok, let's keep things simple to see if we can avoid some issues.

No median filter for noise, no cosmetic correction at this stage. Use regular average for calibration frames (which camera are you using btw, since you don't use darks?) and use kappa sigma clipping for final lights integration. Set sigma to something like 3 and kappa to something like 2-3 (sigma being standard deviation and kappa number of iterations). You can use Ha as reference frame, but tell DSS not to include it in stack (or even better - use groups, DSS should align them all to same reference frame, right?)

Save result as 32bit fits and post those

I'm using a Starlight Xpress H18 mono with the added external cooling fan. 

Yes apologies I failed to mention that I didn't include my Ha reference frame in stacks of sii and oiii👍🏼

OK Vlaiv, when I get back home I will restack following your instructions and post as 32bit fits👍🏼

Thanks again pal

Speak soon

Ryan

[Ryan_86]

39 minutes ago, knobby said:

Hi Ryan, went through this myself when I went narrowband a few months back, it's more like art than science ... I didn't use a lum just Ha Oii and Sii.

Aligned in DSS usung best frame as reference to align the 3 colours, then put into Photoshop and created new RGB image from the 3 channels.

Then much stretching and balancing to my eye trying to match backgrounds as best as possible. Then colour tweaks as in this post - 

 

Good luck and keep at it !

 

Hi Knobby, 

Thank you also for your reply. Im quickly discovering its a dark art and I need a magic bullet. Vlaiv has kindly offered his help which I can't thank him enough for. 

Thanks again👍🏼

Ryan

[Ryan_86]

6 hours ago, vlaiv said:

Ok, let's keep things simple to see if we can avoid some issues.

No median filter for noise, no cosmetic correction at this stage. Use regular average for calibration frames (which camera are you using btw, since you don't use darks?) and use kappa sigma clipping for final lights integration. Set sigma to something like 3 and kappa to something like 2-3 (sigma being standard deviation and kappa number of iterations). You can use Ha as reference frame, but tell DSS not to include it in stack (or even better - use groups, DSS should align them all to same reference frame, right?)

Save result as 32bit fits and post those

Hi Vlaiv,

Here are my three individual raw stacks straight out of DSS. 

6 hours ago, vlaiv said:

Ok, let's keep things simple to see if we can avoid some issues.

No median filter for noise, no cosmetic correction at this stage. Use regular average for calibration frames (which camera are you using btw, since you don't use darks?) and use kappa sigma clipping for final lights integration. Set sigma to something like 3 and kappa to something like 2-3 (sigma being standard deviation and kappa number of iterations). You can use Ha as reference frame, but tell DSS not to include it in stack (or even better - use groups, DSS should align them all to same reference frame, right?)

Save result as 32bit fits and post those

Hi Vlaiv,

Here are my three individual stacks in 32bit Fit straight out of DSS. The amp glow in the Oiii opposite due to a auto meridian flip.

These stacks are integrated exactly as you mentioned above, however I unticked background calibration and debloom tabs as I wasn't sure whether to include or not. 

Thanks for taking a look.

Ryan

FIT 32bit Ha Stack.fit FIT 32 bit Sii Stack.fit FIT 32 bit Oiii Stack.fit

Edited December 15, 2019 by Ryan_86

[vlaiv]

I'm not sure I can make anything out of this data to be honest. It is just too noisy and noise is not "nice" kind of noise.

Why didn't you include dark calibration? That could probably help a lot with that kind of hot pixel noise.

Here is what I could manage to get out of Ha sub (with all kinds of magic):

There is some sort of bright ridge over that image that I can't get rid of - that is not there in other subs. Also, other subs don't contain this much signal at all (Ha is usually the strongest of the three). So I believe you won't be able to get a decent image out of this data unless you try to remove that noise.

I'm guessing that you did not want to take darks because you don't have set point cooling, right?

Why don't you just give it a go with dark scaling? Maybe you will be surprised with results. Do take a set of darks - same exposure and use them as well, but tick "dark optimization" option in DSS (and leave everything else the same, in fact - don't even do sigma clip stacking - use regular average for everything).

Btw, here is OIII sub processed the same as Ha:

Not much there really and denoise makes things too soft.

[carastro]

This is what I managed to get with the data.  However as Vlaiv said it was incredibly noisy, and I only managed to salvage it by using Vlaiv's Ha Process by placing that over the coloured image and blurring the coloured image to get rid of the worst of the noise, and then using the Ha as a luminance layer.

I have no idea how Vlaiv managed to get this result from the Ha, as I couldn't.

I think the first thing is to try to work out why the data is so noisy.

I struggled for the first few months with combining data and getting it to align, but you seem to have sorted that aspect.

Carole 

Edited December 15, 2019 by carastro

[Ryan_86]

47 minutes ago, vlaiv said:

I'm not sure I can make anything out of this data to be honest. It is just too noisy and noise is not "nice" kind of noise.

Why didn't you include dark calibration? That could probably help a lot with that kind of hot pixel noise.

Here is what I could manage to get out of Ha sub (with all kinds of magic):

There is some sort of bright ridge over that image that I can't get rid of - that is not there in other subs. Also, other subs don't contain this much signal at all (Ha is usually the strongest of the three). So I believe you won't be able to get a decent image out of this data unless you try to remove that noise.

I'm guessing that you did not want to take darks because you don't have set point cooling, right?

Why don't you just give it a go with dark scaling? Maybe you will be surprised with results. Do take a set of darks - same exposure and use them as well, but tick "dark optimization" option in DSS (and leave everything else the same, in fact - don't even do sigma clip stacking - use regular average for everything).

Btw, here is OIII sub processed the same as Ha:

Not much there really and denoise makes things too soft.

Thanks for your efforts Vlaiv.

Yes that's correct, due to no set point cooling and SX's H18 manual recommending a 3x3 median noise reduction over dark frames,. I didnt take any. Next time I'm out I will take the necessary darks in between each filter change. 

Can I upload a new stack of my Ha data including the median noise reduction to see if it helps. Also I will apply a cosmetic removal of hot pixels in DSS. What size pixels should I use to remove them? 

Thanks

Ryan

[Ryan_86]

9 minutes ago, carastro said:

This is what I managed to get with the data.  However as Vlaiv said it was incredibly noisy, and I only managed to salvage it by using Vlaiv's Ha Process by placing that over the coloured image and blurring the coloured image to get rid of the noise, and then using the Ha as a luminance layer.

I have no idea how Vlaiv managed to get this result from the Ha, as I couldn't.

I think the first thing is to try to work out why the data is so noisy.

I struggled for the first few months with combining data and getting it to align, but you seem to have sorted that aspect.

Carole 

Hi Carol, 

Thank you ever so much for your input.

The H18 is known to be a noisy camera, however I've been advised to apply a 3x3 median noise reduction during stacking (which this stack does not have). Can I restack including noise reduction to see what you think? 

Thanks again

Ryan

[carastro]

Yes give it another try.  Can I have the Ha separately plus the SHO combined.

Thanks

Carole 

[vlaiv]

Just now, Ryan_86 said:

Thanks for your efforts Vlaiv.

Yes that's correct, due to no set point cooling and SX's H18 manual recommending a 3x3 median noise reduction over dark frames,. I didnt take any. Next time I'm out I will take the necessary darks in between each filter change. 

Can I upload a new stack of my Ha data including the median noise reduction to see if it helps. Also I will apply a cosmetic removal of hot pixels in DSS. What size pixels should I use to remove them? 

Thanks

Ryan

You don't need a clear night to get your darks, nor you need to do it between filter changes. You need single set of darks, even if you are using different exposure lengths for different filters - as you will attempt to scale the darks. Just take a set of darks somewhere where it is close to temperature you were working at (but needs not be precise that temperature - so you can leave it in shed or basement or somewhere during cloudy night to record).

Don't apply 3x3 median filter - it will just kill any chance of stacking properly. You can try single pixel cosmetic correction though - that might work. In fact one of steps when trying to get that Ha signal was to apply my own cosmetic correction thing on whole stack.

You can even post all subs (maybe drop box or google drive) to see what sort of stacking would work the best on your data.

[Ryan_86]

7 minutes ago, vlaiv said:

You don't need a clear night to get your darks, nor you need to do it between filter changes. You need single set of darks, even if you are using different exposure lengths for different filters - as you will attempt to scale the darks. Just take a set of darks somewhere where it is close to temperature you were working at (but needs not be precise that temperature - so you can leave it in shed or basement or somewhere during cloudy night to record).

Don't apply 3x3 median filter - it will just kill any chance of stacking properly. You can try single pixel cosmetic correction though - that might work. In fact one of steps when trying to get that Ha signal was to apply my own cosmetic correction thing on whole stack.

You can even post all subs (maybe drop box or google drive) to see what sort of stacking would work the best on your data.

Ok Vlaiv, I will take a set of darks tomorrow night. All subs were 10mins long, how many do you recommend I take?

What's your drop box address, I will post all my subs 1st thing tomorrow.

[Ryan_86]

21 minutes ago, carastro said:

Yes give it another try.  Can I have the Ha separately plus the SHO combined.

Thanks

Carole 

I am unable to combine FIT files in PS, so I hope you don't mind that I have attached all three new stacks including 3x3 median noise reduction and a 3 pixel cosmetic hot pixel removal. Apologies I forgot to change to 1 pixel hot pixel removal.

Thanks again Carole

Ryan

32bit FIT Ha Stack with median noise reduction.fit 32bit FIT Oiii Stack with Median noise reduction.fit 32bit FIT Sii Stack with Median noise reduction.fit

[vlaiv]

5 minutes ago, Ryan_86 said:

I am unable to combine FIT files in PS, so I hope you don't mind that I have attached all three new stacks including 3x3 median noise reduction and a 3 pixel cosmetic hot pixel removal. Apologies I forgot to change to 1 pixel hot pixel removal.

Thanks again Carole

Ryan

On quick inspection, these look much better I must say, I'll have another go at processing now.

[Ryan_86]

14 minutes ago, vlaiv said:

On quick inspection, these look much better I must say, I'll have another go at processing now.

Amazing thanks  

P.S I forgot to change the hot pixel removal from 3 to 1.

Ryan

[carastro]

Unfortunately I can't use the attached files.  Photoshop won't open fits files.  I've tried opening them in Astroart and Maxim and the files and data show up  in both of those, but if I try to re-save them as a Tiff file and then open in PS, all I get is a black file with no detail.

Carole 

 

[Ryan_86]

Ok Carole I shall attach tiffs now  

Ryan

[Ryan_86]

These TIF stacks include the median noise reduction and hot pixel removal, however they were stacked prior to changing my stacking settings so they are all stacked via kappa sigma clipping.

If this isn't of any use I can restack using average settings and upload 1st thing tomorrow  

Thanks again

Ryan

Autosave028 Stacked Ha with 3pix noise reduction.tif Autosave016 Stacked SII 3pix noise reduction.tif Autosave016 Stacked OIII 3pix noise reduction.tif

Edited December 15, 2019 by Ryan_86

[carastro]

The data was still extremely noisy I don't think I could do anything better than before.

However I think the noise reduction and hot pixel removal has done nasty things to the stars.  I find the cosmetic tab is a culprit in doing this and I never use it if I ever use DSS.  See attached below. 

You might find my video tutorial useful for combing narrowband colour.

 

Edited December 15, 2019 by carastro

[vlaiv]

I was really struggling with this, and mind you, I'm no expert at NB composition, but here is something resembling a result

Here is Ha only:

I think it is quite a bit better than previous.

[vlaiv]

Ok, maybe this is tad better color composition (I did it now the way I proposed in the first reply - via RGB transfer):

[Ryan_86]

16bitTIF Ha HDR Tonning.tifThat looks much much better Im not sure I could replicate that haha.

I've been experimenting with stacks. So far keeping all my DSS settings to average, applying a median noise reduction filter and applying just a 1 pixel hot pixel removal so far seems the best. Anyhow, when converting from a 32bit tif to 16bit in PS and converting to HDR toning, it produces this which looks far better than anything I have done so far with my Ha data, a little bright and soft in areas but I will try adjusting such areas.

What do you guys think?

Thanks

Ryan 

16bitTIF Ha HDR Tonning.tif

Edited December 16, 2019 by Ryan_86
To display image