Am i wasting my time...

[paul mc c]

Finally git a nice clear sky last night,allowing me to image m31 from around 11 until around 3,getting around 4 hours of subs at 300 secs each. Surely thats a decent amount to time to to reasonably happy,for me,to have something to show for it.

I am only using Gimp to process and know i really need something better,but cant really afford to get the more expensive software.

Am i wasting my time doing AP with what i have.I know i have something there but just cant get the data out with what i have.

My finished pis is not much different than my single sub.

I have attached my TIF in case any one wants to see what they can do.

Paul

and.TIF

[vlaiv]

Quick stretch shows that there is a lot going on there:

(again using Gimp 2.10) - I just did basic stretch to show what has been captured, not actual processing.

How did you stack? May I suggest to use 32bit floating point as file format instead of 16bit? Added precision is necessary if you are using DSLR (14 bit single sub, about 48 of subs, right?).

What is it that you are not happy about with this image?

[paul mc c]

Hi Vlaiv,i stacked in DSS,and yes DSLR and around 48 subs,im happy with your image,but not mine,i could'nt get near what you got,is it basic stretch in colours you used.

[vlaiv]

Ah, ok. I'll record my steps to get to above image - very basic levels stretching. Btw, you should really use 32bit precision when saving stack result in DSS. Don't do any stretching in DSS either - just leave result linear (don't know if you did do any stretch in DSS but "bulk" part of M31 did seem quite bright like there was some histogram manipulation applied).

Anyways, here we go:

First thing I did was to convert image to 32bit format (this won't restore missing bits, but it won't introduce any more rounding, at least not significant when working with image).

(convert in linear light, although I'm not sure if it makes any difference)

Next do one round of levels:

Move right "white" slider left until you see that galaxy core is starting to saturate, than back off a bit - you want to move it left while not making any part of galaxy core saturate. Move left "black" slider at the foot of histogram, again leaving some room. Move middle "gray" slider to the left. Don't worry if everything in the image turns white - we will correct this in next step. There is no "proper" place to put it - you need to do it by feel - the more left you drag it - more chance background noise will be seen in next phase - if you don't drag it enough your image will not be stretched enough.

Do another round of levels - this time we will make adjustments to bring everything "in order":

This time don't move white slider - we moved it as far as it needs to go - any more and you will cause clipping in high signal - and we don't want that.

For black  - do the same - bring it at the foot of histogram. Use gray point to make adjustments to how exposed you want image to be.

There you go. Two rounds of levels and you have data visible.

[paul mc c]

Thank you,thats very kind of you to show me that,i am a bit of a dummy when it comes to this,i will give it a go.

[paul mc c]

A latest attempt,guessing really at what i am doing...

[vlaiv]

You blown core a bit.

Well, not a bit, a bit more

You also need to remove gradient and do color balancing ...

Could you post 32bit version of the stack? I think pretty decent image can be obtained with some fiddling around

 

 

Edited September 1, 2019 by vlaiv
One indefinite article too many ... :D

[paul mc c]

2 minutes ago, vlaiv said:

You blown core a bit.

Well, not a bit, a bit more

You also need to remove gradient and do color balancing ...

Could you post 32bit version of the stack? I think pretty decent image can be obtained with some fiddling around

 

 

How do i do that...

[vlaiv]

In DSS, after stacking, just save as 32bit - either Tiff or Fits

For item marked as 2 select 32bit float point

For item marked as 3, depending on type, select some level of compression to get smaller file (TIFF supports compression, but Fits does not)

In options choose not to apply adjustments to the image (either embed it or just ignore it, but don't apply it)

 

[paul mc c]

7 minutes ago, vlaiv said:

You blown core a bit.

Well, not a bit, a bit more

You also need to remove gradient and do color balancing ...

Could you post 32bit version of the stack? I think pretty decent image can be obtained with some fiddling around

 

 

How do i do that...

androm.TIF

[almcl]

I agree with Vlaiv: there's a pretty decent image there with some nice detail in the dust lanes. 

I had a go at your second file and although DSS seems to have done something to it, this is the result I got in a very quick go with StarTools (sorry, don't have Gimp, so couldn't do it with that).

To answer the question posed in the thread title, no, I don't think you're wasting your time at all.  Post processing is a separate skill that takes a little time to acquire, but if you save your data, you can always come back to it in the future and have another go - I've been reworking data from 4 years ago and sometimes produce quite noticeable improvements.

Keep going would be my suggestion and don't be afraid to post results and ask questions - we can all benefit from that.

Anyways, see what you think?

[vlaiv]

28 minutes ago, paul mc c said:

How do i do that...

androm.TIF 111.54 MB · 1 download

Ok, yes, I see what you mean and I probably jumped the gun on gradient removal.

I'm not sure if I can do it in Gimp - will need to check if any plugin is capable of doing it properly. I do it in ImageJ with plugin that I wrote. I can remove the background and then post you results so you can give it a go at processing with clean background.

There are couple of paid software out there that can do it, but I did not try any of them. I've heard of gradient exterminator (plugin for PS). Pixinsight has dynamic background extraction (often abbreviated as DBE). Iris which is free software can also do it:

http://www.astrosurf.com/buil/iris/tutorial4/doc14_us.htm

But it is a bit involved.

Color balancing is rather easy once you have your background removed. Simplest form is to find white star in the image (using Stellarium or similar - look for F2 class star) and measure R, G and B levels. Then multiply needed channels to get white color - or same values of R, G and B (for example if you measure R to be 0.4, G=0.6 and B=0.2 then you need to multiply R with 2.5, G with 1.66666 and B with 5 - which are inverse of measured values).

More advanced version of color calibration would be to measure bunch of stars and determine transform matrix that give you proper RGB triplets depending on stellar class (or rather temperature).

I'll have a go at a bit more serious processing and I'll post results as well as cleaned image (and maybe color calibrated) for you to also have a go at processing it further.

 

[vlaiv]

Ok, this is only in Gimp with 32bit data - fiddling with levels and curves:

But there is gradient in the image, so I'm going to see it removed now and look if results improve things (and of course they should).

[vlaiv]

It's looking better and better ...

[vlaiv]

Ok, so here is 32bit tiff with some things done to it:

1. I binned x2 to recover some of the SNR

2. I wiped the background of any gradients

3. I tried color balance on one star that Stellarium classifies as color index 0.32 (which should be white star). Results are not the best - too much red color, but it can be fixed with curves - like I did with my processing.

4. Vertical flip to put andromeda in more familiar orientation (maybe should have done horizontal flip as well ).

I did not do any noise reduction - just levels / curves (levels as shown above and a bit of curve adjustment to kill off excessive red) stretch and I got this result:

For some reason, I can't upload linear tiff - it fails. Let me try in another post ...

 

 

[vlaiv]

Here we go again (it's 28MB so it should not be an issue, but in previous post it failed twice for some reason):

Ok, it failed again ... I'll upload it to my server and post a link for download here ...

http://serve.trimacka.net/astro/Forum/2019-09-01/post_01/andromeda.tif

In the meantime, here is a bit more tweaked version (need to stop now, as I'll just get into that "which version is better" mental loop ... )

- horizontal flip

- slight tweak on green curve to make galaxy core yellowish rather than reddish

- slight tweak on blue curve to make blue low range a bit more "sparkly"

- selective wavelet denoising ...

[paul mc c]

16 minutes ago, vlaiv said:

Here we go again (it's 28MB so it should not be an issue, but in previous post it failed twice for some reason):

Ok, it failed again ... I'll upload it to my server and post a link for download here ...

http://serve.trimacka.net/astro/Forum/2019-09-01/post_01/andromeda.tif

In the meantime, here is a bit more tweaked version (need to stop now, as I'll just get into that "which version is better" mental loop ... )

- horizontal flip

- slight tweak on green curve to make galaxy core yellowish rather than reddish

- slight tweak on blue curve to make blue low range a bit more "sparkly"

- selective wavelet denoising ...

Brilliant....thats amazing Vlaiv,when you know what you are doing makes a world of difference.....what programe did you do all this in.

[vlaiv]

2 minutes ago, paul mc c said:

Brilliant....thats amazing Vlaiv,when you know what you are doing makes a world of difference.....what programe did you do all this in.

ImageJ / custom plugin written by me was used to remove background gradient. It looks for faintest pixels in the image and tries to do linear fit on those pixels only. I do multiple rounds of this fitting / removal (until residual gradient is small enough compared to signal in the image)

Next was color calibration - I again used ImageJ to measure star value and do pixel math to multiply each channel with proper value for white balance (I used Stellarium to find suitable star, however it looks like Stellarium has wrong/outdated information in stellar color indices due to catalogue used, and this sometimes fails - in this case there were excessive red cast after this calibration).

Result of these two operations is linear Tiff that I provided link to.

After that, it was all Gimp 2.10 - levels/curves to do the stretch and do slight color corrections (toned down red, boosted green in bright areas to get nice yellowish color in galaxy core, and boosted low values of blue to give outer star lanes a bit of a "pop").

Horizontal / vertical flip is self explanatory.

In the end I did a bit of selective noise reduction. This is done by creating another layer of the image. On this layer you perform noise reduction - I used G'mic-qt under filters, then selected "repair" and somewhere down the list there is wavelet denoise.

I ran default settings. This tends to blur details somewhat, so we need to selectively mix this layer into base layer. We do this by adding a layer mask to denoised layer - value only and inverted. This will blend denoised version in dark areas only (mostly background) - you can adjust level of this effect by selecting opacity of this denoised layer lower than 100% (I think I set it somewhere around 70-80% - was not paying attention to actual number and was looking more at the image to find most pleasing mix).

 

 

[vlaiv]

Ah forgot - I did bin x2 in ImageJ as well - prior to background removal and color calibration.

[jager945]

Hi Paul,

You are producing some fairly good data - kudos!

Now I could be wrong, but from the color signature (a lack of yellow) I get the impression you are using a light pollution filter. Is that correct?

The only obvious thing I can recommend to improve your datasets, woudl be to try dithering between frames, as I can see some walking noise ('streaks').

When processing spiral galaxies, things to look out for are a yellow core (older stars, less star formation), blue outer rim (younger blue stars, more star formation) and purplish/pink HII areas dotted around the arms (you captured these too). You'd also be looking for a good even foreground distribution of star colors, ranging from red->orange->yellow (missing here)->white->blue.

You are definitely well on your way!

 

FWIW I processed your image in StarTools 1.5 as follows;

--- Auto Develop

Default parameters, to see what we are working with. We can see some stacking artifacts, gradients, walking noise, dust donut at the top.
--- Crop

Cropping away stacking artifacts and dust.
Parameter [X1] set to [20 pixels]
Parameter [Y1] set to [55 pixels]
Parameter [X2] set to [2129 pixels (-16)]
Parameter [Y2] set to [1414 pixels (-14)]
Image size is 2109 x 1359
--- Wipe

Getting rid of gradients. Default parameters.
Parameter [Dark Anomaly Filter] set to [4 pixels]
--- Auto Develop

Final stretch. Clicked and dragged a Region of Interest (RoI) over part of the galaxy.
Parameter [Ignore Fine Detail <] set to [2.5 pixels]
Parameter [RoI X1] set to [95 pixels]
Parameter [RoI Y1] set to [607 pixels]
Parameter [RoI X2] set to [1963 pixels (-146)]
Parameter [RoI Y2] set to [964 pixels (-395)]

--- Deconvolution

Auto mask. Default settings. Usually worth a try, as the module tends to "know" when and where improvements can be made. It didn't do too much in this case.
--- HDR

Default parameters. Shows a little more detail in the core.
--- Color

See notes above about what to look for.
Parameter [Bright Saturation] set to [3.00] (less saturation in the highlights to hide some color channel alignment issues)
Parameter [Green Bias Reduce] set to [1.09]
Parameter [Red Bias Reduce] set to [2.09]
Parameter [Cap Green] set to [100 %] (only when you're done color balancing)

--- Wavelet De-Noise

Default parameters.
Parameter [Grain Dispersion] set to [7.5 pixels]

Hope this helps at all!

[paul mc c]

Thank you,that looks great......i actually was looking into Startools,i downloaded the free version just last night with the intention of purchasing it,would you recommend it,yes i was using an Astronomic CLS filter on a modded 450d and thank you for letting me know how you processed it,i will give it a go my self later.

Paul

[bottletopburly]

1 hour ago, paul mc c said:

would you recommend it

 

Lols 😂 @jager945 ivo would he developed it 👍

Edited September 5, 2019 by bottletopburly

[paul mc c]

3 hours ago, bottletopburly said:

Lols 😂 @jager945 ivo would he developed it 👍

My apologies......i did'nt realise that,feel like a right muppet now.

[jager945]

5 hours ago, paul mc c said:

My apologies......i did'nt realise that,feel like a right muppet now.

I'm the muppet here! My apologies - I should've made clear I'm the developer. I usually have this in my signature on other forums. I just updated my profile pic to avoid any confusion in the future.

Clear skies!

[happy-kat]

If you want to stick using GIMP then you can follow the great videos on budgetastro.net by Doug as he covers DSS and processing and much can be done in GIMP even though he uses photoshop.

I use StarTools and it's a great astro processor. The best tif output to png converted file (great for uploading if sharing) is irfanview which is free.

Book wise then 'Dark Art Or Magic Bullet' is good.