M3 Processed in PI...advice and critique please

[oymd]

Good evening everyone

Not sure what went wrong here?

This is the result of 60 x 120s subs @120 Gain, -10C, darks, flats & dark flats.

EVostar 80mm and ASI 294MC Pro.

Stacked in DSS and Processed in PI.

I used a tutorial kindly offered to me by @geeklee on processing in PI, but it was for processing M51, so I think it shold only be used for galaxies?

Since I am still taking my first steps in PI, I used the same steps.

Please give me some feedback if the result is good, otherwise, where did I go wrong?

Many thanks

[geeklee]

2 hours ago, oymd said:

I used a tutorial kindly offered to me by @geeklee on processing in PI, but it was for processing M51, so I think it shold only be used for galaxies?

Hi @oymd the tutorial was a high level, fundamental workflow to get something out of PI without feeling overwhelmed. The basic steps can be used but likely need tweaked as you go. The more you added to the workflows the more you'll need to adapt. 

The Inside PixInsight book and available help files (where populated) help immensely in understanding what you're doing and why. 

I've found with a globular cluster, some light touch HDRMultiscaleTransform can help with the core. This would need to be masked and might be too much at 100% - so could perform on an image clone and then use PixelMath to blend the two (I don't think that process has an "amount" slider). There's a section in Inside PixInsight on HDRMT. 

The image above looks fine to me,  the background is quite black but that could just be my phone screen. What do you think has gone wrong?

Edited April 26, 2020 by geeklee

[oymd]

10 minutes ago, geeklee said:

Hi @oymd the tutorial was a high level, fundamental workflow to get something out of PI without feeling overwhelmed. The basic steps can be used but likely need tweaked as you go. The more you added to the workflows the more you'll need to adapt. 

The Inside PixInsight book and available help files (where populated) help immensely in understanding what you're doing and why. 

I've found with a globular cluster, some light touch HDRMultiscaleTransform can help with the core. This would need to be masked and might be too much at 100% - so could perform on an image clone and then use PixelMath to blend the two (I don't think that process has an "amount" slider). There's a section in Inside PixInsight on HDRMT. 

The image above looks fine to me,  the background is quite black but that could just be my phone screen. What do you think has gone wrong?

Hi Lee

Thank you for your reply.

When I zoomed in on the cluster in PI, the stars looked really hazy, even though the outer stars which are not part of the cluster looked sharpish and in focus. When I completed the PI processing, the sky had a weird pattern of blotched black, hard to describe, so I opened the final image in GIMP and played a touch with the curves and black point to darken the sky to conceal the blotches in the sky. I guess I over did it, which is why the sky looks too black.

I will look up HDRMT in the book. There is just SO SO much to read in the book, and its very hard to find the time to give it a good read (and apply it to data), with everything going on!

Do you think if i attach the stacked unprocessed TIF file it would help? I thought after 2 hours of exposure on M3, I would get a cleaner result? Or maybe I am not sure of what to expect with my ED80 ad 294MC.

[geeklee]

8 hours ago, oymd said:

When I zoomed in on the cluster in PI, the stars looked really hazy, even though the outer stars which are not part of the cluster looked sharpish and in focus. When I completed the PI processing, the sky had a weird pattern of blotched black, hard to describe, so I opened the final image in GIMP and played a touch with the curves and black point to darken the sky to conceal the blotches in the sky. I guess I over did it, which is why the sky looks too black.

I see what you mean and it could have been the effects you describe above doing something - it does appear a little bit... washed out.  Saying that, there's some resolution of the stars around the core and the hazyish area behind where so many more stars exist is showing through a bit 👍  I find it hard to tell if focus is only slightly off so I'm not sure here.

Remember also, this is a small(ish) target - with the 80ED + FR and the 294 (~129 x 88 arc minutes). At ~18 arc minutes in size, you'll only be using a small area of the chip so resolution won't be right up there. 

I have a < 1 hour M3 in my astrobin link in signature.  Same scope but different camera - one with slightly smaller pixels. They are similar images.

8 hours ago, oymd said:

I will look up HDRMT in the book. There is just SO SO much to read in the book, and its very hard to find the time to give it a good read (and apply it to data), with everything going on!

It's only a small section in the book, so should be a nice easy one to review.  It won't alter the image much, but might help with the bright core.  

8 hours ago, oymd said:

Do you think if i attach the stacked unprocessed TIF file it would help?

It would be interesting to see what it looks like, but no need if you want to carry on working on it yourself.

[ollypenrice]

It doesn't matter which program you used to make the image. What is wrong with it is very easy to see. It is badly black clipped. Here is the histogram of a screen grab in Photoshop. (I've tested screen grabs of my own data versus the original data for this test and found no difference.)

The histogram peak is jammed up against the left hand edge of the graph when it should have a flat line to start with then a somewhat progressive rise. (Not terribly progressive, however, in an image with just stars and background as opposed to stars, background and nebulosity.)

Here are my 'stock' demo images. The first histogram is like yours, black clipped:

This is a healthy histogram from the same data. Note the flat line on the left before the histo peak. You need that:

See how much faint signal is cropped out by bringing the black point slider too far in to the right? Don't capture it then throw it away!

There is a big temptation to crop out light pollution and other gradients by clipping them out but resist this temptation because you'll clip out the real stuff at the same time. Build up your skills in DBE to remove gradients and leave the black point slider out of it.

Olly

 

 

Edited April 27, 2020 by ollypenrice
Typo

[oymd]

14 hours ago, ollypenrice said:

It doesn't matter which program you used to make the image. What is wrong with it is very easy to see. It is badly black clipped. Here is the histogram of a screen grab in Photoshop. (I've tested screen grabs of my own data versus the original data for this test and found no difference.)

The histogram peak is jammed up against the left hand edge of the graph when it should have a flat line to start with then a somewhat progressive rise. (Not terribly progressive, however, in an image with just stars and background as opposed to stars, background and nebulosity.)

Here are my 'stock' demo images. The first histogram is like yours, black clipped:

This is a healthy histogram from the same data. Note the flat line on the left before the histo peak. You need that:

See how much faint signal is cropped out by bringing the black point slider too far in to the right? Don't capture it then throw it away!

There is a big temptation to crop out light pollution and other gradients by clipping them out but resist this temptation because you'll clip out the real stuff at the same time. Build up your skills in DBE to remove gradients and leave the black point slider out of it.

Olly

 

 

Thanks Olly

A few questions please

1- Other than clipping the blacks which I agree I overdid, is the result acceptable for a ED80mm? Is the core OK, and did I get good detail of the stars in the center of M3? Should I have got a better result with the 294MC Pro and ED80?

2- Once you clip the blacks, can I go back to the TIF file and UNDO it in PixInsight or Gimp, or once the blacks have been clipped I need to REDO the image processing from scratch from the original stacked file?

I am just trying to establish whether my main problems are in the acquisition of the image or in the processing?

Thanks

 

[alacant]

15 minutes ago, oymd said:

can I go back to the TIF file and UNDO

Hi. In gimp use save as rather than export as.

The .xcf created by gimp allows you to view the whole of your history and go back to before you applied the black level.

Export to .tif only when you have done editing.

Nice image with or without though. Maybe mask the core stars before bringing up the rest? If you can throw more focal length at it, even better; probably a lot cheaper than switching cameras!

HTH

Edited April 28, 2020 by alacant

[ollypenrice]

54 minutes ago, oymd said:

Thanks Olly

A few questions please

1- Other than clipping the blacks which I agree I overdid, is the result acceptable for a ED80mm? Is the core OK, and did I get good detail of the stars in the center of M3? Should I have got a better result with the 294MC Pro and ED80?

2- Once you clip the blacks, can I go back to the TIF file and UNDO it in PixInsight or Gimp, or once the blacks have been clipped I need to REDO the image processing from scratch from the original stacked file?

I am just trying to establish whether my main problems are in the acquisition of the image or in the processing?

Thanks

 

Alacant has answered over image history.

The way to check for over-exposure is simply to look at the linear stack (ie the unstretched stack.) If it isn't saturated in the core and you can see some separation of stars then that can, with careful stretching, be preserved. As presented I'd say the core was slightly saturated. Looking at the linear stack will tell you whether it's happened at capture or in processing.

There isn't much in the way of star colour. The cluster is made up of mostly whitish stars with some red and blue ones sprinkled about. This isn't yet showing. There may be two reasons for this: Firstly over-stretching tends to burn all colour to white so a softer stretch would be helpful. Secondly star colour is generally captured in the outer fringes of stars where they are not so bright (as just mentioned) so any black clipping will crop out this information.

You could crop out a good part of the border to save file space and post a Dropbox link to the linear TIFF file if you'd like us to have a look at it from scratch.

Olly

[des anderson]

That`s a cracking image, well done. Des

[oymd]

25 minutes ago, ollypenrice said:

Alacant has answered over image history.

The way to check for over-exposure is simply to look at the linear stack (ie the unstretched stack.) If it isn't saturated in the core and you can see some separation of stars then that can, with careful stretching, be preserved. As presented I'd say the core was slightly saturated. Looking at the linear stack will tell you whether it's happened at capture or in processing.

There isn't much in the way of star colour. The cluster is made up of mostly whitish stars with some red and blue ones sprinkled about. This isn't yet showing. There may be two reasons for this: Firstly over-stretching tends to burn all colour to white so a softer stretch would be helpful. Secondly star colour is generally captured in the outer fringes of stars where they are not so bright (as just mentioned) so any black clipping will crop out this information.

You could crop out a good part of the border to save file space and post a Dropbox link to the linear TIFF file if you'd like us to have a look at it from scratch.

Olly

Thanks Olly, advice noted...

Yes please, it would be very helpful if you can have a look at the unprocessed file. I have attached the TIF file as it came out of Deep Sky Stacker, before doing any processing to it.

Many thanks

Autosave.tif

[oymd]

1 hour ago, alacant said:

Hi. In gimp use save as rather than export as.

The .xcf created by gimp allows you to view the whole of your history and go back to before you applied the black level.

Export to .tif only when you have done editing.

Nice image with or without though. Maybe mask the core stars before bringing up the rest? If you can throw more focal length at it, even better; probably a lot cheaper than switching cameras!

HTH

Thanks Alacant

I will re-open in GIMP, and try to undo clipping.

[alacant]

Just now, oymd said:

re-open in GIMP

Hi. Unfortunately AFAIK, the .tif doesn't store the gimp histrory.

 

[oymd]

1 minute ago, alacant said:

Hi. Unfortunately AFAIK, the .tif doesn't store the gimp histrory.

 

I've actually just opened the .xcf file, and it too does not show history...?

[alacant]

5 minutes ago, oymd said:

opened the .xcf file

No. My bad. Looks like you'll have to start over, but it shouldn't take more than a few moments. I'll have a go...

 

[oymd]

3 minutes ago, alacant said:

No. My bad. Looks like you'll have to start over, but it shouldn't take more than a few moments. I'll have a go...

 

Thanks Alacante!!

Can I ask you a quick question, while I am having your expertise on this thread!

There has been a problem that is confusing me with regards to DARKS taken after my light subs. My understanding is that darks need to match EXACTLY the lights in duration, exposure & temperature.

Since it is OK to combine Lights of different duration and stack them in DSS, how  will I apply the darks?

For example, say I have 30 light subs of 120 seconds each on M101, and on another night I took another 30 subs of 180 seconds each of M101. When I stack the lights in DSS, and then add DARKS, which ones should I choose from my Calibration Library? I have Darks @120s & 180s?

Do I add BOTH sets of darks to to DSS?

Many thanks

Ossi

 

[ollypenrice]

9 minutes ago, oymd said:

Thanks Alacante!!

Can I ask you a quick question, while I am having your expertise on this thread!

There has been a problem that is confusing me with regards to DARKS taken after my light subs. My understanding is that darks need to match EXACTLY the lights in duration, exposure & temperature.

Since it is OK to combine Lights of different duration and stack them in DSS, how  will I apply the darks?

For example, say I have 30 light subs of 120 seconds each on M101, and on another night I took another 30 subs of 180 seconds each of M101. When I stack the lights in DSS, and then add DARKS, which ones should I choose from my Calibration Library? I have Darks @120s & 180s?

Do I add BOTH sets of darks to to DSS?

Many thanks

Ossi

 

Why take different sub lengths to put into the same stack? I wouldn't do that. It's best to find out the optimal sub length and take those. Very exceptionally it is necessary to take short subs to cover the bright parts of a target with high dynamic range - M42 being the classic example. But don't throw the lot onto one stack, make two and then look into blending techniques. In this case I would make two separate stacks and then combine those.

Olly

[alacant]

1 hour ago, oymd said:

darks need to match EXACTLY the lights in duration, exposure & temperature.

Yes.

 

1 hour ago, oymd said:

DSS, how  will I apply the darks?

Pass, I gave up on dss some time ago;)

Anyway, here's what i came up with. I'm hopeless at colour but someone with decent eyes should be able to adjust it:

 

Edited April 28, 2020 by alacant
bg: another 5!

[oymd]

5 minutes ago, ollypenrice said:

Why take different sub lengths to put into the same stack? I wouldn't do that. It's best to find out the optimal sub length and take those. Very exceptionally it is necessary to take short subs to cover the bright parts of a target with high dynamic range - M42 being the classic example. But don't throw the lot onto one stack, make two and then look into blending techniques. In this case I would make two separate stacks and then combine those.

Olly

Hi Olly

I see on Astrobin, and various threads where imagers are combining different sub lengths, so I thought I can do that? Hypothetically, if I do have different subs, do I apply corresponding darks?

Or is that a dead question, as I should not do that from the outset?

[oymd]

6 minutes ago, alacant said:

Yes.

 

Pass, I gave up on dss some time ago;)

Anyway, here's what i came up with. I'm hopeless at colour but someone with decent eyes should be able to adjust it:

Alacant...that is STUNNING!!

thank you so much...!!

Was that in PixInsight? And you took just a few minutes to do that??

Please...I need your steps?? a mini tutorial...??

 

[ollypenrice]

1 minute ago, oymd said:

Hi Olly

I see on Astrobin, and various threads where imagers are combining different sub lengths, so I thought I can do that? Hypothetically, if I do have different subs, do I apply corresponding darks?

Or is that a dead question, as I should not do that from the outset?

I don't think it's very common to combine different sub lengths. M42, yes, every time. I use three different lengths on that. In thousands of hours of imaging I've probably done it on two other targets at most. Unless you know that a target's dynamic range exceeds that of your camera don't do it at all. (Some mono users like myself do different sub lengths for luminance and colour but this doesn't apply to OSC users.) Alacant has shown that you have the full dynamic range in your capture. Nice!

Could you post a TIFF with a large part of the background sky cropped off. 90 meg is defeating the rural French internet connection! Just keep the globular and some surrounding sky if you can.

Olly

[alacant]

13 minutes ago, oymd said:

Was that in PixInsight?

No. StarTools. PI would take several hours. (...runs for cover!)

Cheers

Edited April 28, 2020 by alacant

[ollypenrice]

13 minutes ago, alacant said:

No. StarTools. PI would take several hours. (...runs for cover!)

Cheers

Yes but PI would sort out the background sky! (Runs for cover...)

Olly

[alacant]

Just now, ollypenrice said:

PI would sort out the background sky

Given another 5 minutes, so would StarTools LOL.

[oymd]

3 minutes ago, ollypenrice said:

 (Some mono users like myself do different sub lengths for luminance and colour but this doesn't apply to OSC users.) Alacant has shown that you have the full dynamic range in your capture. Nice!

Could you post a TIFF with a large part of the background sky cropped off. 90 meg is defeating the rural French internet connection! Just keep the globular and some surrounding sky if you can.

Olly

I think that is possibly where I got confused, and I think I confused the L & RGB subs that imagers take with Mono cameras with my situation....where I use OSC.

Ok, got it, no more different subs durations to be mixed together.

Attached is the cropped TIF!

Many thanks Olly

 

Autosave crpped.tif

[oymd]

17 minutes ago, alacant said:

No. StarTools. PI would take several hours. (...runs for cover!)

Cheers

thanks...will have a look at Startools