Advice on processing

[StuartT]

Last night I collected almost 3 hours on Cygnus Wall using my new Optolong L-Extreme filter. I shot bias, dark, flats and darkflats.

I stacked in Astropixel Processor using the default settings
then I used the Astroflats Pro plugin in Photoshop
then the Gradient Exterminator plugin in Photoshop
finally a bit of contrast enhancement in Photoshop

Here is what I ended up with. I think it's quite nice but I am not sure if there is more in there (so I have provided the FITS also, in case anyone wants to try it in their workflow)

 

 

Edited October 4, 2021 by StuartT

[PeterCPC]

I couldn't get the FITs to work but I tweeked the jpg a bit in Affinity - there is a lot of noise. Did you dither your shots?

[ollypenrice]

The FITS file was corrupt for me, too.

What do you do with the bias from this camera? I'd have thought they would be redundant.

There's a lot of vertical banding in the image, which comes as a surprise, and may arise from calibration. It can be reduced using Noel's Actions (Now Pro Digital Astronomy Tools) 'Reduce Vertical Banding,' but killing it at source would be much better.

Olly

[Budgie1]

9 minutes ago, ollypenrice said:

The FITS file was corrupt for me, too.

It's the Master-Flat that Stuart has uploaded.

I agree with Olly, try restacking without the Bias frames and see if it makes a difference to the final image. With my ASI294MC Pro the images were less noisy when stacked with only Darks, Flats & Dark-Flats.

[StuartT]

35 minutes ago, ollypenrice said:

The FITS file was corrupt for me, too.

What do you do with the bias from this camera? I'd have thought they would be redundant.

There's a lot of vertical banding in the image, which comes as a surprise, and may arise from calibration. It can be reduced using Noel's Actions (Now Pro Digital Astronomy Tools) 'Reduce Vertical Banding,' but killing it at source would be much better.

Olly

I must admit to being pretty confused about which calibration frames I am meant to be doing. Someone also said (somewhere here in SGL) that with a cooled camera like the 2600MC you don't really need darks either. Don't know if you agree with that?

18 minutes ago, Budgie1 said:

It's the Master-Flat that Stuart has uploaded.

I agree with Olly, try restacking without the Bias frames and see if it makes a difference to the final image. With my ASI294MC Pro the images were less noisy when stacked with only Darks, Flats & Dark-Flats.

Oops! So it is. 🤦‍♂️

Ok, so I am going to try stacking without any calib frames (as I sometimes find this makes things even better), then also with just darks, flats and darkflats. I'll then post what I get and also the FITS (but the correct ones!)

But I don't really think I know what I am doing in Astropixel Processor to be honest. I am just using all the default settings and pressing the buttons 🤣

[ollypenrice]

58 minutes ago, StuartT said:

I must admit to being pretty confused about which calibration frames I am meant to be doing. Someone also said (somewhere here in SGL) that with a cooled camera like the 2600MC you don't really need darks either. Don't know if you agree with that?

Oops! So it is. 🤦‍♂️

Ok, so I am going to try stacking without any calib frames (as I sometimes find this makes things even better), then also with just darks, flats and darkflats. I'll then post what I get and also the FITS (but the correct ones!)

But I don't really think I know what I am doing in Astropixel Processor to be honest. I am just using all the default settings and pressing the buttons 🤣

So far we have made no darks with the 2600 because it's a bit involved with the RASA 8. (We will need to take off the camera to do them, which means we'll also need to know our flats exposure times in order to do flat darks at the same time.) Since not using darks seems to give clean results we haven't made it a priority to try them, and the F2 of the RASA makes dust bunnies so far out of focus as not to matter, apparently.) I'm not saying you don't need darks because we haven't yet tried them, but some other users of this camera, whose images are good, are not using them. I'd certainly try a stack without.

Olly

[StuartT]

11 minutes ago, ollypenrice said:

So far we have made no darks with the 2600 because it's a bit involved with the RASA 8. (We will need to take off the camera to do them, which means we'll also need to know our flats exposure times in order to do flat darks at the same time.) Since not using darks seems to give clean results we haven't made it a priority to try them, and the F2 of the RASA makes dust bunnies so far out of focus as not to matter, apparently.) I'm not saying you don't need darks because we haven't yet tried them, but some other users of this camera, whose images are good, are not using them. I'd certainly try a stack without.

Olly

this is helpful, thanks.

And you said (above) that bias frames are also not needed with this camera? Why is that?

[Budgie1]

In my case with ASI294MC, the use of the Bias frames in the stack caused a residual of amp-glow to remain in the finished image. Somehow the Bias was causing the Darks not to work correctly. Taking the Bias frames out eliminated all trace of amp-glow. 

I have read that it's because the Bias frames should be taken at high exposure speed (1/4000 to 1/8000) but some cooled cameras are not capable of those exposures and the result is a poor quality Bias frame. Whether this is the case or not, I don't know, maybe someone on here does.  

All I would say is to try stacking both with & without the Bias frames and see if there's a difference, then go with what produces the best image for your camera & setup.  

[wimvb]

I believe that @gorann doesn't use any calibration frames with this camera. And the general consensus seems to be that bias frames on cooled cmos cameras add more problems than they solve, because the bias signal varies between lights/darks and bias frames.

Regarding the vertical lines, what exposure time did you use? Maybe just increasing the exposure will remove the banding.

[StuartT]

Ok, so this is interesting.

It's really looking like in my setup, stacking with no calibration frames is producing the best result!

The first image is with darks, flats and flat darks.
The second image is without any calibration frames. So just the lights stacked.

Both are just as they come out of Astro Pixel Processor and before any post-processing. 

I'm also attaching the FITS for each one, in case any of you want to see what you can get out of them. 

Cygnus_Wall_with_L-extreme-RGB-session_1.fits Cygnus_Wall_with_L-Extreme-RGB-session_1_no calib frames.fits

[tooth_dr]

@StuartT how did the new flats look taken in the dark?

[StuartT]

2 minutes ago, tooth_dr said:

@StuartT how did the new flats look taken in the dark?

not much different, if truth be told!

[StuartT]

so here is the final post-processed result.

2h 50m integration time and no calibration frames (lights only).

 

Edited October 4, 2021 by StuartT

[tooth_dr]

9 minutes ago, StuartT said:

not much different, if truth be told!

Mmm that’s an odd looking flat. 

[Fegato]

Bias frames are a problem with some CMOS sensors. e.g. the IMX294 sensor in the ASI294MC Pro. I think this is due to particular (peculiar) characteristics at very low exposure times. For this reason, with this camera, I always shot flats and flat darks, no bias. Darks were necessary due to amp glow.

However, I think most CMOS sensors, and certainly the recent sensors, are absolutely fine. I have the ASI2400MC Pro (similar characteristics to ASI2600MC Pro) - and a single master bias frame works well with my flats, which saves me time, and avoids the problem of having to upset the image train on the RASA every time I take the flats, as stated above. I do use darks, but I suspect they are largely unnecessary. When cooled to -10, every master dark I have from 10s to 300s has pretty much identical stats, no amp glow, very few hot pixels.

As for flats - well, if you have dust motes, you will need them. And if you have vignetting (which I certainly do), you will also need them. If not, maybe not!

[Fegato]

NB  Yes that flat looks strange. How are you taking the flats?

[ollypenrice]

2 hours ago, wimvb said:

I believe that @gorann doesn't use any calibration frames with this camera. And the general consensus seems to be that bias frames on cooled cmos cameras add more problems than they solve, because the bias signal varies between lights/darks and bias frames.

Regarding the vertical lines, what exposure time did you use? Maybe just increasing the exposure will remove the banding.

That's my understanding as well. CMOS bias are unpredictable.

Let's think when you might use bias with CCD: 1) As darks for flats (AKA flat darks) which is fine for CCD but not for CMOS.)  2) Instead of darks, particularly if using a bad pixel map as well. (I like this with my Kodak full frame CCD but it won't work for CMOS.) 3) As a reference frame for stacking software which can match darks of the wrong exposure time to lights. (I don't see this working with CMOS bias either.)

The bias signal is contained within the dark frame in any case so it must not be double-subtracted. In a nutshell, why would you want bias at all? I can't think of a reason with a CMOS camera but I'm new to them.

10 minutes ago, tooth_dr said:

Mmm that’s an odd looking flat. 

I think I've commented on this flat elsewhere but what I find odd is the absence of vignetting in just the lower left corner. That makes me very suspicious.  I'm not surprised by the red-green gradient because we see that in our lights from this camera, though in our case it's rotated 90 degrees.

The image is now looking good with just a hint of the vertical banding which longer exposures might overwhelm, as Wim says.

Olly

[StuartT]

17 minutes ago, Fegato said:

NB  Yes that flat looks strange. How are you taking the flats?

8 minutes ago, ollypenrice said:

The image is now looking good with just a hint of the vertical banding which longer exposures might overwhelm, as Wim says.

Olly

the image is now looking good because I used no calibration frames at all! Makes life a lot easier, for sure.

[Budgie1]

Okay, so I had a play with the Lights Only version you posted and came up with these two.

Both slightly over processed to show what's there in terms of detailed data and and the vertical banding, although the banding isn't really that bad in this one. I did both standard OSC and used the Hubble Pallet as I find it shows more detail in the image.

Overall I think you've got great data there and, apart from the banding, it's clean.

[Dazzyt66]

I think this is a greatly informative thread @StuartT - and it proves 2 things definitely:

1. You are definitely capturing some great data!

2. As I think we all find as we move through the AP journey - processing is the absolute key on how that data transfers to image quality.

I'm sure with all these great folks offering advice you'll find the solution. One thing I found when trying to see what throws up the curve balls is processing the image with one set of cals at a time - you'll then definitely see where the issue lies. I would try just darks and see what happens...

[StuartT]

25 minutes ago, Budgie1 said:

Okay, so I had a play with the Lights Only version you posted and came up with these two.

Both slightly over processed to show what's there in terms of detailed data and and the vertical banding, although the banding isn't really that bad in this one. I did both standard OSC and used the Hubble Pallet as I find it shows more detail in the image.

Overall I think you've got great data there and, apart from the banding, it's clean.

 

oh boy!! how did you find all that blue? I don't get that in Astro Pixel Processor...

[Fegato]

Just to add to my comments above, I have found the article linked below really useful. Although it's aimed at Pixinsight users, the first 5 sections are pretty generic, and give an excellent background to calibration. There is also some discussion on CMOS sensors and use of bias frames, and links to other material on this (the long Cloudynights post referenced in note 7 that points you at Jon Rista's comments is quite entertaining, come fraught / hilarious in places, if you've got time on your hands to look at it). Bias CAN be used with most CMOS cameras, and it certainly makes life easier.

https://pixinsight.com/forum/index.php?threads/for-beginners-guide-to-pis-imagecalibration.11547/

However, I suppose if you really don't need flats, then with these new CMOS sensors cooled to a suitable level, maybe you can get away with no calibration at all, particularly when dithering. As I have to use flats, I do the whole shebang -  bias and darks are a fixed set of masters that I don't need to shoot each time, so add next to no extra work for each image, just a bit of computing time.

 

[Budgie1]

4 hours ago, StuartT said:

oh boy!! how did you find all that blue? I don't get that in Astro Pixel Processor...

I processed it differently using what's called the Hubble Palette to produce a false colour image. You normally see this used with narrowband images for Ha, OIII & SII but you can create it from broadband images as well.

Basically you split the colour image into its three channels of R, G & B, then throw away the Blue channel because there isn't much data there. You now need to create a new Blue channel image by combining 60% of the Red channel with 40% of the Green. Then combine the channels using Red for the Red channel, Blue in the Green channel and Green in the Blue channel.

Once combined again the image comes out with blue & yellow/gold instead of mostly red.

I use PixInsight for processing but if you do a Google for "OSC Hubble Palette"  for Astro Pixel Processor or PhotoShop you should find tutorials to do the same with them.

Having said all this, I don't want to add confusion or complication into the mix and this method may be something to put into your back pocket for a rainy day, when have time to sit down and play with it. 

 

[StuartT]

11 hours ago, Budgie1 said:

I processed it differently using what's called the Hubble Palette to produce a false colour image. You normally see this used with narrowband images for Ha, OIII & SII but you can create it from broadband images as well.

Basically you split the colour image into its three channels of R, G & B, then throw away the Blue channel because there isn't much data there. You now need to create a new Blue channel image by combining 60% of the Red channel with 40% of the Green. Then combine the channels using Red for the Red channel, Blue in the Green channel and Green in the Blue channel.

Once combined again the image comes out with blue & yellow/gold instead of mostly red.

I use PixInsight for processing but if you do a Google for "OSC Hubble Palette"  for Astro Pixel Processor or PhotoShop you should find tutorials to do the same with them.

Having said all this, I don't want to add confusion or complication into the mix and this method may be something to put into your back pocket for a rainy day, when have time to sit down and play with it. 

 

this is very interesting. Thanks! No doubt somewhat has scripted a Photoshop plugin for this process too.

I assume you worked off my FITS file?

But I guess from a more philosophical point of view, it does make me wonder about the issue of 'realism' in astrophotography. To what extent does swapping out and replacing an entire colour channel really reflect any kind of reality? or it it more artwork? Though it would reasonably be argued that all the stuff we do in AP (even stretching) is 'unreal'... what does reality even mean in this context? (answers on a postcard 😉 )

Edited October 5, 2021 by StuartT

[ollypenrice]

3 minutes ago, StuartT said:

this is very interesting. Thanks! No doubt somewhat has scripted a Photoshop plugin for this process too.

But I guess from a more philosophical point of view, it does make me wonder about the issue of 'realism' in astrophotography. To what extent does swapping out and replacing an entire colour channel really reflect any kind of reality? or it it more artwork? Though it would reasonably be argued that all the stuff we do in AP (even stretching) is 'unreal'... what does reality even mean in this context? (answers on a postcard 😉 )

My feeling is that the proper Hubble pallette is perfectly valid as a colour map of gas distribution. That's why it was invented. SII is mapped to red, Ha to green and OIII to blue, so it's a full tricolour system. A geology map might map different rock types to different colours. There is no resemblance intended between the natural colour of the gas or rock and its colour in the image.

'Fake' Hubble Palettes don't have this validity but people like making images that way because they like the way they look, which is fine by me, though I don't do it myself. If they are bicolour they are not, in truth, Hubble palette at all, they just end up with similar colours. In truth I think you can tell bicolour from tricolour anyway because there is a more restricted gamut in bicolour.

The other way to exploit NB data is to use it to enhance the colour channel in which it really belongs, so Ha to red and OIII on the green-blue border.

Olly