Stacking with washed out colour

[Astronut1982]

I took 21 x 600s subs of M81 & M82 the other night but have been unable to successfully stack them. No matter what I do or try I cannot get anything other than washed out colour which is almost non-existent. I have tried many different settings but nothing has helped.

I originally used DSS which I have never had a problem with before. I stacked an image of the flame & horsehead from the previous evening where the camera & imaging train hadn't been touched & that image had no issues with colour so nothing should have changed.

I then stacking in PixInsight using Harry's tutorial but that hasn't worked either.

I'm pretty sure that there's nothing wrong with the data as I can see colour in every single sub, I'm sure that focus is fine & I have the correct Bayer patterns selected for my QHY8 as well so I think I can rule those things out. I just can't see what's wrong & it's now starting to hurt my brain trying to work it out so could do with a fresh pair of eyes going over it.

I was wondering if I stick the data onto dropbox, does anyone fancy having a go at stacking it to see if they can get it to stack with colour ok? If anyone does manage, can they give me an idea of program used & settings. I can stack with PI or DSS so preferably if you can use either of those so I can have a bash at it afterwards that would be great.

https://www.dropbox.com/sh/up72rae4f7py572/AABy40BWkDD0Vd4lIrXXsozga?dl=0

Thanks in advance,

Jeff

[FunkyKoval35]

Well.. I downloaded 5 subs and there are black and white only - no color at all.

 

Maybe You changed some setting on your camera and now you are getting b-w images?

[Astronut1982]

That's weird, they're all definitely colour as it's a OSC camera & it only shoots in colour. Also when I open in DSS, the shots are in colour.

Here is a screen shot from Maxim showing a single sub with M82 in colour.

Jeff

[Astronut1982]

Here is a screen grab of the first 5 subs stacked. As you can see they are in colour, just not in the correct colour or having very much of it.

Jeff

[Sp@ce_d]

Well.. I downloaded 5 subs and there are black and white only - no color at all.

Untitled.jpg

 

Maybe You changed some setting on your camera and now you are getting b-w images?

Did you debayer? I've just pulled one down for a look, definitely colour subs

Jeff, your light train needs a clean   I'll pull the rest down & take a look.

[sharkmelley]

I downloaded one of the lights.  It is a raw image that just needs debayering. Once you've debayered you have a full colour image.

Mark

[Astronut1982]

Thanks, hopefully one of you can see where I've gone wrong.

I actually had it all apart today to give it another clean so hopefully it is clean now. That squiggly thing on the left near the top is a thread from the the sponge thing surrounding the chip, luckily I wasn't imaging anything large or it would've been a major pain in the butt.

Jeff

[Sp@ce_d]

Hmmm.. It must be 2 years at least since I looked at OSC subs. I've done a really quick stretch on these. I haven't pulled down the flats/darks etc.. so No Calibration, just a default batch debayer in Nebulosity (because I've never done it in PI).

A very quick defaults Staralign > integration>crop>backgound neut>DBE> colorcalib>Hist stretch in PI

I concur... There is some amiss here... Not much colour there I can see.. looking at the fits these were done a couple of nights ago. I was shooting Ha & SII then & they came out ok... you have the same skies so... are you using a LPF? Theres a lot more gradient on these than I get running through an IDAS P2. But I'd expect to see some colour noise anyway....

The single subs show more colour... must be the debayer... hmmmm have to think

[Astronut1982]

I'm using a IDAS 2 LPS with the camera Graham.

Here is an image I took the night before using the exact same kit with all software running the same, the colour has definitely come through on this one - very strange!

Jeff

[sharkmelley]

The answer to this will be what format the acquisition software is saving the files.  Which software are you using for acquisition?  Is it possible that a setting has been changed to save the files as raw data (non-debayered) whereas previously you saved them debayered?

Mark

[Sp@ce_d]

I'm using a IDAS 2 LPS with the camera Graham.

Here is an image I took the night before using the exact same kit with all software running the same, the colour has definitely come through on this one - very strange!

Jeff

So that looks ok for colour. The thing is the individual subs show colour that stacking seems to wash out. I haven't played with OSC data for some time so the debayer method is bothering me. I've only done OSC with my modded 1000D & not used it for a couple of years. I seem to remember reading somewhere the debayer method for the QHY's was slightly different. Not sure if it was here or on another forum involving sheds..

But it worked ok for the horsehead.. so, fault finding. Look at the fits header info.. is that temperature right?. See if there's anything different. Go into PI & check the history explorer for what you did on the previous night.

I'm surprised to see that much gradient with the IDAS but maybe LP has got worse for us in the last couple of years with OSC   (The new D1 is supposed to be better for OSC)

I must say I've had subs where the sky looked brilliant & they just came out below par... but I was shooting OIII & SII the night you did the horsehead & SII on the night you did the M81/82. Mine seem fine.. & our skies (apart from local light pollution) are pretty much the same.  Your single subs show colour however, they don't show a lot of difference between them.. certainly not as much variation in sky background.. etc as I see in mine. Have they been neutralised or something?

[Sp@ce_d]

The answer to this will be what format the acquisition software is saving the files.  Which software are you using for acquisition?  Is it possible that a setting has been changed to save the files as raw data (non-debayered) whereas previously you saved them debayered?

Mark

Possibly.. Jeffs using Maxim looking at the .fits. If he's doing some auto calibration with it that could  be it.

[msh1]

just a quick process in nebulosity how does this look

[Astronut1982]

Yeah, I'm using Maxim but don't do any calibration whatsoever until I put them into DSS or PI.

All I do is set the type of sub [ie. lights, darks etc], suffix, filename, exposure length, number of subs & where to save the target & then just let it do it's thing. At the end of the run I take the subs from the folder & fire them into DSS & let it stack using the auto settings as they've always worked for me. After that I save it & open it in PI where I do my processing.

I've had a look at the fits header window for M81 & HH & everything is the same apart from the date. The temp is a bit weird though as it was definitely not 25 degrees! It always says that but it isn't right because without a dew heater on the barrel close to the chip, it freezes over. Even on nights before I had the dew heater & it would freeze up, it still read 25 degrees.

I had a look at the history explorer in PI but it only covers M81 & M82, doesn't have anything for the HH so I can't check that unfortunately.

That still has too much blue msh1, the starburst region in M82 should be glowing red but I just can't get it to do so. Thanks for having a go though.

Jeff

[msh1]

i must admit the colour is washed out and to blue. not sure how much a difference this would make have you changed the gain and offset at all 

[sharkmelley]

The colour looks fine to me (see below from PixInsight) - just a bit of RGB balancing required after this.  When you debayer, make sure to select the correct Bayer pattern for your sensor otherwise you do get strange colour casts.

Mark

[Oddsocks]

Hi Jeff,

Here is my attempt at your data, processed in PixInsight using the sequence: Script Batch Pre-Processing, Crop, DBE, Colour Calibration, Histogram Transformation, HDR Multiscale Transformation and ACDNR noise reduction. TIF downsized and converted to JPG in Photoshop.

Needs a little selective colour boost with layer masks in Photoshop and cropping a little smaller to hide the "snake" artefact in the top left corner but given it is Feb 14th I have certain other demands on my time this evening!

William.

 

[Astronut1982]

Thanks for having a go guys.

Your attempt is the best so far William but I don't know how I'm not getting any colour like your version as I'm using the exact same processes in the exact same order?! Is there any chance you could let me know what settings you used in BPP to see where I'm going wrong as It must be in BPP settings where I'm screwing up.

Jeff

[Oddsocks]

Hi Jeff,

I am still very inexperienced with image processing in PixInsight and have to refer back to Harry's tutorials often.

I have been processing in Maxim Dl for about fifteen years and in the past have achieved reasonable images but PI is my favourite now even though processing takes longer and there are a lot more things to tweak but the end results are usually better.

When I posted yesterday we were on the way out for the evening so it was a little hurried for which I apologise, the script batch pre-processing settings I used are posted in screen shots below.

The only changes from the default script settings are in options, "CFA images" box is ticked for auto debayer and "Optimise dark frames" is un-ticked.

After the script batch pre-processing ran I did not take default output image for further processing as Harry says in his tutorial, the default output file is often not the best image possible.

So the next step was to run the process "Image Registration" -Star Alignment- module with the calibrated debayered frames that were output from the batch pre processing script and the frames output from this process were saved to a new folder.

Next step was to take this new stack of registered images and combine them in the process "Image Integration" module with default settings.

Next step was to apply the process "Screen Transfer Function" to the output image from the integration module and at this stage the image looks awful with strong colour casts and gradients.

Next step was to crop the image to remove any white or black borders so as not to confuse the later processing steps with false black level and peak white values.

(From my limited experience with OSC images I have found that running DBE on the combined RGB image does not always achieve good results, I think this is because the gradients in the colour channels are not all the same, as the image acquisition run proceeds during the night the gradients from light pollution change and affect the different colour channels to varying amounts so for OSC images produced from long acquisition runs I always split out the RGB channels and run DBE separately on each channel).

Next step therefore was to split out the three colour channels using the "Split RGB Channels" process from the shortcut on the top toolbar.

Next step was to run the process "Dynamic Background Extraction" on each of the three separate RGB images.

In your images the signal to noise ratio is very low so I upped the detection tolerance to 1.5 before generating the sample points and then gradually reduced the tolerance by .2 steps and used "resize" to check each sample stayed "green", in the three RGB channels I ended up with around 0.6 as the final setting with a few "red" outliers showing.

After running DBE then use the STF tool to check the output auto masks from the RGB channels, the mask in each colour channel was markedly different for your image.

Next step is to normalise the three colour channels before recombining them.

Open the process "Linear FIT" from Intensity Transformations module.

Run the STF tool on each of the three open RGB DBE flattened images and choose the image with the least noise as the reference image for Linear Fit.

For your data I used the green channel as the reference, this is normal for Bayered OSC using RGGB masks as there are twice the number of green pixels as red or blue so the signal to noise ratio is higher in the green channel, then apply the Linear FIT  process to the red and blue channels.

Next step is to recombine the RGB channels using the process "Channel Combination" from the Channel Management module.

Now run the STF process on the combined RGB image, the image should be fairly well colour balanced at this stage but if required you can run the process "Colour Calibration".

Next, run Histogram Transformation,

Next, run HDR MultiScale Transform,

Next, run ACDNR noise reduction,

Next, run the process "Saturation Boost" from the Intensity Transformations module (missed this from my first post).

And that was it for my attempt except for downsizing and .jpg conversion in Photoshop for uploading to SGL.

I stacked all your light frames but about a quarter of them had slightly oval star shapes which has probably robbed some detail from the final stacked image, I guess you have some backlash in one of the axis that needs tweaking a little, if there had been more light frames I would have ditched the frames showing the oval stars and that would have improved the resulting image resolution.

In your original post you said you didn't know why the same set up you used for the Horsehead produced so little colour and suspected something had gone wrong with the acquisition for M81 / M82 but the two targets are very different, the Horsehead is local and very bright so the signal to noise ratio is high, also the colour is in vast blocks covering hundreds of pixels so processing this type of image is relatively simple.

The M81 /M82 target is very different since it is dim and small so the signal to noise ratio is low and with an OSC camera the chance of the red starburst regions actually corresponding with a red pixel is reduced.

When I use a OSC,cooled or DSLR, for dim and small targets I usually set up mount dithering to move the camera a few pixels between each sub, this will increase the chance that the dim and small colour details actually correspond with one of the correct colour Bayered pixels.

Set up images follow...

William

 

 

 

[Astronut1982]

Thanks for that William but unfortunately it hasn't helped as I'm using the exact same settings as you but it isn't working for some reason!! I've even just tried them again & it's just coming up blue. This is really starting to annoy me so I think I'm just gonna throw away all the data as it clearly isn't right & I can't get it to work no matter what I do.

I Imaged M81 & M82 last January for the supernova using the exact same equipment & processed with the same programs & the colour was extremely vibrant without having to do a saturation boost so something has definitely gone wrong somewhere this time. Below is a link to that image, I've got the black point wrong but you can clearly see the colour in that image.

I could have sworn the conditions were as good as the previous evening but maybe it wasn't after all. It is supposed to be clear tomorrow so might try & have another bash to see if that is the problem.

You probably are right about the backlash though, I need to get that sorted sometime soon.

Jeff

[sharkmelley]

Hi Jeff,

The 2 PixInisight parameters most likely to cause your blue colour problem are "Up-bottom FITS" and Bayer Pattern "RGGB".  If you do indeed have those set the same as William does then I really don't know what could be going on.

Mark

[Astronut1982]

Hi Mark,

Yeah I'm pretty stumped as I checked it a couple times over to make sure & I am using the same settings as William. Just on the off chance,  I even tried turning off the UP-bottom FITS & changed the Bayer pattern to GBRG after finding this old post via Google-

http://stargazerslounge.com/topic/193753-pixinsight-debayering-qhy8/

It actually got me close but not anywhere close enough to call it a success. It is to be clear tonight so I might have another go to rule out the conditions being the problem, I'm just going to leave this one though as I just can't see the problem.

Thanks for your help & advice though.

Jeff

[Astronut1982]

I've just remembered that when I used the camera to take these images I didn't plug in my powered USB when using the camera which I normally do. Does anyone think that have something to do with it? If it isn't getting enough power, I could imagine that this might affect the image quality.

Jeff

[Oddsocks]

Hi Jeff,

Sorry to hear you didn't get any further with your data.

The QHY8 gets all its power for internal electronics via the external 12V 4A power supply, the USB cable only handles the digitised control signals and image transfer.

If the USB cable was suffering from power loss it would not affect the internal parameters of the camera amplifiers or CCD operation but only cause a loss of control or failure to transfer a completed image.

Looking at your light subs the signal to noise ratio is very low so perhaps there was some thin high cloud or low level mist about.

I did wonder if perhaps the cooling was not working on the camera but there are not many hot pixels visible so this seems unlikely but if you want to test this at some stage then place a blocking mask of some kind over the CCD window to stop light getting to the sensor, wrap the camera in a freezer bag or cling film with just an opening left for the USB and power cords and put it in the freezer or ice box for half an hour or so to cool. Then take a dark frame of five minutes while the camera is still cold.

Take the camera out of the freezer and leave for a few hours wrapped in the freezer bag / cling film while the camera comes back to ambient temperature, now remove the freezer bag or cling film and connect the camera again and leave it for an hour so that the camera electronics warm up but the CCD itself is cooled by the internal Peltier element, now take a second five minute dark frame.

When you apply max stretch to the two dark frames they should appear almost identical, if the background level of the out-of-freezer dark frame is much higher than the in-freezer dark frame then possibly the CCD cooler is malfunctioning but my money would be on the seeing /weather being the problem.

I had a last go at your data at lunch time today trying a different integration setting in PI just to see if there was anything else there and have attached the image below.

Hopefully you will get back on track with some new data.

William.

 

[Astronut1982]

Thanks for the advice about checking if cooling works but although I could be wrong, I'm pretty certain it is working. If I forget to switch the dew heater on to full & leaving it for a few minutes before switching the camera on, the chip freezes over instantly. Although I now know how to check to make sure & may do so just to confirm it.

I think you're probably right about high cloud as I have taken this image of NGC 4565 last night which shows much more colour:-

I also got roughly 24 x 600s subs of M81 & M82 which I have yet to process although there does seem to be more colour. I'll get that processed tomorrow to see how it has turned out.

Cheers,

Jeff