Struggling (less) with PI

[Marky1973]

Hi all. I'm after some help. 

The background - A night out (or a few nights) with M81:

36 x 600s LIGHTS

23 x 300s Red Blue and Green (binned 2x2)

All the requisite FLATS/DARKS/LIGHTS

And I am working through Warren's PI book, but am struggling to get anything out of the images, there just seems to be too much noise in the images and very little detail when all stacked up.

If someone has five minutes, can they have a look at the images and have a quick fiddle to see what they can get out of them. I am sure it is just my inexperience, but it would be good to have that confirmed! :-) I have attached the separate CALIBRATED, COSMETICALLY CORRECTED, REGISTERED and CROPPED masters that I have produced following Warren's tutorials. Once I start getting into DBE/ABE etc, it just seems to get worse. I hoped that 6 hours of Lum and just under 6 hours of RGB would produce something half-decent. I may be rushing things, but even just fiddling about with the master, and combining RGB and then the full LRGB seems to really struggle to see any great detail - stretching seems to produce blotchy and noisy colour very quickly.

I am going back to the start to see if anything can be done differently, but if anyone does have a chance to have a quick look, it would be much appreciated - sorry to ask, but it is doing my head in a little and I just need to know if I am starting from a decent position or perhaps my data is just a bit rubbish - perhaps the LUM frames were just too long and there is too much LP involved?

BLUE INTEGRATED MASTER cropped.xisf

GREEN INTEGRATED MASTER cropped.xisf

LUMINANCE INTEGRATED MASTER cropped.xisf

RED INTEGRATED MASTER cropped.xisf

Thanks in advance and no problem if no one can help... I will soldier on!

[wxsatuser]

Not a full blown expert on PI but I get a reasonable background on the luminance with DBE but there appears to be
a sort of halo around M81. Wonder if something went wrong with calibration, it does appear quite noisey to me
but looks like reasonable detail in the galaxy itself.

May be someone with more experience can point you in the right direction.

[sloz1664]

Hi Marky,

I've checked out your subs and yes there is substantial noise and patchy gradients. I don't know how much you have cropped your images but looking at the noise level it looks as if it is a significant amount. Can you post your original master files, this may determine whether it's coming from your processing or just effects of seeing, substantial cropping or just plain old light pollution.

Steve

[Marky1973]

4 hours ago, wxsatuser said:

Not a full blown expert on PI but I get a reasonable background on the luminance with DBE but there appears to be
a sort of halo around M81. Wonder if something went wrong with calibration, it does appear quite noisey to me
but looks like reasonable detail in the galaxy itself.

May be someone with more experience can point you in the right direction.

Thanks Mike - it is the Halo that is bugging me. And it does appear to be in the original subs as well - I have attached a couple to this post as Steve suggested. I have also attached a screengrab of an open original in ARTEMIS where I did the capture. It is over-exposed, but I didn't think it would be "too" bad, but perhaps I need to keep a better eye on my histogram. 

Like you the DBE background map looked "okay" but didn't really seem to improve the image. I tried it without FLATS (and with FLATS and without DARKS) in the calibration as well, but it wasn't any better. I am not 10% sure my FLATS are as good as they can be, but I am working on that as well!

31 minutes ago, sloz1664 said:

Hi Marky,

I've checked out your subs and yes there is substantial noise and patchy gradients. I don't know how much you have cropped your images but looking at the noise level it looks as if it is a significant amount. Can you post your original master files, this may determine whether it's coming from your processing or just effects of seeing, substantial cropping or just plain old light pollution.

Steve

Thanks Steve. I've uploaded four of the original SUBs from across the evening. They were taken on 29th November, a fairly clear night and no moon and with an IDAS P2 LPF. They started lowish, but ended quite high up, just before M81 crossed the meridian. 

It wasn't a big crop, just the borders really, just enough to get rid of the stacking artefacts on one of the binned RGB sets and the crop was done dynamically.

Thanks you both your help!

(ignore the Hs prefix, I forget to change it from the previous session)

Ha_583.fit

Ha_589.fit

Ha_602.fit

Ha_618.fit

[wimvb]

Had a look at your data as well. The integrated images still have a lot of vignetting, which (together with your findings that using no flats doesn't make much difference) indicates that your flats don't do their job. Are you sure that the illumination of your flat box is evenly distributed?  You could try taking flats some other way and using these instead. (Don't worry about dust bunnies at this stage). I have found that with my DSLR, my flats work best when exposed such that the histogram in the camera display is at about 1/3 from the left edge. From what I've read on this forum, this seems to apply to ccd flats as well.

The dark halo in your images may very well be a brighter ring, as is visible in the attached jpeg. Since this is an optical issue, good flats should take care of it.

The good news is, you also seem to have captured some IFN. This will make DBE sample placement more critical.

BTW, any image will seem noisier after DBE, because DBE takes away light signal. It doesn't take away the noise associated with this light.

 

Just my € 0.02

 

[Marky1973]

Hi Wim. Thanks for taking the time to have a look. I am using  Huion light box - it is new, so I have been using it with all the kit on the scope, held a few inches from the end of the scope, with a white t-shirt over the end.

The screengrab below shows an individual FLAT, the same FLAT stretched and a stretched CALIBRATED/INTEGRATED Master FLAT

What I am not sure about is whether the circular area in the middle is the problem - the fact the lack of flats in the PI process suggests not, but I don't feel it is right.

I have exposed them to around 1/3 of the histogram, but could perhaps be a bit longer - although I doubt that will lose that circular pattern.

Might try the BPP script in PI next to see what that comes up with!

Will keep practicing!

[wimvb]

If your flat light box has a perfect even illumination, then anything not even on the image when you use it together with your optics (scope or lens), has its origin in the optical system. This is the purpose of flats. The master flat that you create to calibrate your light frames, should show the same vignetting and optical "defects" as your light frames. To me it seems that it does. If your flat calibration works, the light frames will have an even background after calibration. But since you get the same result with and without master flat calibration, it seems to me that your calibration routine is the problem. (Otoh, you might want to investigate the cause of the brighter circle in both your light frames and your master flat. That cause should be somewhere in your optics.)

[Marky1973]

1 hour ago, wimvb said:

If your flat light box has a perfect even illumination, then anything not even on the image when you use it together with your optics (scope or lens), has its origin in the optical system. This is the purpose of flats. The master flat that you create to calibrate your light frames, should show the same vignetting and optical "defects" as your light frames. To me it seems that it does. If your flat calibration works, the light frames will have an even background after calibration. But since you get the same result with and without master flat calibration, it seems to me that your calibration routine is the problem. (Otoh, you might want to investigate the cause of the brighter circle in both your light frames and your master flat. That cause should be somewhere in your optics.)

Thanks Wim - been following the PI box for calibration, and did a Batch PreProcessing script run which din't seem to make any difference either. Have stacked all in DSS as well and the results are fairly similar...very confusing...will try again, but you have me worried now about the optical train....Atik 414Ex - Filter wheel - Filters - ED80 Reducer and the ED80 itself - plus a few connectors. Having no experience in this is there anything in that list that could be a problem? It is all tight and secure!

 

[wimvb]

Sorry, no idea. I'm still at the dslr stage. But there should be a bunch of people on this forum with a similar setup, since all your components are quite standard. Maybe if you ask around in the equipment section. I also know that people have had problems with flats before, but Idon't know how they solved it. You can always do a search in old threads.

Good luck

[Marky1973]

No problem, thanks for you help Wim!

[Marky1973]

Well, I've worked through everything from the start of the PI book and managed to get my first LRGB image together (or at least the first one that doesn't look like it was drawn by an ogre with blunt crayons and fat thumbs). It's still a long way off where it could be, I would like to see more of the faint nebulosity that I first thought was noise....thanks to you all for pointing me in the right direction.

So this was 6 hours of 10 minute Lum and just under 6 hours of RGB combined (5 minute subs binned 2x2). All processing done in PI. I have definitely overdone the deconvolution, but I just need to practice with that more and start exploring the use of Masks in more detail - masked stretches etc - before moving on to the more advanced stuff. So the picture isn't the best it probably could be (well, it definitely isn't) but it gives me the confidence to keep trying and not think the PI investment is foolish! ;-)

(I also don't think having only a laptop to process on is much help either as the images always seem to look different on every screen!)

(in fact, now I have uploaded it, it looks even worse and I have not done the background any favours....but hey ho, every mistake is another lesson learned - onwards!)

 

 

[wimvb]

Nice image. It certainly isbetter than my most recent attempt.

The background can be corrected by carefull sample placement in dbe. Try larger than default samples, few of them, and in positions where you know you have true background. You can start with automatic sample placement, but then examine each sample individually for best position.

[Marky1973]

Cheers Wim.....the more I look at it, the more I think it could be better though. Interestingly, ABE seemed to work better (or as good as DBE) as the automatic placement of the samples was all over the place and not, from what I could see, on neutral background. I think I may have made this harder for myself as I didn't realise, until posting the original question, that there was so much faint nebulosity in the region, making it tricky to find neutral background. ABE and DBE both helped bring that out....and I helped push it back into the background :-)  - so the next plan is to figure out how to bring it back after LRGB combination (if not before) and dial-back a bit on the DECON and saturation.

Cheers

 

Mark

 

[wimvb]

I agree, sometimes abe is better. Dbe needs very precise sample placement to get any decent results. But if you use abe, samples can be all over, even over stars, and you still can get good results. Maybe it has something to do with statistics, but it generally is worth it to compare both methods.

[iwols]

hi mark i have the same setup as you(ed80 414ex) and i too get the coloured stars you have (is that normal),...like yourself finding pi a very large hill to climb ,but it certainly looks to be very much in the right direction,and i too seem to get quite a lot of noise sometimes,but seems to depend on what the image is,,orion neb always has no noise but the triangulum and horse head always seems to be noisy,time,wind,clouds and moon all add the to problem of image time but ha ho,we will get there in the end

[Marky1973]

On 10/01/2017 at 15:37, wimvb said:

I agree, sometimes abe is better. Dbe needs very precise sample placement to get any decent results. But if you use abe, samples can be all over, even over stars, and you still can get good results. Maybe it has something to do with statistics, but it generally is worth it to compare both methods.

Definitely worth comparing, especially with the "difficult" background on this image....but it is so much less of a pain to do that with the smaller files from the ATIK than the RAW files from my DSLR...

[Marky1973]

9 hours ago, iwols said:

hi mark i have the same setup as you(ed80 414ex) and i too get the coloured stars you have (is that normal),...like yourself finding pi a very large hill to climb ,but it certainly looks to be very much in the right direction,and i too seem to get quite a lot of noise sometimes,but seems to depend on what the image is,,orion neb always has no noise but the triangulum and horse head always seems to be noisy,time,wind,clouds and moon all add the to problem of image time but ha ho,we will get there in the end

I'm developing a love-hate relationship with PI but, at the moment, I know that the shortcomings are due to my knowledge/experience rather than the software! As for the coloured stars, I presume there is some colour there, but I think I ramped the saturation up a bit more than was necessary - there is a setting in the RGB combination to boost it and I did that first time around as I was worried the image didn't look very colourful as it was....will try it without next time. I haven't got to grips with denoising just yet, but getting better with masks, so will be working on those processes as well....I may have said in other threads that i am not a very patient person often, just because the time I get on AP is so limited, but I am learning to try and be a bit more relaxed about it all....just ordered a copy of The NewAstro Zone System for Astro Imaging as that was recommended by Sara (Swag 72) for getting to grips with Photoshop and, as I have the license, I might as well work out how to use that properly as well!

I/we will get there in the end, so good luck! (I'm also tempted to join you in the C8 EDGE-HD world....although I am also still tempted by a C9.25.....)

[iwols]

hi mark re the c8 , i have to be honest with you ive only used the c8 twice  as the dso imaging  on my ed80 seems to have taken over the limited time i have with a scope,so much to do and so little time after clouds ,bad weather ,moon,work etc 

[Marky1973]

Yeah that's why I am thinking of a (cheap) 9.25 as well....for those nights when the moon is out or I just can't be bothered with dragging EVERYTHING out of the house... don't want to spend a fortune as I know it will be a once in a while thing.....but do miss looking at the planets and moon!

[iwols]

see where you are coming from mark,think a pier might be the way to go with a small wooden hut round it,and just remove scope camera etc when not in use,my setting up used to be about 1 - 2 hours and must admit on a good night ive knocked it back to 30-40mins,but really is, all time i could be playing,only good thing is im using everything remotely from inside the house when set up

[iwols]

and mark just been looking at your link,very impressed especially the bubble in mono,which ha filter did you get and also how did you get the cross in the larger stars was this something post processed as i really do like that touch

[Marky1973]

2 hours ago, iwols said:

see where you are coming from mark,think a pier might be the way to go with a small wooden hut round it,and just remove scope camera etc when not in use,my setting up used to be about 1 - 2 hours and must admit on a good night ive knocked it back to 30-40mins,but really is, all time i could be playing,only good thing is im using everything remotely from inside the house when set up

Yeah I think I may be looking at more remote working over the summer this year.

1 hour ago, iwols said:

and mark just been looking at your link,very impressed especially the bubble in mono,which ha filter did you get and also how did you get the cross in the larger stars was this something post processed as i really do like that touch

Thank you! I was very happy with the bubble, especially without any calibration frames - the bonus of Ha for these targets can't be underestimated! I got the 7nm Ha filter from FLO

https://www.firstlightoptics.com/narrowband/baader-narrowband-ccd-emission-line-h-alpha-filters-125.html

Have found that some spikes appear naturally on larger stars but the Bubble was in Photoshop using Noel Carboni's astro tool set

http://www.prodigitalsoftware.com/Astronomy_Tools_For_Full_Version.html

Which is great value for what you get - although I am only scratching the surface with PS at the moment....the problem with the bubble though is that some bright areas in the actual bubble were picked up as stars, so got a few I didn't really want!

Hope to get out a bit more with the Ha filter this year again!

[alan4908]

On 1/2/2017 at 01:08, Marky1973 said:

I am going back to the start to see if anything can be done differently, but if anyone does have a chance to have a quick look, it would be much appreciated - sorry to ask, but it is doing my head in a little and I just need to know if I am starting from a decent position or perhaps my data is just a bit rubbish - perhaps the LUM frames were just too long and there is too much LP involved?

Hi Marky

I've had a quick look at the LRGB data that you posted. My conclusion is that you may have flat errors which are giving you the halo effects in your original data. To correct for these, you need to select the correction "division" in DBE. Anyway, after a little processing, I ended up with this:

 

Alan 

 

[Marky1973]

5 minutes ago, alan4908 said:

Hi Marky

I've had a quick look at the LRGB data that you posted. My conclusion is that you may have flat errors which are giving you the halo effects in your original data. To correct for these, you need to select the correction "division" in DBE. Anyway, after a little processing, I ended up with this:

 

Alan 

 

Hi Alan

Wow, thanks for having a look at that. I couldn't work out whether it was flat error or faint nebulosity which surrounds M81. Did you use the flats in this process? I am well aware I have a long way to go, but if you wouldn't mind sharing your workflow that would be great - just the overview of processes, wouldn't expect the full details....and only if you have time.

Cheers

 

Mark

[alan4908]

49 minutes ago, Marky1973 said:

Hi Alan

Wow, thanks for having a look at that. I couldn't work out whether it was flat error or faint nebulosity which surrounds M81. Did you use the flats in this process? I am well aware I have a long way to go, but if you wouldn't mind sharing your workflow that would be great - just the overview of processes, wouldn't expect the full details....and only if you have time.

Cheers

 

Mark

Hi Mark

I just used the data you uploaded (at the top of the post).  

For my LRGB processing, I tend to follow the techniques and workflow of Adam Block, which uses a combination of PS, Pixinsight and CCDstack for post processing data.

In a nutshell: after calibrating your data, you process the L and RGB data separately, only sharpening the L. When you have some reasonable colour within your image, you then combine these and subsequently boost the colours and correct the image for cosmetic imperfections.  If you want to know the details,  then I suggest you purchase a copy of  "dimensions of Photoshop" video tutorial - see  http://www.adamblockphotos.com/store/c3/Tutorials.html

Alan