Yet another M31 - and some questions....

[Alveprinsen]

Ok, so I was out last night and managed to get a 4h 10 minute exposure of M31.

Lights: 25x 10 minutes iso 800

Darks: 15

Bias: 30

Flats: 30

As you can see, the noise is absolutely insane... Isnt 15 x 10 minute dark frames enough?

Towards the core however, things arent that bad... Because of the lower exposure of course. The image is not as "stretched" in the center:

How would I go about getting rid of... or rather - abvoid getting all those red specs? I take it they are hot-pixels?

Here towards the lower satelite galaxy you can clearly see the huge amounts of red dots/streaks/specs:

Can anyone explain to me just what this is, and how to best avoid it?

Sincerely, Alveprinsen.

[MartinB]

Hi Alveprinsen, you are asking how to avoid noise in astroimages, that's a big question!

The red and blue specks are the due to a loss of reliability of info contained in the pixels.  This happens when signal is very low and there is significant random variation between pixel levels. The signal to noise ratio only allows so much stretching before noise becomes a big issue in dim areas.

How to improve noise - get lots of signal, long exposures and lots of them.

Don't stretch beyond the limit of your data

Small amounts of noise can be dealt with when processing but this is very limited and if you over do smoothing you end up with ugly images.

Your signal to noise doesn't look too bad.  Everyone will have noise around the dim outer areas of M110 if they do a big enough stretch.  I think you have enough data to improve the look of this image a lot.  Something has gone wrong in the processing and you have ended up with the outer regions over stretched and the inner areas under done.

Keep playing with the processing.

[Alveprinsen]

The red and blue specks are the due to a loss of reliability of info contained in the pixels.  This happens when signal is very low and there is significant random variation between pixel levels. The signal to noise ratio only allows so much stretching before noise becomes a big issue in dim areas.

Don't stretch beyond the limit of your data

Your signal to noise doesn't look too bad.  Everyone will have noise around the dim outer areas of M110 if they do a big enough stretch.  I think you have enough data to improve the look of this image a lot.  Something has gone wrong in the processing and you have ended up with the outer regions over stretched and the inner areas under done.

Keep playing with the processing.

Thank you very much for your input Sir!

Ok... So what you are saying is that if I get more data, I can stretch further without ending up with allot of noise?

This exposure is 4 hours 10 minutes... Lets say I gather more, and end up at 8 hours, or 15 hours... Would that make a big difference, or would I eventually hit diminishing returns?

Sincerely, Alveprinsen.

[Alveprinsen]

Ahh, I just did a test...

I did one single stretch, and ended up with the following result. If I look close, I can see the noise... But its clear to me now that if I stretch that any further, its going to come out real clear. Same goes for upping contrast, or clarity...

I guess I need about 15 more hours of data.

[nmoushon]

Thank you very much for your input Sir!

Ok... So what you are saying is that if I get more data, I can stretch further without ending up with allot of noise?

This exposure is 4 hours 10 minutes... Lets say I gather more, and end up at 8 hours, or 15 hours... Would that make a big difference, or would I eventually hit diminishing returns?

Sincerely, Alveprinsen.

More data will allow you stretch your image more without adding as much noise. But there is a cut off point where no matter how much data you add you wont be reducing the noise anymore. Especially when using a noisy DSLR. Have a look at my 8hr M31 with my DSLR. I explain my own little experiment. Just keep in mind I used 2min sub where you are using 10min so you will be able to go past 8hrs...how much past I don't know.

[Alveprinsen]

More data will allow you stretch your image more without adding as much noise. But there is a cut off point where no matter how much data you add you wont be reducing the noise anymore. Especially when using a noisy DSLR. Have a look at my 8hr M31 with my DSLR. I explain my own little experiment. Just keep in mind I used 2min sub where you are using 10min so you will be able to go past 8hrs...how much past I don't know.

Hmm, I see....

My first attempt was using 2 minute subs. I noticed I hot diminishing returns after a while. Only after doing 10 minute exposures do I get details of the outer dust clouds.

I wonder... would my cheap Nikon D3100 handle 20 minute exposures? ... or perhapse it would just noise up anyways...

Anyhow, I will attempt to go as far beyond 8 hours as I can, and see how things turn out. With any luck, I might be able to push 4-5 hours of data tonight.

Alveprinsen.

[lensman57]

Hi Alve,

If you wish you could drop the original raw stack file witouth any stretching or colour balancing applied in DropBox and I see If more could be done with your data, 250 minutes of data should be plenty to play with but I have found both M31 and M33 difficult to process properly and quite noisy too.

Regards,

A.G

[Alveprinsen]

Hi Alve,

If you wish you could drop the original raw stack file witouth any stretching or colour balancing applied in DropBox and I see If more could be done with your data, 250 minutes of data should be plenty to play with but I have found both M31 and M33 difficult to process properly and quite noisy too.

Regards,

A.G

Hello.

I'll see about doing that within this weekend. I just came back from my observatory... I crashed my camera into my mount last night, and alignment was all messed up. Had to re-do everything...

Anyhow, camera is now running a 4h 30m shooting session of M31. 10 minute subs, iso 800. With some luck, the few clouds in the horizon will stay there... in the horizon... I had 9 hours of clear skies last night, 9pm - 6am.. I am hoping for something like that tonight as well. With some luck, I can switch from M31 to M42 at 02:00am, and keep shooting it until the sun comes up at about 06:30am... That is, if my batteries last.. hehe.

I will check out this drop-box thingy, and see about getting the next TIF file up there.

Sincerely, Alveprinsen.

[lensman57]

Hello.

I'll see about doing that within this weekend. I just came back from my observatory... I crashed my camera into my mount last night, and alignment was all messed up. Had to re-do everything...

Anyhow, camera is now running a 4h 30m shooting session of M31. 10 minute subs, iso 800. With some luck, the few clouds in the horizon will stay there... in the horizon... I had 9 hours of clear skies last night, 9pm - 6am.. I am hoping for something like that tonight as well. With some luck, I can switch from M31 to M42 at 02:00am, and keep shooting it until the sun comes up at about 06:30am... That is, if my batteries last.. hehe.

I will check out this drop-box thingy, and see about getting the next TIF file up there.

Sincerely, Alveprinsen.

Hi,

I am really sorry to hear about the mishap, I too am very nervous about ng the CCD attached to the scope unattended as once when I trto bring it into home position things almost got out of hand. You also need to watch the camera if you are imaging near the zenith and the meridian. If you wish to share your data it is best if it is in 32 bit fits format but tiff will do nicely too. Wish you success with your latest attempt.

Regards,

A.G

[MartinB]

Happy to have a play with your drop box data.  16bit fits will be fine for me.

[Alveprinsen]

Just got back home... Battery powering laptop and camera gave up after 3 hours, laptop continued guiding running on its own internal battery, camera stopped shooting. So, 1h 30 minutes lost.

Then the 7ah battery for the NEQ6 Pro mount gave up, effectively making M42 my new "park position"... Had to manually drive it back to its ACTUAL home position... And then alignment was off again. Had to delete all my alignment data, and re-align... Working on the last star of my alignment (star #13 or so), the battery I switched to (the one powering laptop and camera, which I assumed still had at least enough juice to power my mount for an alignment) started giving off too low current, and that little power-led on the mount started blinking frantically... Then the mount started making some incredibly ugly grinding noises, refusing any go-to commands, and finally just stopped after a few sec. Still had power though, but wouldn't function.

So once again, I switch to another battery (fresh one, 9ah) and start it up. My new park position is now some random star... so once again I drive it back manually... and realign AGAIN .... This time no ugly noises, no led-blinking... got a 16 star alignment.. Which is a pain, because my observatory is cramped, and I need to lie on my back on the sand floor, staring into the eyepiece, controlling the mount with my laptop in hand.

So yeah... 9 hours of the most perfect astronomical sky... You Brits wouldn't believe it... And I managed to utilize 3.................... I am so [removed word] right now..... Oh, and did I mention I've spent 8 hours in total to get 3 hours of footage..... I need my own fusion reactor to power this stuff...

Alveprins.

[Alveprinsen]

Link to the M31 ISO 800 4h 10 minutes exposure: HERE.

Sincerely, Alveprinsen.

PS: Had no idea this drop-box thing was so easy... cool...

[Alveprinsen]

Oh, and here are some 2 minute ISO 800 exposures as well.

[lensman57]

Link to the M31 ISO 800 4h 10 minutes exposure: HERE.

Sincerely, Alveprinsen.

PS: Had no idea this drop-box thing was so easy... cool...

Hi,

Here is the image of the trouble maker as promised. Processed entirely in StarTools and converted to PNG and resized in PS. Hope you like it. I only used your original stack BTW.

Regards,

A.G

[Alveprinsen]

Hi,

Here is the image of the trouble maker as promised. Processed entirely in StarTools and converted to PNG and resized in PS. Hope you like it. I only used your original stack BTW.

Regards,

A.G

Looks good.

However, it doesn't feel quite "real", if you know what I mean. Its like, the image is very rough. You have brought out the overall details much better than I have yet been able to do, however I feel my images are a bit more "smooth". Perhaps its just because the image is re-sized to such a small one.

I managed to gather 3 hours of data last night, and I combined a total of 440 minutes of footage into this image. Note that I have been very careful with the stretching to avoid noise. There is much more data to be had, I just haven't chanced it.

I haven't tweaked the core as small as you have, however I can see how that would be quite attractive. This is my core however. Less noise, more smooth and it "feels" more real:

And a closeup of the satellite to compare with the previous satellite image. Less noise in this one after an additional 3 hours of exposure.

My stars are super-green though... I wonder how that happened.

Oh, and if the stars in the corners seem weird, its because I've used the clone stamp to fill in some voids. I've rotated the image, and frankly, M31 takes up my whole chip... so there were some dark ugly corners...

Sincerely, Alveprinsen.

[glowingturnip]

Hope you don't mind, but I had a go with your data too - couldn't really use the first one, but this is from the 2 min stacks.   Got rid of the core burnout, and the colour noise seems to have gone, but I'm not 100% happy with the colour balance.  Mind you, the star cloud (NGC 206) in the spiral arms on the right pops out quite nicely.

Cheers,

Stuart

[Alveprinsen]

Hope you don't mind, but I had a go with your data too - couldn't really use the first one, but this is from the 2 min stacks.   Got rid of the core burnout, and the colour noise seems to have gone, but I'm not 100% happy with the colour balance.  Mind you, the star cloud (NGC 206) in the spiral arms on the right pops out quite nicely.

Cheers,

Stuart

Not at all.

That is nice also, and yeah - the color is a bit off.

You couldn't use the 10 minute exposure file? the 2 min is very short exposure... You pretty much brought out all you cold from it....

A question though... I see your picture, as well as the picture done by Lensman57 - are pretty "grainy" if you know what I mean. As my pictures appear more smooth, the pictures the two of you have provided seem quite grainy. Is this because it is uploaded and somehow looks different on this forum from how it would look if you simply opened the original file?

I see this trend in many pictures I see on this forum. Very grainy images, almost as if someone had turned "clarity" and "sharpen" in Adobe Lightroom to 100%....

Sincerely, Alveprinsen.

[lensman57]

 " see this trend in many pictures I see on this forum. Very grainy images, almost as if someone had turned "clarity" and "sharpen" in Adobe Lightroom to 100%...."

Hi Alve,

Thanks for your comments regarding the reprocess of your data. The space contrary to popular belief is not smooth it is full of cosmic dust and gas, it has grain. In processing and this is my personal opinion, I always back off a bit from using too high a noise reduction if not absolutely necessary, for me at least it is much preferable to have just a hint of noise rather than an iron out effect. The grain just makes the image more "realistic". The second point is that space is not jet black, even if it appears empty, it has a level of background luminance.  LR, and PS to a lesser degree, have never been designed to develope or  process DSO images, they were and are for daytime photography so a lot of the tools are designed to take care of a different type of artifacts than the ones we deal  with. A DSLR is capable of recording minute detail that could be brought out of the data should the imager decides to do so, that is what I did with your original data ( it took less than 10 minutes BtW) just  to show what was really hidden there . When you decide to reprocess and combine exposures you may wish to consider these points. You  may also wish to retain some star colour during the process, without this the image may look sort of over processed. Galaxies emit a full spectrum of light and a balanced image will have a range of the important emission lines such as Ha, Hb and oiii present. You have some very nice data there for a DSLR and as you re process the data the more detail and colour will emerge. If I may just give one piece of advice is that  I think you have reached the point of diminishing returns with longer and longer exposures, there is a point beyond which no more derail could be added to the data and it is best to spend the time on a different target. Wish you the best of success with your imaging.

Regards,

A.G

[Alveprinsen]

 " see this trend in many pictures I see on this forum. Very grainy images, almost as if someone had turned "clarity" and "sharpen" in Adobe Lightroom to 100%...."

...space is not jet black, even if it appears empty, it has a level of background luminance.  LR, and PS to a lesser degree, have never been designed to develope or  process DSO images...

...that is what I did with your original data ( it took less than 10 minutes BtW) just  to show what was really hidden there ...

...Galaxies emit a full spectrum of light and a balanced image will have a range of the important emission lines such as Ha, Hb and oiii present. You have some very nice data there for a DSLR and as you re process the data the more detail and colour will emerge....

...you have reached the point of diminishing returns with longer and longer exposures, there is a point beyond which no more derail could be added to the data and it is best to spend the time on a different target...

Some very enlightening points there, thank you very much.

I see you've used Star Tools. I guess I will have to check that out. When I tweak and stuff in both Adobe Lightroom as well as Photoshop, I spend quite allot of time actually... I guess I should try some more astro specific software.

Actually, after last night I kinda got tired of M31... I've moved back to M42 now. 6h10 exposure so far, and camera is down at the observatory right now working on another 4 hours for me. Will go down there at 6am, do flatfields, and start doing darks... With some luck, I might be able to get 4 more hours tomorrow as well, but... that is up to the weather...

At least now I know why grainy images are more of the real deal than my own ironed out ones. Its just that I've always preferred to smooth things out...

[lensman57]

Some very enlightening points there, thank you very much.

I see you've used Star Tools. I guess I will have to check that out. When I tweak and stuff in both Adobe Lightroom as well as Photoshop, I spend quite allot of time actually... I guess I should try some more astro specific software.

Actually, after last night I kinda got tired of M31... I've moved back to M42 now. 6h10 exposure so far, and camera is down at the observatory right now working on another 4 hours for me. Will go down there at 6am, do flatfields, and start doing darks... With some luck, I might be able to get 4 more hours tomorrow as well, but... that is up to the weather...

At least now I know why grainy images are more of the real deal than my own ironed out ones. Its just that I've always preferred to smooth things out...

Good Morning Alve,

It was a pleasure working with your data. Yes I use StarTool,  I am now just starting to get the hang of it after nearly 4 months of use, it is just me being slow to pick up. You may also consider PixinSight, a rather pricy proposition but a very capable and complete package with tons of specific tools. I tried the trial version of it( fully functional for 45 days on a special license ) but I just didn't have enough time to put it through its paces. M42 is a glorious nebula, you can never spend too much time on it and while you are there use some of your imaging time to go for the Horse_head and Flame nebula nearby, that is my next target if the weather permits.  Looking  forward to seeing the results of your endeavours soon.

Regards,

A.G

[glowingturnip]

hmm, on reflection, looking at that again this morning, the colour balance is definitely off isn't it - looks like a torch in a puddle.  I was thinking about that after having played with it last night, will probably be instructive for me to see at which step I'd introduced the colour cast in the first place, so will probably have another play. 

Grain does seem to be the curse of the DSLR doesn't it, I always seem to run into a wall eventually where any attempts to bring out any more detail bring out more grain, then a vicious circle of smoothing and sharpening and levels and...

My own take on the grain is that it's coming down to the individual photon level - some of the pixel censors in the camera receive a few photons, and depending on the ISO, will produce a certain output level, other pixels don't receive as many, and the grain develops.  Only way round that really is to average it out, ie as many exposures as possible.  Not being a cooled censor, and so having dark-current artifacts, won't help either 

[ollypenrice]

Here's my go.

Pixinsight DBE and SCNR green to calibrate the colour and de-colour noise the background then zap the rogue greens.

Photoshop, stretching and noise reduction (of the background sky only.)

Contrasts enhanced in layers.

Soft Light and Lab colour mode colour enhancement in Layers (without touching Saturation at all.) Getting colour out of it was like getting blood out of a stone but it's in there. A bit short on blue sensitivity, though.

A bit of sharpening on strong signal.

Star reduction and colour enhancement.

FInal custom noise reduction using Colour Select with the median filter.

I only used the long data so didn't try too hard with the core but I made a separate stretch for it, hammered the contrast, blurred out the hard edge of saturation and layered it in.

Good fun! Since you posted the link I'm assuming you're OK for me to have a play?

Olly

[dph1nm]

One should remember that ALL images have graininess due to shot noise i.e. the random variations in the number of photons emited by the source (good old quantum mechanics in there somewhere!). This is proportional to the square root of the total number of photons you collect - you cannot change this except by collecting more photons (bigger telescope, longer exposure). So if an image is perfectly smooth it is a 'faked' in some way!

NigelM

[Alveprinsen]

Here's my go.

Dude... That cannot possibly have been made using my data, because my pics sure don't look like that. Absolutely spectacular.

Btw, does everyone lower the resolution of their images to pre 1990'ties resolution, or is that done automatically when uploading a picture? I link mine from an off-site FTP, so I wouldn't know...

That being said, I guess I need to learn some other software than Lightroom and Photoshop. Which brings me to this thread I just started... Which software is THE software for astro-image post processing?

Sincerely, Alveprinsen.

[lensman57]

Dude... That cannot possibly have been made using my data, because my pics sure don't look like that. Absolutely spectacular.

Btw, does everyone lower the resolution of their images to pre 1990'ties resolution, or is that done automatically when uploading a picture? I link mine from an off-site FTP, so I wouldn't know...

That being said, I guess I need to learn some other software than Lightroom and Photoshop. Which brings me to this thread I just started... Which software is THE software for astro-image post processing?

Sincerely, Alveprinsen.

Hi Alve,

Capturing the data is the easy part of the job, converting it to something like Olly's is where the fun starts. The road is long and painful but it will happen. I have  just started like yourself so I can tell you that there is no single software that does everything, there is no magic wand. Having a huge data file from a DSLR is good but unless you have the latest Mac Pro with multiple 8 core processor and 32 Gig of ram, I'd advise you to Bin the files down for processing, you can always keep the original gigantic stack if you wish.

A.G