Processing is hard lol

[nickp87]

as per the title 🤣

I'm finding this processing lark rather frustrating. 

I don't know if it's because I am staring with garbage data, or I am just rubbish at processing it, but i can't seem to get a satisfying image.

Bare in mind i am extremely new to this and not working with the best equipment (az gti, canon 550D and 300mm f5.6 lens), target is M31

Anyway, i have a attached my fit file from siril and would greatly appreciate if anyone could have a go at a stretch and see what they can come up with or if there any obvious flaws in the data, i will post my result shortly.

Many thanks

Nick

andromeda forum.fit

[Fegato]

Haven't got time to do anything right now, but might take a look tomorrow. First impression is that it's quite noisy, and you have a fair amount of "banding" type noise, which I think is common with DSLR / Canon cameras (although I'm no expert on that) - you can research that or someone will pipe up with a comment about how best to manage it.

Obvious first questions are:

1. How much integration time went in to this? 

2. What LP conditions were you shooting in, and were you using a filter of any sort?

2. Did you calibrate the subs (with darks, bias, flats...)?

[vlaiv]

Data is nice, but SNR is rather poor - total integration time needs to be much longer.

[nickp87]

53 minutes ago, Fegato said:

Haven't got time to do anything right now, but might take a look tomorrow. First impression is that it's quite noisy, and you have a fair amount of "banding" type noise, which I think is common with DSLR / Canon cameras (although I'm no expert on that) - you can research that or someone will pipe up with a comment about how best to manage it.

Obvious first questions are:

1. How much integration time went in to this? 

2. What LP conditions were you shooting in, and were you using a filter of any sort?

2. Did you calibrate the subs (with darks, bias, flats...)?

Thank you for the reply

This is I think 20 minutes of integration 

Bortle 4, but in a urban area with close lighting from houses etc, no filters used

Darks, flats and bias frames used.

39 minutes ago, vlaiv said:

Data is nice, but SNR is rather poor - total integration time needs to be much longer.

Thank you Vlaiv for taking the time, this is similar to what I am getting. I plan on adding more integration, hopefully some clear weather this evening.

The reason for this image time is is that i was following a video on Youtube for DSLR astro, they did 1000 1 second exposures (no tracking) and I wanted to see what i could do with the same kind of integration, this is there end result (i know a lot will depend on there location/skies etc.

[vlaiv]

2 minutes ago, nickp87 said:

The reason for this image time is is that i was following a video on Youtube for DSLR astro, they did 1000 1 second exposures (no tracking) and I wanted to see what i could do with the same kind of integration, this is there end result (i know a lot will depend on there location/skies etc.

A lot will depend on camera model used and lens / telescope used as well.

 

[Gerr]

This is what I've managed with your data. I used photoshop CS and applied levels and curves stretches. This revealed a very noisy image with colour banding and blotching. I used astro flat pro to try and remove the dark noise from this and then from Carboni's tools I used colour blotch and noise reduction.

I also used Camera raw filter to enhance clarity and do some further noise reduction. I used selective mask on andromeda to improve vibrancy and dodge and burn tool to improve contrast. This was just a quick edit.

Not bad considering you used a relatively slow f/5.6 zoom lens at 300mm. What was your ISO setting?

My edit:

[nickp87]

1 hour ago, vlaiv said:

A lot will depend on camera model used and lens / telescope used as well.

 

Very true, I think the image from the video was captured with a pricey canon f2.8 lens

33 minutes ago, Gerr said:

This is what I've managed with your data. I used photoshop CS and applied levels and curves stretches. This revealed a very noisy image with colour banding and blotching. I used astro flat pro to try and remove the dark noise from this and then from Carboni's tools I used colour blotch and noise reduction.

I also used Camera raw filter to enhance clarity and do some further noise reduction. I used selective mask on andromeda to improve vibrancy and dodge and burn tool to improve contrast. This was just a quick edit.

Not bad considering you used a relatively slow f/5.6 zoom lens at 300mm. What was your ISO setting?

My edit:

Many thanks Gerr.

Images where shot at iso 1600, I usually shoot at 800 but was trying to some new settings.

Is the noise a symptom of low integration time, iso or something else?

I'll have a look into colour banding and blotching and find causes etc. 

Again many thanks all, it really helps and makes me feel a little less stupid :). 

[Gerr]

53 minutes ago, nickp87 said:

Very true, I think the image from the video was captured with a pricey canon f2.8 lens

Many thanks Gerr.

Images where shot at iso 1600, I usually shoot at 800 but was trying to some new settings.

Is the noise a symptom of low integration time, iso or something else?

I'll have a look into colour banding and blotching and find causes etc. 

Again many thanks all, it really helps and makes me feel a little less stupid :). 

Yes, high ISO (>3200) means high noise. Battle with longer exposure time before star trails and take many many images!! High ambient temperature adds to sensor noise too!!

I like using ISO 800-1600 on a Canon 6D. Aperture between f2.8 and f4. Aim for at least an hour of integration time.

[Elp]

I had a look at the data, theres hints of the trademark "dark rain" noise pattern similar to what I had with my Canon 600d, dithering is required to remove this (if you're not autoguiding you have to physically move the camera very slightly, we're talking pixels here, say after five or ten minutes of imaging and repeat in the same intervals). The general blotchiness and noise is due to short total time. Due to the colour noise, background extraction (removing LP gradient) also struggled a little, the temperature of the camera can contribute here, little you can do, maybe turning off the LCD screen whilst it's imaging may help a little keeping the temperature down.

Imaging at F5.6 isn't bad, my Z61 native is F5.9 but 60mm aperture. I've used camera lenses at similar f ratios, you need more total imaging time.

Yes, processing is a learned skill. You can have the best equipment in the world, the data will look like nothing if you can't process it. You can have average data and get a presentable result with good processing skills. Your data wasn't bad at all, there just wasn't enough of it.

As it's a Canon, do not use bias frames in calibration, take darks for your flats (dark flats) same settings and duration as your flats and subtract the darkflat master from your flats before you make a flat master, or subtract a constant bias value when calibrating.

Edited December 10, 2023 by Elp

[ollypenrice]

You are absolutely right, processing is hard. In fact it has no ceiling: you can get better and better at it over a lifetime. Capture, on the other hand, is a mechanical process which you can, without an enormous amount of difficulty, get to be perfect within the constraints of your equipment.

My advice would be to watch tutorials or read books by people who know what they are talking about. Adam Block, Warren Keller, our own Steve Richards, Robert Gendler, R Jay GaBany. The net is full of U-tubing clowns who flounder around dragging sliders this way and that till they say they 'Get something they like.' The instant you hear that phrase, turn them off. 

As you are learning processing, make it a rule to understand what you are doing. Clicking and thinking are two different activities!

Olly

[tomato]

It’s interesting that there are lots of resources available where you can pay someone to provide you with good data, but unless I’m missing something, paying someone to process your data to a quality final image appear to be much less popular.

[Elp]

I think you'll find many people offer these services, but the question is what's in it for the purchaser?

[nickp87]

18 hours ago, tomato said:

It’s interesting that there are lots of resources available where you can pay someone to provide you with good data, but unless I’m missing something, paying someone to process your data to a quality final image appear to be much less popular.

 

18 hours ago, Elp said:

I think you'll find many people offer these services, but the question is what's in it for the purchaser?

Yeah seems a little pointless to me, but I feel the same about remote imaging, the fun and enjoyment for me is physically doing things and mucking it up

I astro modded my camera last night, and looks like there could be a few hours of clear skies this evening so going to try and some more data, and use some of what I've learnt here.

@Elp, not dithering or guiding yet, have all the necessary stuff ready but as I am using mostly in alt az mode I haven't setup it up yet, is dithering useful in alt az? 

Thanks all for the help, really great forum. 

[Elp]

5 minutes ago, nickp87 said:

the fun and enjoyment for me is physically doing things

For sure, it's why people generally take photos in the first place as a memento of something they've seen or where they've been, remote is a bit soulless to me even though you have access to other targets, equipment and more imaging time, it's a valid resource if you don't have the means to build and use a rig and keep it going.

As far as I know, auto dithering only works in equatorial mode, never seen it in alt az.

[tomato]

19 hours ago, Elp said:

I think you'll find many people offer these services, but the question is what's in it for the purchaser?

I see several folks prepared to have a go here on SGL for no  payment, but who offers a service for a fee, people like Adam Block, Warren Keller? How do they charge, by the hour or a fixed fee?
What’s in it for the purchaser is a finished image processed by acknowledged experts using all of their experience and the latest software with as near as you can get to a guarantee that there is no better image possible from their data.

If I had 25+ hrs of first class data on a hard to process target, I might be tempted.

[Elp]

If you look at many freelance platforms there are numerous people offering AP processing services, pretty much anything can be subcontracted nowadays, even subcontractors sub subcontract elements of their project work.

But processing is subjective, it's impossible to have "the best" result from provided data, case in point they could produce something which you feel is insufficient to what you personally were expecting it to look like, then what happens? It's much like any commissioned work. The gratification for me (and I suspect a lot of us) is doing it ourselves and the effort we put into it, otherwise what's the point of doing it in the first place, it'd be much easier to just look at any space agency image or astrobin which already exists, if you wanted to you could simply copy an image and make adjustments to it to your liking and be satisfied (though it's borderline plagiarism). That's my view anyway.

My own images arent the best, but the results I end up with satisfy me at the time and I know it is of my own creation (even though the rig does most of the acquisition work), and that provides the gratification for all the time and effort put into each image. It's also one of the reasons I don't advocate the use of scripts as you're not learning anything.

No one gets good at anything by not putting the effort in and learning, even if it's one step at a time.

Edited December 12, 2023 by Elp

[tomato]

Not sure a final image is always that subjective, NB nebula images with reduced stars and “false colour” palettes perhaps being the exception. It’s not often on SGL that a well respected imager posts an image and gets numerous replies saying this or that should be tweaked, most just heap praise, quite justifiably on the OP. Maybe folks don’t like to make critical comments as, like me, they don’t feel sufficiently competent to do so.

As for always having a go yourself, I support that approach, because as you point out, this is how most people learn and improve a skill. However, it is sometimes good to know your limitations. Some years ago my brother embarked on a project to make a long case clock. He had the tools and skill set to make the movement but he knew his cabinet making skills were not to the same high standard so he farmed this part of the project out with the result that the finished article was not compromised, the right decision in my view.

[paul mc c]

"We choose to process......not because it is easy...we choose to process because it is hard".

Swear I heard something along those lines said before.🤔🤔😁.

Edited December 12, 2023 by paul mc c

[Elp]

I don't think I'd enjoy it so much if it were easy. I think a lot of us just enjoy tinkering to whittle away the boredom.

[Clarkey]

On 12/12/2023 at 18:30, paul mc c said:

"We choose to process......not because it is easy...we choose to process because it is hard"

Showing your age🤣

[paul mc c]

53 minutes ago, Clarkey said:

Showing your age🤣

My granda told me about it..🤥🤥