Why horizontal lines in image?

[Seanelly]

How frustrating!

3 sessions and 11 hours total of data on M81 and M82 and I can't get rid of these horizontal lines that litter the full image. I've cropped M82 to give you a better look.

Both examples shown here are 11hours of 5m exposures at 1600i with modded Canon T6i and went untouched from DSS with 50 each of dark, flat, and bias to Adobe PS. The d,f, and bias frames all came from the first 3.5 hour imaging session, at about -15C, and the other two imaging only sessions were around -10C.

The first (1) image shown has a careful levels adjustment only in AdobePS, and the second (2) image has an additional very light curves adjustment.

After the first 3.5h imaging session was processed I thought the lines would go away as additional data was added, but that is not the case. Something tells me the problem is from the dark, flat or bias frames, but I was very careful in collecting them and I've looked at each one individually along with every light frame and all data looks good. The only thing that I can think of is that perhaps the flat frames were not bright enough?

I've left both images as tif files to download so that you can see them exactly as they are.

Further info, if possible, available upon request. Ideas, suggestions, or a solution would be greatly appreciated, as I have no idea what in Heaven's name (or Earth's) is happening here.

 

(1)

DSS 11h 2m 5ex 1600i level adj only.tif

(2)

DSS 11h 2m 5ex 1600i level adj and slight curve.tif

Edited March 9, 2019 by Seanelly
additional info

[carastro]

It looks like a grid pattern to me ? wrong bayer matrix.  Did you select the correct camera in DSS. It's in the settings Options Raw/Fits box near the bottom.

Carole 

 

[Rush]

Suspect its comming from the Cali Frames. try only with darks and see if its the same. Looks as if those flats too are not matching.

Cs

Rush

[Seanelly]

4 minutes ago, carastro said:

It looks like a grid pattern to me ? wrong bayer matrix.  Did you select the correct camera in DSS. It's in the settings Options Raw/Fits box near the bottom.

Carole 

 

Thanks for responding, Carol. You're on to something here, as I'm checking the settings now and the box was not checked for a DSLR (just before you answered I edited my post to add the modded Canon T6i 750D I'm using).

There's no hiding my ignorance in all this!

But while there are three selections for the Digital Rebel, I don't see a selection that exactly matches my DSLR, a Canon EOS REBEL T6i (750D). Is close enough okay, or do you know which of the three I should choose? (I'm thinking you have access to DSS).

[Seanelly]

18 minutes ago, Rush said:

Suspect its comming from the Cali Frames. try only with darks and see if its the same. Looks as if those flats too are not matching.

Cs

Rush

Thanks for responding. I will try as you suggest if Carol's discovery doesn't pan out (the DSLR box was not checked in DSS). I'll restack with that done and see what I get. (Man, every time I think I've got everything under control, something else pops up!)

[Rush]

49 minutes ago, Seanelly said:

Thanks for responding. I will try as you suggest if Carol's discovery doesn't pan out (the DSLR box was not checked in DSS). I'll restack with that done and see what I get. (Man, every time I think I've got everything under control, something else pops up!)

No worries, We r all here to help?.

(Man, every time I think I've got everything under control, something else pops up!) Same old story

happens at regular intervals. Usb hubs and usb ports start playing game too.?

Lum looks like this.

[carastro]

Quote

I don't see a selection that exactly matches my DSLR, a Canon EOS REBEL T6i (750D). Is close enough okay,

I am pretty sure I read somewhere that to long as you tick a Canon camera of some sort it will be OK.  I never had my camera listed when I used to use one and DSS and just ticked the closest match and was always OK.

Carole 

[Rush]

Core and continum looks like this.

[Anthonyexmouth]

i've been seeing the same thing. from what i can find out its the AF pixels on the sensor, only shows up when really stretching the data, i could be wrong, but thats the only explaination i found for my issue, count them, if its 7 or 8 pairs of lines thats what it is. cant remember if its 7 or 8 off the top of my head

[Seanelly]

1 hour ago, Anthonyexmouth said:

i've been seeing the same thing. from what i can find out its the AF pixels on the sensor, only shows up when really stretching the data, i could be wrong, but thats the only explaination i found for my issue, count them, if its 7 or 8 pairs of lines thats what it is. cant remember if its 7 or 8 off the top of my head

Hi, thanks for the input. The latter half of what you've stated is bang on, as I'm counting 7 bands of 5 or 6 tightly grouped lines in each, but I'm baffled by the stretching you mention as a possibility, as the bands are evident even without a curves stretch, as shown in the first image I posted, which only has a level adjustment, while the second image does have a slight stretch, but very slight, and could not be defined as 'really stretching' in any way.

I thought Carol's reply might be the ticket (I did not select my DSLR camera in DSS), but as it turns out, while the right DSLR selection has seen the bands faded to a degree after restacking, it is still evident with a slight curve adjustment.

I'm going to try several restackings again if necessary while leaving out of each either the darks, flats, or biases, as Rush has suggested, to see what effect that has, as I'm not 100% confident in my flats being bright enough, at least, but your pinning the tail right on the donkeys ass with that info about the sensor has me wondering if I made the wrong choice in going with a modded DSLR over a CCD camera. The Canon T6i (750D) has the upgraded DIGIC 6 processor, one of the reasons I was convinced to go with it, but if this is the best it can do, I'm surprised that anyone is using it for astroimaging. There surely must be a way around it in Adobe, but I've only had the program for a week and don't know it very well. I'm a rookie with pretty much everything I touch in this pastime!

[Anthonyexmouth]

i've been told there are PS tools to get rid of it. 

the banding should be 7 pairs of lines. its the slightly less sensitive auto focus pixels apparently. i've got the 760d

[Anthonyexmouth]

this is the pattern in mine

[Seanelly]

2 hours ago, carastro said:

I am pretty sure I read somewhere that to long as you tick a Canon camera of some sort it will be OK.  I never had my camera listed when I used to use one and DSS and just ticked the closest match and was always OK.

Carole 

Hi again, Carol. Well, a restacking with a Rebel DSLR selection in DSS has improved the image, but not completely eliminated the banding, as even with a mild curve stretch they start making their ominous appearance, and now that I know they are there, it's all I can see. I'm reminded of the time my father took me with him to get new tires for the family car (this was 50 years ago when just about every car had whitewall tires) and all the way home I could look at nothing but the whitewall tires on every other car around us!

Seriously, Anthonyexmouth above has delivered an unsettling bit of proof that it could be the AF pixels on the DSLR sensor, which if true leaves me wondering why anyone would use a DSLR for astroimaging. But the fact that they do and do it well makes me wonder even more what the heck I've done wrong here. Maybe when I add more data, and understand Adobe PS better, I can hide these artifacts. 

I've never been 100% sure of my flats, as it seems such an arbitrary process, with many stated ways of achieving results, but then this may just be an indication that the parameters are wide, and so I will restack without them and see what gives, also the darks and biases if the bands persist. Also I will try another Canon selection in DSS, as there are three for the Rebel. Thanks for hanging in there with me.

Edited March 9, 2019 by Seanelly

[Seanelly]

11 minutes ago, Anthonyexmouth said:

this is the pattern in mine

Definitely the symptoms I'm seeing if I really stretch out the image. Perhaps someone with a good grasp of PS will chime in, but in the meantime I'll start asking questions on their forum and reading up on the options. Thanks for the help.

[Anthonyexmouth]

Noels tools for photoshop apparently work well but not tried it myself. 

[Seanelly]

26 minutes ago, Anthonyexmouth said:

i've got the 760d

By the way, do you know what the difference is between the 750D and 760D? I can't find any variation in the specs.

[Anthonyexmouth]

not a lot, 760 has an lcd screen on top and maybe a couple of other little tweaks but the same camera 

[Seanelly]

48 minutes ago, Anthonyexmouth said:

not a lot, 760 has an lcd screen on top and maybe a couple of other little tweaks but the same camera 

My banding problem is now reduced to a workable state. It takes two good curve stretches to start hinting at them. In DSS I de-selected the 'EOS REBEL XTi' (the closest the list comes to my camera) and selected instead the 'EOS REBEL' (there is also an EOS REBEL XT, but I did not try it). If you are using DSS do you know what camera is selected? I'd like to think I helped someone for a change.

[Seanelly]

3 hours ago, carastro said:

I am pretty sure I read somewhere that to long as you tick a Canon camera of some sort it will be OK. 

On a whim I de-selected the 'CANON EOS REBEL XTi' (seemed closest to my REBEL T6i) in DSS and selected instead the 'CANON REBEL' and my problem is now manageable. It takes a couple of strong curve changes with the usual levels adj to start noticing the bands, and I can work around this. I do so love SGL and all the great members willing to spend time on others' problems. Thanks, and the way things are going, you'll probably read me soon enough on another technicality!

[carastro]

I am glad things are better, but is "manageable" good enough?  Perhaps you could upload a stacked image (unprocessed) and let us have a "go" at it to see whether it is in fact "good enough".  Fingers crossed it will be. 

Quote

I do so love SGL and all the great members willing to spend time on others' problems. 

We have all been there ourselves so we know what it is like, and benefited from other people's experience. 

Carole 

 

Edited March 9, 2019 by carastro

[michael8554]

The banding problem due to AF sensors on the imaging sensor robbing light is getting worse with every generation of camera. 

There only used to be a handful of AF sensors, now there are maybe a hundred arranged in bands, I believe to improve focus when shooting HD video.

Not usually noticeable on daytime shots, but dim astro images with high gamma stretches brings it out nicely ?

Started to get noticeable with the 600D.

Michael 

[Seanelly]

14 hours ago, carastro said:

Perhaps you could upload a stacked image (unprocessed) and let us have a "go" at it to see whether it is in fact "good enough".  Fingers crossed it will be. 

 

Here it is, Carol. About 11 hours of 5m raw exposures at 1600i unprocessed from DSS with 40d, 48f, and 75bias frames (all in master frame format), taken with a modded Canon T6i. I'm looking forward to seeing what you can do with it.

I have a question about the two calibration frames, darks and flats, that I'm hoping you can enlighten me on. I've read that they should be taken at the same iso and temp as the light frames, and I took care of that in the first session (there were a total of three sessions (actually four, but last night I could only get eight frames before clouds gathered) in the image provided), but what about added data in later sessions when the temperature is different? The first session, when the cali frames were produced, was in -15C, the next two sessions were in -10C, and last night's meager eight images were at -5C. Are these temperatures close enough to use the same darks and flats masters for all the data? If not, how do you get around adding more data to an image when they are taken at different temperatures?

Autosave004.tif

[alan4908]

20 hours ago, Seanelly said:

Further info, if possible, available upon request. Ideas, suggestions, or a solution would be greatly appreciated, as I have no idea what in Heaven's name (or Earth's) is happening here.

I've had to look at your image and in addition to the horizontal lines you have a collection of dust donuts.  To minimize these, I suggest take some flats, the most consistent and reliable method that I've found to do this is via an EL panel.  If you find flats/darks/bias a bit of mystery then I'd getting a copy of Steve Richards excellent book  "Making Every Photon Count"  - https://www.firstlightoptics.com/books/making-every-photon-count-steve-richards.html

You can also minimize both of these unwanted effects in PS.  The horizontal lines can be minimized by one of Noel Carboni tools called Horizontal Band noise reduction, the dust donuts via the context sensitive spot healing brush in PS. You can download Noel tools from https://www.prodigitalsoftware.com/Astronomy_Tools_For_Full_Version.html  and represent excellent value at $21.95.

Alan

[prusling]

I suffer from two distinct dark horizontal bands on my Canon and get rid of them after stacking by using the excellent free app Fitswork4 - using the function Processing, Background Flatten, Lines to Equal Values with standard parameters.

[michael8554]

Just what I used to do with the 600D !

Michael