Could use some help diagnosing a few issues

[JP50515]

Hey guys,

 

I have some imaging woes going on. This is about 4 hours on the whale. Was hoping to go after it again tonight, but these issues have me questioning things. 

 

https://www.dropbox....TsS8ZGSO3a?dl=0

 

Canon450D

Iso800

Subs - 240s exposure time (shooting for 1/3 Histo)

Darks - 240s 

Flats - 1s (shooting for middle of histo) using tshirt+torch method

Bias - .0025s

 

I am using FastStone Image Viewer to check the Histogram on each picture. 

 

There are a number of different issues going on here.

 

I see:

 

• really bad horizontal banding going on if I do a quick ABE and autostretch in PI using the default settings. (see screen capture below)
• Fixed Noise Pattern
• Artifacts that will not go away...have been there for months now and I've literally cleaned everything numerous times. It is not dust...beginning to think it may be sensor damage of some type. 
• Oblong Stars

1. I know the canons sometimes have banding, but I've never run into it like this before..and suddenly it's a common occurance. I'm inclined to assume that this has to do with the dark and bias frames. Which when viewing individually seem bad to me. I take them with the scope cap on, and a black garbage bag over the entire rig...in the dark. I'm not quite sure if they're bad or not, but if so maybe someone can suggest an alternate technique? Can I pull the camera and throw the sensor cap on and try that, or does it need to remain on the imaging train like a flat would?
2. I believe the solution to FNP on a DSLR is to dither more aggressively...I already have my dither set to high..perhaps I just try maxing out the aggressiveness in SGP and see what happens here?
3. These artifacts are the bane of my imaging rig right now. I can't figure out where they are coming from at this point (I've cleaned everything many times) but you can see them in the flats clearly, and the final stacked image. 
4. This is possibly a collimation issue. I moved the scope cross country and have yet to re-collimate. The stars seem to be moving in the same direction in each sub. (perhaps this indicates something else?)  I used sharpcap's PA process and had that sucker right on top of the target so I'm inclined to rule out a bad Polar Alignment. I'll be re-collimating today, and star testing it tonight to see if that helps at all. The mount will not be moving between last night's imaging and any imaging/diagnostics I do tonight. 

 

 

 

Anyway, I included the Masters, and individual images for the stack in the link above if anyone wants to try and help me figure out what's going on. 

 

Thanks!

[wimvb]

The canon banding reduction script should clean this up rather nicely.

For bias and darks, you don't need the optics. I put my camera (dslr) in a black lined box for darks. With lens cap on and aluminium foil, to block any light. Lens caps made of plastic let (ir) light through, so don't rely on just them.

I used the synscan routine for pa, until it showed me perfect alignment. Nowadays I still use it for initial alignment, but follow up with drift alignment, using PHD's routine. Star drift can have several causes, such as insufficient pa, unbalanced setup, differential flexure (guiding), periodic error (guiding or not guiding).

[JP50515]

Unfortunately...  it creates it's own set of issues when I run it under default settings. 

[JP50515]

EDIT: Nvm...figured it out  dude thanks for that. Had no idea that script was hiding in there!

 

Double EDIT: nope...after a stretch it's still there. Any suggestions on settings for that script?

[wimvb]

I don't have access to my image processing computer right now, but will download your image later this evening, and have a look. Default settings don't always work. I almost never use abe (dbe gives me more control), and generally use highlight protection in the cbr script.

[Oddsocks]

On 06/05/2017 at 18:14, JP50515 said:

There are a number of different issues going on here.

When Wim has time he will answer you with more expertise than me but FWIW here is my take on the problems.

First though a question, is your camera modified, have you removed the I.R. blocking filter and if so what if anything have you replaced it with?

The reason I ask is because looking at the sample flat I see a huge peak in the red.

You are using a flashlight for your flats light source, if this is using an incandescent (filament) bulb these emit much more I.R. than visible light, even worse, the ratio of I.R. to visible varies hugely depending on the state of the batteries, as battery power diminishes the proportion of I.R. increases while the visible decreases.

If your camera has been modified by removal of the IR blocker and not replaced with a wide band IR blocker then the camera will be overly sensitive to IR and this will produce a bad flat.

Opening your sample single sub flat in PS you can see the red histogram pushed far to the right, this flat is overexposed in the red, barely adequate in the green and ok in the blue:

 

After a little levels adjustment in PS you are left with a flat that is vastly overcompensating in the centre field in the red and doing virtually nothing in the green:

 

 

Compare this to the single sub light frame that you provided, here you see a more normal histogram with the RGB peaks closer together:

 

 

 

So the first problem with the flats is the colour balance and this is down to the light source used for the flats. Incandescent bulbed flashlights are ok for a mono camera that is fitted with an IR blocker, you can adjust the exposure time for each filter and the I.R. blocker will prevent most of the the I.R. reaching the sensor but incandescent bulbed flashlights are no good for OSC single shot or DSLR cameras and especially where the IR blocker has been removed to increase sensitivity in the far red and near I.R.

There is another problem with I.R. that under this wavelength black anodised surfaces that you find inside telescope focuser draw tubes and couplers become almost white and reflect I.R strongly so that what looks like normal vignetting is really reflected I.R. boosting the central part of the image. You don't say if this is with a telescope or a stock lens on the camera, if it is a telescope then internal reflected I.R. from the flashlight may be creating a false vignetting profile and distorting the flat.

Even with an I.R. blocker filter fitted some I.R. will still pass the filter, they are not 100% efficient so if your flats source produces a huge amount of I.R. this will still produce bad flats.

See this document for an in-depth explanation of this phenomenon : http://diffractionlimited.com/flat-fields-stray-light-amateur-telescopes/

Back to your flats, and the problem contaminants in the image.

In the image below you see two groups of contaminants, the smaller and darker objects are close to the sensor, within a few mm of the surface of the sensor, here circled in red. The larger and fainter objects, circled green, are further away, possibly on the rear surface of the last element of a stock lens if used, or a flattener in a telescope.

The exception being the large smudge in the centre, I suspect this is a fibre lying on a surface far from the sensor, but it also could be a dirty smudge on the sensor surface, it is not easy to say.

 

 

 

What is noticeable is that while the smaller and better defined objects can be seen in the single sub light frame below, the smudge can not be found though that may just be because it is lost in the background and only shows once more lights are stacked, don't be surprised if some of the objects I circled are difficult to see in the forum images, they are compressed and downsized, the objects can be seen in full resolution in PS.

 

 

 

If I take your single diagnostic CR2 flat, colour balance then convert to monochrome and blend it with your single light sub in subtraction mode you see that all the contaminants are removed and this is what should happen if the flats were functioning properly:

 

 

Having got his far and looking at you autosave TIFF the obvious thing that jumps out is that your flats have become muddled, some flats are the correct orientation up/down left/right and some are flipped so that besides overcorrecting they are overcorrecting in the wrong place, you can see that several objects are mirrored as well as the banding artefacts, again some of this will be difficult to see here in the forum but they can be found easily with the raw image in PS:

 

 

For this image set the main problems are that the flats are contaminated with I.R. and not all the flats have been stacked in the same orientation.

I would suggest forgetting the flashlight and tee shirt method for the flats. Depending how large the telescope is one of the LED tracing panels will provide you with an I.R. free light source, some of them work on a 12v battery, the USB port of a computer, or a 12v to 5V automobile USB power adapter. They need to be used on full brightness setting to avoid the PWM banding problem caused by the brightness control and are dimmed down using several sheets of plain printer paper or grey perspex.

e.g: https://www.amazon.com/Huion-L4S-LED-Light-Box/dp/B00J3NRAV2/ref=sr_1_2?ie=UTF8&qid=1494340208&sr=8-2&keywords=huion+light+pad

If the telescope aperture is modest then an iPad or laptop set to white screen works well, again a paper or grey perspex attenuator is needed, in this case partly to avoid PWM banding but also to depolarise the light which is a characteristic of TFT screens.

Otherwise, just sky flats taken about half and hour to an hour before sunrise, or after sunset, with the camera/scope pointed to an area of sky directly opposite the suns current location, this to avoid gradients that are seen when pointing close to the suns current position. A single sheet of printer paper over the telescope/lens will help avoid picking up the brighter stars that may just be visible and moving the telescope/lens a little between each shot will blur away any small gradients.

Don't bother with LED flashlights, they are too monochromatic and heavily biased to the blue end of the spectrum so you just end up with a poor quality blue flat instead of a poor quality red flat.

Once you have a good correlation between flats colour balance and intensity distribution with what is seen with the real image than any smudges and foreign objects will be removed properly in the final image.

For DSLR's the use of darks that match the lights is not necessary, in some ways they introduce more problems than they solve because it is impossible to match the sensor temperature for the flat to the temperature for the light. You will reduce the banding artefacts by creating an artificial dark purely for calibrating the flats and this is done by stacking forty to fifty or more zero time bias frames using plain averaging, not sigma reject, save the resulting stacked image with the name "dark for flats" and use this specifically for calibrating the flats.

For the rest of the lights, just calibrate with flats, use a sigma reject stacking algorithm and a bad pixel map. Together with a modest mount dither this will give you calibrated images with minimum banding.

When you take the bias frames, if possible put the camera sealed in a plastic bag in the fridge and use an intervalometer, remote, or laptop control to take the bias frames, leave the camera at least three minutes between frames to minimise sensor heating which makes banding worse.

With some of your images being autosave.tiff are you using Deep Sky Stacker? From your later comments you also have PixInsight? in which case use the batch preprocessing script in P.I. and use the one piece of software for the complete process rather than mix-and-match, your results should improve, even if not, it is easier to pin down problems in a single piece of software rather than use several and not know where it is going wrong.

Finally, check in the camera setup menu that auto portrait / landscape detection is turned off, although the stacking software should not take any notice of this I keep seeing images from Canon cameras that have calibration artefacts due to the camera changing the orientation of the images during an imaging session.

HTH.

 

[wimvb]

@Oddsocks has already given a very comprehensive answer to your question. I can only confirm most of his findings.

Here's your autosave, after background extraction and canon banding reduction in PixInsight (screen stretch applied)

As you can see, the banding cleans up quite nicely, but the dust is much harder  impossible to get rid off in processing. In this case, I also wonder if the flats match the light frames. The way I take flats is by putting a white cloth over the scope (in parked positsion) and light an LED flashlight from a considerable distance on that. My flats correct dust bunnies, and most of the vignetting. The rest can be removed during processing.

The symmetric placement of the dustbunnies makes sense if your camera was rotated during the imaging session (such as in a meridian flip, but that would have to be perfectly executed). In this case, it may also be coincidence.

I would suggest that you have a look at the sensor of your camera. Dust bunnies are never further from the sensor than a few mm, so what you see here is either on the sensor or close. A light pollution clip filter is in this range.

The "fixed noise pattern" is walking noise, caused by a constant displacement between subs (which you already noted yourself). It isn't severe and can be greatly removed by noise reduction, but it shouldn't be visible at all if you dither. If you have backlash in your mount (even just DEC), your dithering may not work. One way to find out is to make a movie of your subs and examine that:

http://wimvberlo.blogspot.se/2016/07/the-effect-of-dithering_21.html

Dithering should be 15 pixels per step in both RA and DEC, on the imaging camera. During dithering, the guiding must be off, otherwise the guiding software will compensate for the dithering.

Walking noise can occur even if you use guiding, if there is movement between the guiding camera and the imaging camera during a session. Causes are differential flexure (mirror movement in a reflector) or the focuser flexing (heavy dslr with flimsy focuser), to name just two.

Hope this helps