Hot pixel problem

[planetdnb]

Hi Guys,

I am having an issue with hot pixels and can't seem to solve it. I've not experienced it before but this is a different camera (quite a bit older) with I guess lots more dust on the sensor.

I have 8 subs of 10 mins of the horse head. 24 darks (taken the following evening). 40 Flats (same time/evening as lights) and 25 bias frames.

I believe the flats/bias frames are working as they should. I combine the Bias frames then subtract them from the flats. Then combine the flats and subtract them from my lights.

I can see the images have lots of hot pixels in them. When I apply my darks the hot pixels seem to just turn from white to black.

Does my process sound incorrect?

[planetdnb]

ps. I tried dropping the bias frames altogether and subtracted my combined dark from my flats.

when I applied the flats its as if they hadn't done anything. Amp glow was back along with vignetting.

[roundycat]

not sure what the connection is between hot pixels and dust on the sensor. Dust is easily dealt with anyway and flats should take it out.

If you subtract your master dark from each light frame it should take care of the hot pixels but only if the temperature of the lights and darks match. The pixels going from white to black is a sure indication that you have a temperature mis-match. Basically your darks are too hot and are over correcting.

You should subtract bias from the flats if you do not use flat darks. The master flat is not subtracted from the lights, it is a double division operation.

What you are doing sounds about right but I never use more than ten flats.

You cannot subtract your master dark from the flats as they are almost certainly a different exposure time.

Dennis

[FraserClarke]

I'd agree with Dennis that it is probably a temperature mismatch if you're getting lots. You do occasionally get dodgy pixels on CCDs which float around and are impossible to correct -- but it's only usually a few individual pixels on a chip. In this case, you just have to mask them out and tell the combining software to ignore those pixels. I have no idea how you do that in DSS though I'm afraid

Also, make sure you divide the lights by the flats, not subtract.

[themos]

From the DSS manual :

Automatic Detection and Removal of hot pixels

The goal of the automatic detection and removal of hot pixels is to replace hot pixels with a value computed from neighbor pixels.

First the very hot pixels are identified by an analysis of the dark frames (or the master dark frame if available). Every pixel which value is greater than [median] + 16 x [standard deviation] (sigma) is marked as a hot pixel.

For all those pixels the value in the calibrated image (after offset/bias subtraction, dark subtraction and flat division) is interpolated from the neighbor pixels.

[RobH]

As other have said, it sounds like you have a temperature mismatch.

Always subtract bias from flats, but if you are dark subtracting your lights, you don't need to bias subtract them too.

[planetdnb]

Thanks for the response guys. I think the temperature mismatch is the likely candidate. It was probably 5 degrees warmer when I took the darks.

I use Nebulosity 2 for all my stacking/subtracting so I'm not sure what methods it uses, there aren't really any choices of methods.

What's the difference between a bias frame and a dark flat?

I basically took my bias frames by setting the camera to the shortest exposure time and taking pics with the lens cap on.

Also, when taking darks, I usually just put a lens cap on the camera and leave it outside. Is this ok or do I need to have the camera attached to the telescope as it would be when taking my lights?

Thanks

[RobH]

A dark flat (or flat dark) is a dark frame done with the same exposure length as the flat.

In practice though, flats are normally of short enough exposure length that a bias frame will do jst as well, and you don't need to shoot them every time you shoot flats.

Leaving your camera outside with the lens cap on for darks is fine, it doesn't need to ba attached to the scope.

A 5 degree difference between your lights and darks is certainly enough to cause problems.

The way you are shooting your bias frames is fine.

Cheers

Rob

[dph1nm]

I can see the images have lots of hot pixels in them. When I apply my darks the hot pixels seem to just turn from white to black.

If the hot pixels are saturated (i.e. truly white) then you can't correct for them with a dark - you will just get 0 left where the pixels were. If you have a non-zero sky background then these will show up as black spots.

NigelM

[RobH]

If the hot pixels are saturated (i.e. truly white) then you can't correct for them with a dark - you will just get 0 left where the pixels were.

NigelM

Yes you can....I did it yesterday with subs from my SXV H18.

If they are the same on the darks as the lights, they will be corrected.

[planetdnb]

thanks very much guys. I reckon temparture is definitely my issue in this case. I will have to try and touch up the hot pixels with photoshop clone and stamp tool.

[dph1nm]

Yes you can....I did it yesterday with subs from my SXV H18.

If they are the same on the darks as the lights, they will be corrected.

If your camera has, for example, a range of up to 65535 and your hot pixel has a value of 65535 then there is no information from the real exposure left. I used to have this problem with my old P&S camera (although 255 was the maximum there!).

NigelM

[ollypenrice]

thanks very much guys. I reckon temparture is definitely my issue in this case. I will have to try and touch up the hot pixels with photoshop clone and stamp tool.

There is a quicker way. Use Select-Colour Select to pick the hot pixels. The fuzziness slider lets you control how far either side of the selected pixels you want to choose. Don't forget to use the plus tool on the selector.

Then try select modify expand (by one) and then filter, noise, median. Try one or two as the value. You can zap the beggars by the dozen that way.

Or there's healing brush, which is quicker than clone stamp.

Olly

[Photohound]

from looking at the reply dates, i guess this thread is drying up, so maybe i should look further afield, but just in case someone is still following.... As a newb it would really help me (and probably others) to gain info from this thread if someone could give a glossary of the terms used here. i'm familiar with the hot pixel theory, just getting into astro imaging now - all this dark frame, flat, white pixel, non zero sky, master dark etc.... when i study 'lens cap' shots of my DSLRs i see a couple of pure white squares, a couple of blue smudges - oval i think, and a few red blobs too. Sorry about the lay terms - lol.

[roundycat]

see if there is anything here that will help.

Astrophotography

Dennis

[Photohound]

cheers dennis, that is a good start - many thanks

Anymore 'easy reading' suggestions or comments on the 'white-blue-red dots' gratefully accepted.

[Psychobilly]

If your going to be using DeepSkystacker (DSS) then it might be worth reading the online help files... It uses slightly different terminology...

Billy...

[Photohound]

Hi PB, thanks - i have registax 5.1 loaded at the mo, whats the real term pro's and con's?

[Starflyer]

The camera's internal temperature is written in the image's Exif data. Using a tool like Kuso Exif Viewer you can load your lights and darks and compare what the temperatures were at the time you took them.

I believe that the APT program now displays, and can write the camera's temperature into the file name - very handy for building library darks for a Canon DSLR. There's a thread here, started by Ivo; the author of APT.

[Psychobilly]

Both "Hot" and "Cold" pixels can both be removed in DSS the settings you need to look at are under the cosmetic tab....

Billy...

[planetdnb]

coming back to this thread. I managed to capture 25 darks in pretty much identical temperatures as my lights.

I then went through the whole stacking/combining/dividing process with the new darks and the hot pixels where still as abundant as before. I guess maybe they were saturated 65535 or whatever the value is.

I discovered yesterday in Nebulosity 2 that I can create a bad pixel map from my master dark and use that instead of dark frame subtraction.

that successfully removed every hot pixel from the image! YAY!

So that said, with my obviously old sensor that has hundreds of hot pixels, my process will be as follows:

1. Take Lights
   
2. Take Flats (same night)
   
3. Take Bias or Flat Darks (same night)
   
4. Subtract Bias from lights and flats
   
5. Combine flats
   
6. Use hot pixel map on lights
   
7. divide flat master with lights.
   
8. combine lights.
   

I believe that Hot Pixel Mapping also has the added benefit of not introducing any noise that dark frames may do.