Do I correctly subtract dark frame?

[crovax]

Hi,

I've recently went through a wonderful article on afocal astrophotography (Afocal Astrophotography) and decided to try to use the theory I've learned. Well, since I haven't seen clear skies for months already my test object is a book in a dark room. All the photos below are ISO 1600, and 4 sec exposure. I decided to go with 1600 because I thought this would give me very noticeable noise and therefore very noticeable results of dark frame technique. The exposure is 4 sec because I wasn't able to make the room any darker

First I took 5 photos of the object. Then I took a dark frame photo. So far so good. Then I went to PC, opened GIMP and put all the 6 images on different layers. For the first 5 I used "Addition" mode and the result is quite expected. And then I tried to use "Difference" mode for the dark frame. I also tried "Subtract" but these two different modes produce result that look the same so I decided to stick with "Difference".

And that's the place where I get the unexpected result. I sort of hoped that all the horrible noise from the photo will be gone but this didn't happen. The photo changed a little but I can't tell whether it became better or worse. I've uploaded all my result here

Picasa Web Albums - eugene.bondarenko - testing...

If you hit "Download" you can get any image in the original size.

Could you please tell me what I am doing wrong. Is it because of big ISO? And if yes, how is this noise called and is it unremovable? On the other hand if this is Dark Current Noise after all, then why didn't it get removed after I subtracted the dark frame? Or did I subtracted it in a wrong way? Or is the result not noticeable because I stacked only 5 frames and if I had stacked 100 photos, it would have been noticeable? Uh...sorry for so many questions

[RobH]

You need to get a whole bunch of darks and stack them first or you will end up adding noise.

I typically shoot 25-40 darks and stack them using average combine, then apply both the master dark, and a master bias made of 50 subs, plus flats.

I don't use GIMP so can't comment being a Maxim user.

Obviously, using lots of darks means a lot of time spent shooting them. As an example, last night it was cloudy but cold, so I set the camera off doing 20 minute darks and left it all night, and by this morning I had almost 30 of them.

Cheers

Rob

[crovax]

Ohh, that makes sense now!

Just to clarify things, "master dark" is the resulting image that we get after combining our X dark frame shots, right?

Also I don't understand very well what master bias is and what "subs, plus flats" are? I read about nature of bias noise and article suggests taking shots with 0 sec exposure. I doubt I can do this since my camera is not a pro one, it's a simple canon powershot a580. Is it possible to go with master dark only since as far as I understand it contains bias noise (as well as dark noise)? Or are the bias 0 sec shots a must?

[astrotonio]

Usually I substract dark frame (or I would say median or average of dark frames) from each individual image prior to add the dark substracted frames. Since dark is there to mimic the electronic noise that builds up in the CCD matrix photosites and adds to the electrons generated from the incident light, you have to use same temperature and same exposure time for dark frames and light frames. Having added 5x4s light frames, you have also added the dark currents contribution, but you substracted only one 5s dark frame so you left a major part of the noise. The dark current will have a fairly constant, temperature dependant component that will build up steadily over time and a random noise component. This is the reason why you need to average several dark frames, because average of the random component is essentially zero, leaving only the constant portion. This technique by definition will remove only the "proportional to exposure time" component of the noise in your light frames, leaving the random dark current contribution. Again, when adding your light frames individually corrected by the average dark frame, the random noise component leftover will essentially disappear.

Clear skies

[RobH]

Ohh, that makes sense now!

Just to clarify things, "master dark" is the resulting image that we get after combining our X dark frame shots, right?

Yes, correct.

Also I don't understand very well what master bias is and what "subs, plus flats" are? I read about nature of bias noise and article suggests taking shots with 0 sec exposure. I doubt I can do this since my camera is not a pro one, it's a simple canon powershot a580. Is it possible to go with master dark only since as far as I understand it contains bias noise (as well as dark noise)? Or are the bias 0 sec shots a must?

Here is a quick synopsis....

Lights are your images....the shots the camera takes.

These must then be calbrated, and calibration frames are used.

These are,Bias, Darks and flats, and if you are really picky, flat darks

An individual exposUre is known as a sub.

Bias frames are a record of your system's inherent noise with zero exposure.

They are of the minimum exposre length you can get, if possible, zero.

They are taken with no light entering the scope.

Darks are a record of the noise when exposing and are very temperature dependent.

They need to be of the same length as the exposure.

They are taken with no light entering the scope.

Flats are a record of all the optical irregularities in the system, such as vignetting, dust particles etc etc in the optical train.

They are taken using an evenly illuminated lightsource, commonly a lightbox, or electroluminescent (EL) panel.

Flats need to be bias subtrated in order for the correct subtraction calculation to be made. If they aren't, they will either over, or under compensate.

There are also flat darks. These are dark frames of the same exposure length as the flats.

I find that the flat exposures are normally quick enough not to worry about noise buildup, and bias frames handle the rest, so I don't do flat darks, but I am being sloppy

I apply bias subtracted flats, bias, and temperature matched darks to my individual subs. The master bias, dark and flat are comprisd of more than 20 individual sub exposures.

It takes ages and is really boring though....scopes should come with an assistant!!

In your case, with a DSLR, just take lots of darks to begin with, stack them, and then apply the master dark to your individual frames, you will be amazed at the difference it makes.

Forget bias, and leave flats until later.

HTH

Rob

[crovax]

Hi,

Thanks for the awesome explanations!

I've already took all the darks and now I am trying to create a master dark of them. Unfortunately I got stuck at this point. It seems GIMP doesn't have this feature (or it is named in a way that I can't recognize it). I found a plugin that is supposed to do this but it is very buggy and I can't get the results out of it.

I also tried looking in DSS and Photoshop but still no luck. Could you give me link to Maxim software you mentioned earlier. Also do you by chance know Linux software that can average/median images? My main and the most powerful PC is running Linux and I feel lazy about installing virtual Windows on it. However if this operation is not very resource hungry I can always perform it on one of my old computers that run Windows.

[crovax]

Oh, never mind, I guess I found the median tool in photoshop. It's located in layer -> smart objects -> stack mode -> median

[crovax]

Uh, well, I've created a median layer but after all the noise didn't go away. I believe that I may have messed up something when taking the photos so I am planning to redo everything from the start. However before I do this, could you please let me know whether the following procedure is correct:

take X subs, take X darks, average darks into master dark, subtract master dark from each sub, merge subs using "addition".

If I am mistaken here, please correct me. Also I am not sure whether there is some difference when I "subtract" master dark from sub or when use "difference" mode. And the final place where I have doubts is ISO. I've been playing with ISO 1600 but I wonder whether this is too extreme or not?

[Daveskywill]

Hello Crovax:

My one recommendation is, for one thing, in each one of your dark frames, it looks like you didn't put the lens cap over your lens.

If you were in the field taking astro photos, you'd need to try doing that (putting lens cap over front) or you won't get the right results. Like you know that "darks" are only meant to sense the dark current, like hot pixels of your sensor. So you don't need to also look at the image during this.

I sort of realized this because your darks all showed a very faint image of what you wanted in your "lights".

However, it looks to me that there is hardly any noise in your darks even considering that. And so either you took those darks in a cold place in your house, and/or your camera has litttle noise anyway.

Also what I'm getting at is it's sort of hard to do this test indoors at a subject that's not stars (in my opinion.)

So my recommendation is to go with a fast ISO that includes noise more, if you want to test it.

David

[dph1nm]

I sort of hoped that all the horrible noise from the photo will be gone but this didn't happen.

Dark will NOT remove the random (shott) noise from the image (which I think is what you have in your images) - it is not their purpose. The only way to beat this down is to expose for longer.

NigelM