Calibration in DSS..... Is it doing it wrong?....RESPONSE FROM LUC OF DSS

[RobH]

Hello all.

Peter (psychobilly) and I were having a chat yesterday about calibration in DSS.

He pointed out that in this program, it tells you to both subtract darks and bias (plus flats of course) from your lights.

My understanding is that this is incorrect as bias information is already contained in the darks, so if you also subtract bias, you are double subtracting, and thus adding noise.

It's irrelevant what type of sensor produces the image, the principle of calibration applies to them all.

The complete calibration process as I know it is thus.....

Calibrate your lights with either darks, or bias, but not both.

Subtract flats from your lights, but either flat dark, or bias subtract your flats before doing so. Again, don't do both as the bias info is contained in the flat darks.

Peter knows his stuff, and is an experienced DSLR imager and also has lots of experience with DSS, so I'm interested in knowing why DSS does this....do they have some way of avoiding the double subtraction, or are they, in fact, doing it wrong!

Cheers

Rob

[roundycat]

Hi Rob,

I wouldn't touch DSS with a barge pole, partly because I have been satisfied with Maxim for years and partly because I hear about so many people having trouble of one sort or another with DSS that it seems to be more trouble than it is worth. My views on DSS may therefore not be very worthy!

Bias is contained in any and every exposure that you make. It is, quite literally, a snapshot of the chip as it switches on.

I guess it would be easy enough to take a light and subtract bias from it twice. That would give you a good read on the effect on the noise and also any odd effect due to subtracting the pedestal twice.

The flat fielding process is a double division rather than a subtraction but other than that my understanding of the calibration process is exactly the same as yours. I think maybe you should use darks rather than bias to get the best result but bias with a bad pixel map made from a master dark will take care of the very hot pixels just as well. Add in dithering and you get a nigh on perfect result.

Dennis

[RobH]

Thanks Dennis.

I also would subtract darks rather than bias, plus a bit of dithering to do the final cleanup so to speak in practice.

I've never used DSS as I too use Maxim, but obviously Maxim isn't cheap, so I think it would be interesting to clear this up for those who don't have it.

Cheers

Rob

[narrowbandpaul]

Hi all

Do not use both darks and bias under any circumstance.

As dennis correctly points out every single exposure that ever comes off the camera contains the bias data. Either do darks and flats only (if you have long subs) or bias and flats only (if you are using very short subs)

of course you need to remove the offset (bias) from your flats for them to work properly.

but never use both darks and bias to correct your lights. bad things will happen!

paul

[RikM]

In the help section of DSS there is flow chart of what it does. If I read it right when you use just Darks (including the bias signal) they are subtracted in the normal way. If you include specific bias frames these are subtracted from the Darks before the tweaked darks are subtracted from the lights, the bias is also subtracted from the lights and flats so everything matches.

[SteveP]

How DSS processes darks and bias frames doesn't make sense at first glance. As others have said, every dark already contains the bias 'information'. My only explanation could be that DSS' dark fames are not really dark frames but rather flat darks, ie having had the bias component already removed, which makes any subsequent scaling of the 'darks' that much easier

Steve

[RobH]

If you include specific bias frames these are subtracted from the Darks before the tweaked darks are subtracted from the lights, the bias is also subtracted from the lights and flats so everything matches.

That means that the bias has been subtracted from the darks, so they no longer contain bias information. If this is the case, then you would need to subtract bias from your lights as the darks wouldn't be doing it.

It would work, but doesn't make a lot of sense to do.

Rob

[narrowbandpaul]

that would be utterly pointless if DSS did that. It makes absolutely no sense to do so, and would increase the random noise contained within the subtracted frame......

[jgs001]

Whenever I use bias frames in the general mix in DSS I get a horrid green cast in the image that's really hard to deal with. So now I load up one light, the bias frames and the flats and stack that. Then take the master flat and use that along with the darks when stacking my lights. This seems to work well, it's certainly producing better results than using bias for the main stack. Why this should be I'm unsure, but it's how I've dealt with.

[Psychobilly]

[RikM]

You are right, it doesn't seem to make sense, but that's what it says it does.

I never use bias frames with DSS (because there is no need), only darks and flats. But if you did put bias frames in there as well it wouldn't do any harm??? Just make the calibration take longer and use a bit more computer power.

EDIT: must type faster...

[roundycat]

I can understand why there is so much confusion. The charts above look like so much cobblers to me. Firstly, the writer should stick with convention and refer to Flat Darks rather than Dark Flats. If only software writers would all sing from the same hymnsheet it would make life easier.

Sheet one above seems to say that we need to subtract Bias from the Darks to make a Master Dark. This is only necessary for Scalable Darks which do not actually get mentioned.

The middle set of blue-grey arrows seem to imply that Bias is subtracted from both Darks and Flat Darks which is quite wrong.

If you are not making scalable darks and have the proper Flat Darks for your Flats there is no need for Bias. Ever.

Needless to say that whilst your Darks might be 600s to match your Lights, the Flat Dark might only be 3s as it needs to match the exposure (and temp) of your Flat.

Sheet 2 is a bit clearer but it still seems to show that Bias is in some way involved in the Darks. All that should happen in this case is that the Master Bias is subtracted from each Flat before they are combined to make the Master Flat. Clearer but still badly drawn.

Both sheets one and two have blue-grey arrows going from the Bias down to the 'Light -> Calibrated' line. This, to me, clearly implies that you use the Bias to do something to the Light before you Dark subtract. Quite wrong.

Sheet three is the simplest of them all and looks about right. I'm sure it could still be put more clearly such as in AIP 4 Win page 153. I'd copy it but I'm not too sure about copyright.

Dennis

[RikM]

Sheet three is the way I use it.

The 'dark flats' shown here should really be called flat darks right? They are dark frames taken for the flats, so it's written the wrong way round in the DSS help files? That is what I thought, but wasn't sure. It's always confused e a bit.

[martin_h]

So what is the conclusion from all of this, bias or not with darks?

[davew]

It could be the algorithm writer isn't the software writer.

If an answer is needed once and for all maybe the software writer needs contacting.

DeepSkyStacker : DeepSkyStacker

Dave.

[RobH]

So what is the conclusion from all of this, bias or not with darks?

The correct way is not to use bias if you're using darks, but you must either bias subtract, or flat dark subtract your flats (not both)

It looks like DSS does something else though....that's why I started the thread, to try to find out what!

It's not terribly clear.

I would suggest that even if you are using DSS, you do it the correct way.

Cheers

Rob

[martin_h]

So when making a master Flat - use Bias frames, but thereafter use master flat, master dark and nothing else.

Ive just joined the DSS Yahoo group and will post the question there and see what they come up with.

[RobH]

So when making a master Flat - use Bias frames, but thereafter use master flat, master dark and nothing else.

Correct.

Let us know what they say if you don't mind Martin.

Cheers

Rob

[martin_h]

I found this on the DSS Yahoo Group site -

"I would certainly like to hear from experimental results but I would

like to add something first: most software are just forcing the user

to manually do the same steps over and over while DSS is automating it

(why bother the user and let her/him make mistakes?).

This does not mean that DSS is doing it differently since all the

software are agreeing on what is the calibration process:

- creation of the master bias (if any)

- creation of the master dark (by using the master bias)

- creation of the master flat dark (id°)

- creation of the master flat (by using the master dark flat and

master bias)

- image calibration by

- subtracting the master bias

- subtraction the master dark

- dividing by the master flat

[roundycat]

I must comment on Martin's list above.

I won't reference DSS because I know nothing about it and do not pretend to understand the dodgy looking help files such as those posted above by Peter.

I am pretty much against automation in software if it means the user is obliged to follow the instructions blindly. Particularly with something as potentially imortant as data reduction or calibration it is really helpful to know what is going on from first principles.

Then Martin goes on to quote as follows.

"creation of the master dark (by using the master bias)". What nonsense is this. You shoot twenty darks, combine them using no alignment and a median output. That's it. No bias frames are needed.

Then, "creation of the master flat (by using the master dark flat(sic) and master bias).

To create a master flat you first shoot your flats. Then either subtract the correct exposure flat dark from each flat and combine them using a median output. Or, if you can't be bothered to shoot flat darks (could be as many as eighty) then subtract bias from each flat before combining.

Finally the quote talks about image calibration. Subtract the master bias and then subtract the master dark. WRONG!!! Completely wrong.

calibration is:

Subtract the master dark.

Apply the master flat (double division, once to get the flat ratio and then again to correct the light).

Job Done.

The master dark is as explained above. No Bias frames come into it unless you intend to make scalable darks. Depending on the linearity of your sensor and the difference between the master dark and your lights in terms of exposure this scalable dark may not work as well as you expect.

Proper flats should be dark subtracted to take care of the hot pixels. If you just subtract Bias from each flat you should also apply a bad pixel map as this will take care of the hot pixels.

For the most part temperature is important. If your darks are too hot or too cold they will over or under compensate. You will be left with oceans of dark or light pixels.

Dennis

[RobH]

Thanks for the quote Martin.

So, as Dennis says, it's being done incorrectly if the program is doing what you quoted from the yahoo group.

I think I might contact the DSS team and point them to this thread to see what they say....we may be misunderstanding their process and they shoud be allowed to comment, as it's not at all clear.

Cheers, and thanks to all.

Rob

[RobH]

It seems that there's no direct way to contact the DSS folk, except via the yahoo group, which I don't really want to do.

Martin, as you've already joined, if you don't mind could you point the relevant folk to this thread via PM.

Cheers

Rob

[martin_h]

I found that information in a previous post on the group, (it was a very early post) I'll wait and see if I get a direct answer to my post (things might have changed) and then point them in this direction.

[martin_h]

From the man himself! quote -

Hi Martin,

In fact the bias *must* be subtracted from the flat frames (unless you are using

dark flats and no bias frames).

But what would be the point of not subtracting the bias from the light and dark

frames?

The bias does not contain a constant value, but a signal that a proper

calibration process must use.

This is why DSS is subtracting the bias from all the frames.

Now, if you want only to subtract the bias from the flat frames, just tell DSS

that your bias frames are dark flats. This will do the trick.

Clear skies,

Luc

[RobH]

Well done Martin.

So, DSS does it wrong.

Now we know.

Cheers

Rob