Jump to content

Welcome to Stargazers Lounge
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. If you already have an account, login here - otherwise create an account for free today!
Photo

Calibration in DSS..... Is it doing it wrong?....RESPONSE FROM LUC OF DSS

- - - - -

  • Please log in to reply
95 replies to this topic

#61
roundycat

roundycat

    Banned User

  • Beyond The Event Horizon
  • 1,402 posts
  • Location: near Harwich
I really have never seen or heard of anything quite so daft. I wonder how many finished images could look a lot better if the DSS writer/s had used a common sense approach?

Dennis

#62
BlueAstra

BlueAstra

    Sub Dwarf

  • Advanced Members
  • 1,641 posts
  • Location: Manchester-UK
I must admit I am one of those people who throws lights, flats, darks and bias into DSS, and after selecting preferred options, lets it get on with it. I can see now that this may not be the right thing to do, and illustrates my lack of understanding of this process and some of the terms used. I think it would be really useful, and benefit a lot of users, if someone could compile a simple summary step by step process list specifically for DSS users based on the discussion here. Maybe it could become a sticky for 'preferred' DSS processing.

Scopes: RC250-TT, AG10, Meade 5000 ED80, Mounts: PMX, Cameras: SX Lodestar & SXVH16, DMK
Optics: Hutech IDAS LPF; Baader S2/Ha/O3, LRGB; SX OAG, Quantum 7x2" FW; My Telescope Shops Location List is Here


#63
RogerTheDodger

RogerTheDodger

    Brown Dwarf

  • Advanced Members
  • 2,188 posts
  • Location: Halesowen UK.
As I'm just starting out I'd appreciate that too :)

#64
martin_h

martin_h

    White Dwarf

  • Advanced Members
  • 3,263 posts
  • Location: Grimsby-UK

So does this mean, that the way I'm using DSS is producing results that are not properly corrected ?

I load an image... (whatever I have handy, although normally a light), all my flats and the master bias (from previously, although if I wanted to create a new master bias, I'd do all the bias too). Stack it, creating a bias corrected master flat. Clear the list, load my lights, darks and master flat, and let DSS do it's thing on all the frames.

If I was combining multiple nights data, I'd create that nights master flat in the same way, but then load up my saved file list, and add the new nights data to a new group.



No you are doing it the correct way

Edited by martin_h, 14 February 2011 - 11:45 AM.

Telescopes: William Optics Zenithstar ZS71 ED. CCD: QHY9 mono., Filter Wheel: 2" True-Tech Supaslim Manual Filter Wheel+Baader LRGB CCD Filter set & Baader Ha,Oiii, Sii. Guiding: Skywatcher ST80 + QHY5. Mount: NEQ6 Pro Synscan + EQMod



http://www.wix.com/martin_h/shed_01
Grimsby- UK

#65
jgs001

jgs001

    Hyper Giant

  • Global Mods
  • 16,581 posts
  • Location: West Sussex
thanks for the confirm guys... and that also explains why when I used bias and flats in the way you'd expect I got some very odd results...
John

Celestron 80ED, Konus Vista 80s, GSO 6" RC

#66
narrowbandpaul

narrowbandpaul

    Brown Dwarf

  • Advanced Members
  • 2,369 posts
  • Location: Central Scotland
I admit that I too use DSS. I liked the fact that you just threw images at it and it did the business. I assumed that the guy who wrote knew what he was doing. Clearly not.

So I think I will ditch DSS, as the number of subtractions that are performed is more than is neccessary which will add noise. What a stupid method. I will inform my friend in the states who really knows what he's talking about when it comes to CCD imaging. Im sure he will have a few words to say about this....

I have a 90 trial of maxim, so if that works nicely, which im sure it will, then a purchase of a real programme like maxim will be required.

DSS fail!

paul
10inch; F4 Custom Made Orion Optics
1/10th wave optics
William Optics ZS66 Apo with FF2 (duff flattener)
Canon 300D

#67
RobH

RobH

    Global Moderator

  • Global Mods
  • 6,946 posts
  • Location: South West England
I'd be very interested to see what he has to say Paul.
It would be nice to be able to summarise this properly and then contact the DSS designers.
Unless they are particularly precious, it may well halp them improve the program, which is great in the faxct that it clearly is useful for many people who can't or don't want to, invest a lot of money in Maxim or the like.

Rob.

#68
dph1nm

dph1nm

    Sub Dwarf

  • Advanced Members
  • 1,445 posts
  • Location: Durham
As far as I am aware, DSS does allow for scalable darks. So you do have to subtract the bias from the dark (although you might prefer to do it only when you have to)!

NigelM

Edited by dph1nm, 14 February 2011 - 02:18 PM.

Fullerscopes 8.5" f6 Newtonian on an original HEQ5 (with GOTO upgrade) in an Alexanders Observatory,
Nexstar SLT 102mm refractor, Canon 1000D DSLR, Panasonic TZ18 digicam, Ioptron Skytracker.
Professional astronomer working on the Pan-STARRS and VST ATLAS surveys.

Kielder Observatory trustee.


#69
dph1nm

dph1nm

    Sub Dwarf

  • Advanced Members
  • 1,445 posts
  • Location: Durham
If I have one bias, one dark and one light, and I subtract the bias from the dark, then subtract the resulting bias-subtracted dark and the bias from the light, this is the same mathematical procedure as subtracting the original dark from the light. No extra noise is added. I can make the same argument for N x bias and N x dark. So where is DSS going wrong?

NigelM

Edited by dph1nm, 14 February 2011 - 02:44 PM.

Fullerscopes 8.5" f6 Newtonian on an original HEQ5 (with GOTO upgrade) in an Alexanders Observatory,
Nexstar SLT 102mm refractor, Canon 1000D DSLR, Panasonic TZ18 digicam, Ioptron Skytracker.
Professional astronomer working on the Pan-STARRS and VST ATLAS surveys.

Kielder Observatory trustee.


#70
Dangerous-Dave

Dangerous-Dave

    Sub Dwarf

  • Advanced Members
  • 1,338 posts
  • Location: County Durham
Fascinating thread.

I wish getting the calibration slightly wrong was the limit of my problems with it. I seem to constantly get a very soft image that looks like it's been run through a median filter but damned if I can find out where it's happening. Simple answer - use Maxim!
EQ6 Pro, Vixen GP, WO Megrez 90, WO Megrez 72, Vixen R200SS, SXV-H16, Modded 20D, Samyang, Leitz and Zenitar lenses, DMK21AU04, various bins

Dave's pics

#71
RobH

RobH

    Global Moderator

  • Global Mods
  • 6,946 posts
  • Location: South West England
OK....Two things now need to happen.

Firstly, any untoward bashing of DSS needs to stop.
For the record, I think that the guys who write DSS are to be applauded.
They have come up with a program that many use very successfully, at no financial benefit to themselves, and it must be a heck of a job writing a program like that.

Secondly, for the authors of DSS to be brought in on the discussion as we are now going round in circles, and they may well be able to provide answers that will finally clear up any confusion.
Without knowing how the DSS algorithms work, we are shooting in the dark.

Cheers
Rob

#72
martin_h

martin_h

    White Dwarf

  • Advanced Members
  • 3,263 posts
  • Location: Grimsby-UK
I have posted a link to this thread on the DSS yahoo groups for discussion, so hopefully we will get a definitive answer.
Telescopes: William Optics Zenithstar ZS71 ED. CCD: QHY9 mono., Filter Wheel: 2" True-Tech Supaslim Manual Filter Wheel+Baader LRGB CCD Filter set & Baader Ha,Oiii, Sii. Guiding: Skywatcher ST80 + QHY5. Mount: NEQ6 Pro Synscan + EQMod



http://www.wix.com/martin_h/shed_01
Grimsby- UK

#73
RobH

RobH

    Global Moderator

  • Global Mods
  • 6,946 posts
  • Location: South West England
I've just emailed the DSS owner too :)

#74
Shibby

Shibby

    Brown Dwarf

  • Advanced Members
  • 2,750 posts
  • Location: aka Lewis, East Anglia
Well said. I like DSS very much, but then I am... Biased... (pause for effect). It's made at least one step of the difficult process of learning deep sky imaging much easier, and I see no reason why not to continue using it.

Thankyou all for this thread, I'm learning much! The way I understand it then... (or at least the way I will use it from now on)

Stacking images using DSS:
1) Load flats, bias frames, and a dummy light.
2) Stack
3) Clear the list
4) Load lights, darks and master flat (from 2)
5) Stack
6) Job done

Sorry, I think I'm just reiterating, and as you say Rob, shooting in the dark (another pun? :) )
Lewis
_____________________________________
SW 150PDS
Vixen GP + EQMOD
Canon 450d (self modified)
SPC900 (unmodded) + Orion Mini Guidescope
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Download my Android app: Astro Panel
_____________________________________

#75
RobH

RobH

    Global Moderator

  • Global Mods
  • 6,946 posts
  • Location: South West England
Didn't even realsise I'd made a pun :):D:D

#76
jgs001

jgs001

    Hyper Giant

  • Global Mods
  • 16,581 posts
  • Location: West Sussex
That's how I'm doing it Lewis
John

Celestron 80ED, Konus Vista 80s, GSO 6" RC

#77
dph1nm

dph1nm

    Sub Dwarf

  • Advanced Members
  • 1,445 posts
  • Location: Durham

that would be utterly pointless if DSS did that. It makes absolutely no sense to do so, and would increase the random noise contained within the subtracted frame......

Well I agree it is somewhat pointless, and a waste of processing power, but as far as I can see mathematically

Light - dark

is identical to

Light - (dark-bias) - bias

and does not add noise. I have always assumed this is what DSS does. My only concern might be if you use some fancy median combine for your master dark/bias frames, but even then I expect the added noise is trivial.

NigelM

Fullerscopes 8.5" f6 Newtonian on an original HEQ5 (with GOTO upgrade) in an Alexanders Observatory,
Nexstar SLT 102mm refractor, Canon 1000D DSLR, Panasonic TZ18 digicam, Ioptron Skytracker.
Professional astronomer working on the Pan-STARRS and VST ATLAS surveys.

Kielder Observatory trustee.


#78
RikM

RikM

    Many Sequins

  • Moderators
  • 7,389 posts
  • Location: Gloucester, UK

Stacking images using DSS:
1) Load flats, bias frames, and a dummy light.
2) Stack
3) Clear the list
4) Load lights, darks and master flat (from 2)
5) Stack
6) Job done


This makes sense to me so I will do it this way in future.:)

I really like DSS and would not be able to combine frames without it as I can't afford fancy software. If this simple 1,2,3 list is the way to get the best from it, it should help plenty of DSS users. What a smashing thread:hello2:
Rik
Skyliner 250PX; Explorer 150P; Televue Pronto; ST80; ES68°; Televue; Pentax XW; NEQ6 Pro; EQ3-2; AZ4; SXV-H9; QHY5; Canon 1000D: Sketches Deep Sky Images
Member of Cotswold AS "No anchovies unless otherwise specified. If conditions persist, consult your physician."

#79
davew

davew

    Sub Dwarf

  • Advanced Members
  • 1,568 posts
  • Location: South Manchester
An updated response from Luc - Yahoo! Groups

Dave

#80
RobH

RobH

    Global Moderator

  • Global Mods
  • 6,946 posts
  • Location: South West England
I've been in contact with Luc, who wrote DSS, and he has kindly explained how DSS approaches the suubject of calibration.

Here are my questions, and any relevant replies.......
It seems to me that the importanty question here is whether using multiple bias adds noise ior not.
NBP says it does, Luc says not.
I don't know....I'm a snapper, not a scientist!!!
BTW.....Can we please try to keep any responses civilised pleased :)

Rob.


Hello.

You may know that there's a thread active on Stargazerslounge at the moment regarding calibration in DSS.
I started the thread as there have, on occasion, been questions regarding calibration that have come up, and often DSS has seemed to go against the standard methods of calibration in the way it applies bias frames.

From what is being said, DSS subtracts bias from lights, darks and flat darks..

Bias doesn't need to be subtracted from darks, or flat darks, as the bias information is already in them, and will be removed once the light is dart subtracted, or the flat is flat dark subtracted.

If bias is removed from everything, it means that there is multiple bias subtraction going on, which adds noise.

The understanding of the experienced imagers on SGL, who use Maxim, or CCDSoft, is that calibration is thus....

Lights should be either dark subtracted, or bias subtracted, but not both.

Flats should be bias subtracted, or flat dark subtracted, but not both, before being applied to the lights.


If you don't mind, can you let me know how the DSS algorithms deal with the calibration process, and do they, in fact, multiple subtract bias?

I'd like to clear up the confusion on this matter, and put an end to what is now turning into speculation on the thread, and which badly needs input from yourselves.

Thanks for writing a program that is used by so many with great success.
You have my greatest respect for doing this, especially for no financial gain.Now, let me explain how DSS is doing calibration and you will understand that all is good and soundly done.



Hello Rob.
First, let say that you have the full set of calibration files (lights, darks, flats, bias, dark flats).

In this case, DSS is subtracting the bias from every other frames. (this is the first workflow diagram on the web site)

So:

Dark’ = dark-bias

dark flat’ = dark flat-bias

flat’ = flat-dark flat’-bias (which is strictly identical to flat-dark flat)

light’ = (light-bias-dark’) (which is strictly identical to light-dark) / (flat’)

So the bias is not subtracted twice from the light because it has already been removed from the dark.

Indeed : light-bias-dark’ = light-bias-(dark-bias) = light - bias - dark + bias = light - dark.

As a side note, as at least one contributor has noted, subtracting the bias frame is *adding* the noise of the bias frame to the light.

This is one important reason to create a master bias as noise free as possible by using a lot of bias frames (they are cheap to come by).

Of course, there is also a bias signal to be removed and it would be a shame not to get rid of it.


Now, if you don’t have the full set, you have two choices (and only two choices).

If you are using flats (and not using them will create vignetted images) you still need to remove the bias signal from the flat frames.

You can do it by using bias frames or dark flat frames. Either way is working since the dark flat frames also contain the bias signal.

The second workflow diagram is for when you are using only dark flats, and the third workflow diagram is for when you are using only bias.

Both are perfectly sound ;)

BTW, using dark flats is only making sense when the dark signal in your flat frames is strong enough to justify its subtraction.

Some CCD camera have a significant dark signal (and even hot pixels) even with very short exposures and/or they need a minimum exposure time.

In both cases the use of dark flats is mandatory.

PS: DSS is using the only calibration methods there is, the same as recommended by Iris and I still you like to see another mathematically sound working method.

Clear skies,

Luc



Thanks Luc.I'd figured out that you are effectively neutralising the use of double bias.

However, as you say, you do add noise, the more calibration steps you use.

What your method does is exactly the same as calibrating your lights with just darks, and bias (or flat dark) subtracted flats, but with more stages, where a small amount of noise is added at each stage by using bias subtraction more than once. (If noise is added at each stage....I make no claims to be a theoretician).

You don't need to bias subtract your lights when using darks, as the bias signal is there in the dark, and is subtracted when the light is calibrated by the master dark.

The only other bias component that needs to be removed is that contained in the flat, so this needs bias subtraction.

Consequently, would it not be easier to simplify the process, and cut out the extra bias stages?

On the other hand, I understand that in a lot of ways, DSS has been designed to be simple to use, and if you just throw everything into the mix, and do it the way you describe,the results are, to all intents and purposes, the same.

I agree completely that with all calibration frames, you need to cut down any possibility of adding noise, which is why I personally have bias masters made of 50 or more subs, and I use not less than 30 each of darks and flats.

For the reason you state, I almost never use flat darks, as they are of such short exposures that bias does the job just as well.

Cheers

Rob.



Hi Rob,

In fact subtracting the same master bias from the darks and the light frames is not adding any noise compared to just subtracting the master dark (with its bias signal and noise) from the lights.

On a theoretical point of view, the noise is added at each stage only if you are using *different* master frames in the stages.

Thus removing the same bias signal/noise is only adding the noise once.

Moreover, since you can easily achieve a noise free master bias by using a lot of bias frames (100+), the end result is that using a good master bias is often better than just using a good master dark unless you are using a equally large number of dark frames to create it (and it does come cheap :) ).

However, like I said before, if you insist on not using bias subtraction from light and dark frames, you can just tell DSS to view your bias frames as dark flats and only the flat frames will be bias subtracted.

In this case, for all intent and purpose, the dark flats subtraction will be the same as a bias subtraction.

Clear skies,

Luc







0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users