General processing question!

[Gasman]

HI guys

Regarding processing fits files and in particular for spectra I`ve always been of the opinion that if I wanted to take lights with exposures of say 60secs then I must use darks of 60 secs, flats of 60 secs, bias 60secs etc ie all the same exposure  but I`ve been looking on a spectro web site going through a tutorial on processing spectra quoting one example where the guy used lights of 60secs, darks of 300secs,flats of 3secs, bias ?? so is this method ok to use?. I thought all exposures of the relevant fits files had to be the same !, for instance you could take a light of say 60 secs where no hot pixels are visible but a dark of 300 secs might cause hot pixels to appear so the processing software might try to get rid of hot pixels that are not present in the lights!

Is it ok to use flats,lights,darks and bias with varying exposures of the same star?

cheers

Steve

[robin_astro]

Hi Steve,

I think provided the camera is stable (eg well temperature controlled), scaling the thermal part of the signal (bias - dark) depending on exposure is quite common in CCD image processing and quite a time saver as it takes a long time to make a stack of darks every time that is big enough so it does not degrade the SNR.  (The dark exposures should be longer than the light) Pixels that are hot or cold or likely to become saturated in the light are best handled with a cosmetic frame which flags pixels likely to be saturated and calculates their value in the image based on the surrounding pixels. In fact with some cameras where the thermal signal is very low (eg the Sony CCD) It can be best to just use a bias (which can be a big stack to reduce noise without taking up too much time) plus a cosmetic and forget the thermal as the resulting SNR will be better.

Robin 

[robin_astro]

Note that to work properly the software must calculate the thermal component (bias-dark), scale that component depending on the ratio of the exposure and add it back to the bias.

I must admit though I personally prefer to run my camera at the same temperature and keep a library of calibration frames at various standard exposures (eg 1, 5, 10, 20 min) produced on cloudy nights so I can normally match the exposure of darks to the light exposure.

Robin

[robin_astro]

sorry, in the above for (bias-dark) read (dark-bias) if course 

Robin

[Gasman]

Thanks for that Robin. It can seem a bit hit and miss sometimes thinking back before I did any spectro and just tried imaging DSO`s. I built up a  library of darks varying from 5secs up to 90 secs at -12degs which I used for sometime quite successfully but how long can you rely on them if your ccd developes more hot pixels not shown on the library shots. Not having to take darks or bias shots every time would certainly help a nights observing as I have been checking a fresh spectrum for saturation and reducing the exposure accordingly only to then take a set of fresh darks/bias shots at the new exposure time . Needless to say not many stars are checked each night!

Steve

[MattJenko]

Each calibration frame type does a different job, and therefore they are not all the same duration as the lights. Darks are the only one which technically needs to be the same duration as it shows the accumulation of various noises in the exposure (thermal noise etc), but as Robin says, this can be scaled. Bias frames are to get an image of the noise generated by extracting the frame from the sensor, so is the readout noise, so these are very short - ie the shortest exposure time you can use. Flats are used to find and remove the imperfections in the image train itself: dust, vignetting etc, so the exposure time only needs to be enough to highlight these, you don't want to overexpose, just get a decent ratio, so AV mode on dslrs is generally considered fine (it's what I do) and for CCDs, it is enough exposure time to get a decent histogram for the calibration software to be able to deal with the various imperfections when calibrating.

Cheers

Matt

[Gasman]

Cheers for that Matt. I suppose we that do spectroscopy tend to concentrate on a single star which in my case is placed on the 25 micron slit of my spectroscope and so have a very restricted fov to work with so maybe noise with its hot pixels etc is as important if not more so than for other astro imagers and maybe not so much the flats??

best

Steve

[robin_astro]

Hi Steve,

A spectroscopic flat is very interesting as it includes both the usual spatial defects that conventional flats correct like dust, vignetting etc but also for slit spectrographs, it includes a spectroscopic dimension as you are effectively taking a spectrum of the lamp you are using for the flat.

At first sight this might appear to be a problem but in fact it can be a significant benefit as by doing a flat correction you are effectively exchanging the spectral response of the instrument (mainly due to the camera and grating which can have some nasty humps bumps and ripples in it) for the much smoother blavk body spectrum of typically a Halogen lamp. This means when you come to calculate the "instrument response" using a standard star the process is much easier as you can fit it using a nice low order smooth curve.  Here are some examples of the sort of ripples which the use of a flat will remove which would otherwise have to be removed in the instrument response calculation stage. 

http://www.spectro-aras.com/forum/viewtopic.php?f=8&t=1068

When making flats, you should ensure that none of the pixels are in the non linear region of the camera and it is typical to expose so the mean is roughly half the saturation level.

It is important to average many of these exposures to get enough total counts in each pixel in the resulting master flat so that you dont introduce excessive noise. (bias, dark and flat all introduce some random noise into the image, the price we pay for eliminating the systematic defects)

 If we talk in terms of signal/noise then this improves with sqrt of the number of electrons the CCD produces so if you want say an SNR of 100 in a single pixel in the flat you need 100,000 electrons which for my camera is ~300,000 counts.  To achieve this SNR in a flat covering the full spectrum, particularly at the blue end where the halogen lamp does not produce as much light and the CCD is less sensitive while avoiding saturation in the central bright region, you may need to sum many tens of flats. it is stll worth making the effort to produce good flats though if you want the best quality in your final spectrum.  (Dont forget when generating your master flat to correct the individual flat images with bias and darks too before summing them)

Robin

[robin_astro]

Sorry, that should read "if you want an SNR of 300 in a single pixel...)  not 100

Robin

[Gasman]

Thanks for that Robin, makes a lot of sense . I have perhaps in the past been a little lazy taking a few spectra of a star, making sure they didn`t saturate, and putting them in ISIS with a single 120sec calibration shot of my Neon/Argon lamp and expecting ISIS to produce the goods, didn`t always bother with darks or flats. Rereading above make that `very` lazy. Must do better

Steve

[robin_astro]

... putting them in ISIS with a single 120sec calibration shot of my Neon/Argon lamp and expecting ISIS to produce the goods, didn`t always bother with darks or flats. 

Hi Steve,

You are lucky ISIS lets you do that now. Early versions insisted on darks and flats or it would not run  (Christian Buil  can be very strict sometimes   )

Robin

[andrew s]

Steve, don't feel bad. As I am sure Robin would agree the amount of processing needed/required depends on what you want to do with the spectra.  While doing a search for candidate Be stars I did no processing at all. I just looked at the spectra raw in RSpec (with background subtraction to improve visibility). I knew which pixel corresponded to H alpha. Of the 350 stars I took spectra of I only processed the 4, two of which turned out to be candidates.

Regards Andrew

[robin_astro]

Yep, absolutely.

If possible, it is always a good idea to start with what you are trying to measure (eg how faint, how often, what accuracy, precision, resolution wavelength range etc is needed etc,) Armed with that information, you can see whether you have the right equipment, what factors are important and what can safely be ignored).

For example recently I have been measuring spectra of a small region of the Deneb spectrum just for the purposes of measuring day to day variations in the radial velocity. Key factors there were resolution, SNR, the accuracy and repeatability of the wavelength calibration, heliocentric correction, careful removal of interfering telluric lines. Factors such as correcting for the broad instrument response, atmospheric extinction  and absolute flux calibration could safely be ignored so no real need for a reference star (Though I did use one anyway just to independently confirm the accuracy of the radial velocity measurement.)

If you are just taking spectra speculatively, perhaps to add to a general database, then it is useful to know you have everything you need to do a full data reduction  if needed so that if something unusual and perhaps transient turns up in the spectrum you can demonstrate that it is real.

Cheers

Robin