Flats

[Luna-tic]

I understand the purpose of shooting dark frames, but what are 'flats' used for?  Is there a general ratio of darks and/or flats to process with the collection of images when you stack them?

[geoflewis]

6 minutes ago, Luna-tic said:

what are 'flats' used for?

Flats are a picture of your optical train illuminated with a even (flat) light source to capture any imperfections that will also be captured in your light frames, e.g. vignetting, dust bunnies, dew spots, etc. The must be taken without moving the camera, ideally just prior to, or just after shooting your light frames.

I generally use 15-20 flats flatdarks and darks, but some folks shoot many more, but I find that number a good balance between quality and practicality

[Luna-tic]

Can you do without them entirely? What does it do to the processed composite if you don't make flats?  If you know your optical train is clean, does it make them unnecessary?

[geoflewis]

Flats are also used to correct any vignetting (bright centre fading to the edges and corners) caused by the optics, but there are other ways to achieve this, e.g. lens profile software, so yes you can process without shooting flats, but they don't take long to capture so I always use them.

[Galen Gilmore]

1 minute ago, Luna-tic said:

Can you do without them entirely? What does it do to the processed composite if you don't make flats?  If you know your optical train is clean, does it make them unnecessary?

Without flat frames you will have vignetting in your image, it gets more apparent as you make your initial curve stretch. So you can either just leave it as it is (image with vignetting), or crop the image to remove it.

Also your optical train can't be perfectly clean. Even if you have no dust on any lenses in front of the sensor, there can still be dust on the sensor of the camera. 

HTH

[Ken_nneth]

5 hours ago, Luna-tic said:

Can you do without them entirely?

For me the short answer is NO, without them your image is just not complete.

[Starwiz]

6 hours ago, geoflewis said:

they don't take long to capture so I always use them

Yes, only a few minutes to capture the flats.  The biggest pain for most initially, is finding or buying a uniform light source.

[ollypenrice]

Here's a demo. Do you need flats? Is the Pope a Catholic? 

So what's going on here? At the top we have a photo of an evenly illuminated flat light source (a 'flats panel.') Does it faithfully record the appearance of that panel? Ahem, not really! There is a huge fall-off in brightness into the corners and a smattering of faint dark circles caused by tiny particles fairly close to the sensor. This irregularity of illumination will appear in every image you take. Why does this not bother us in daytime photography? Because we leave daytime images more or less linear. We don't 'stretch' them because we are working with plenty of light. Left as a linear image the flat at the top would not look so unevenly lit but, when we stretch it (as we are going to stretch our images) we see just how bad the situation is.

At the bottom, the flat at the top has been corrected by the application of a flat - and now we have a faithful representation of the appearance of the light panel. This is one of the most striking demos to perform in real time during an imaging tutorial. You click on 'apply flat' and, bingo, you get the second image.

Olly

[geoflewis]

48 minutes ago, Starwiz said:

The biggest pain for most initially, is finding or buying a uniform light source.

Many people use twilight sky for flats, though that never worked for me, so what I used to do was shoot daylight sky flats. This would allow me to correct for vignetting, but not always dust, dew, etc. in the optical train. The way I would shoot these flats is to use a clear blue sky at the time of the month when the Moon was visible during the day. I would use the Moon to focus the camera, then slew to a part of the sky where there were no clouds and the light was mostly even, typically the opposite part of the sky to where the Sun was. I would then take a series of test shots checking each image where the central region was 18000 < ADU >28000, or camera histogram about 40% away from origin. If possible I would shoot these flats during the day prior to an imaging session the same night and then leave the camera hooked up, so that I very likely would also capture any dirt in the optics, etc., for best calibration. However, I would also keep the flats in a library and reuse them taking care to try and orient the camera in the same position in following sessions. As mentioned this may or may not deal with any dust bunnies, etc, but it would correct for vignetting.

Hope this helps, Geof

[vlaiv]

Just to mention, some CMOS sensors have uneven amp on pixels that presents itself as additional noise in the picture - flats take care of that as well.

Here is comparison between blue and red flats extracted (debayered) from master flat on OSC camera - you can see that there are differences in images and unevenness across the flats not caused by dust or vignetting (small sensor).

And yes, please take as many flat frames as possible and calibrate them well (subtract flat darks). Noise calculation for flats is somewhat complex (because operation is multiplication rather than division), but one wants their flats to be as noise free as possible, otherwise you will be multiplying image noise as well as adding more noise to it.

[ollypenrice]

45 minutes ago, vlaiv said:

Just to mention, some CMOS sensors have uneven amp on pixels that presents itself as additional noise in the picture - flats take care of that as well.

Here is comparison between blue and red flats extracted (debayered) from master flat on OSC camera - you can see that there are differences in images and unevenness across the flats not caused by dust or vignetting (small sensor).

And yes, please take as many flat frames as possible and calibrate them well (subtract flat darks). Noise calculation for flats is somewhat complex (because operation is multiplication rather than division), but one wants their flats to be as noise free as possible, otherwise you will be multiplying image noise as well as adding more noise to it.

Yes, do subtract flat-darks but, like many others, I find that a master bias makes a perfectly good flat-dark. While chasing a different problem I did a large number of comparisons between applying 'real' flat-darks and master bias as flat-dark and could find no difference whatever. If you don't apply flat-darks (either 'real' ones or master bias) the flats may over-correct as well as adding noise.

Geof's idea of focusing on the moon by day is crafty! Newt owners may not find daylight flats work for them because of light ingress at the bottom of the tube.

Olly

[vlaiv]

13 minutes ago, ollypenrice said:

Yes, do subtract flat-darks but, like many others, I find that a master bias makes a perfectly good flat-dark. While chasing a different problem I did a large number of comparisons between applying 'real' flat-darks and master bias as flat-dark and could find no difference whatever. If you don't apply flat-darks (either 'real' ones or master bias) the flats may over-correct as well as adding noise.

Geof's idea of focusing on the moon by day is crafty! Newt owners may not find daylight flats work for them because of light ingress at the bottom of the tube.

Olly

Yes, flat darks are so close to bias in cooled cameras anyway, but I did purposely call them flat darks because I advocate not using bias frames at all unless absolutely necessary (bad pixel map or something). I'm instead for using large number of darks (hundreds) which is feasible when using CMOS and short exposure, not so much when using CCD .

[ollypenrice]

10 hours ago, vlaiv said:

Yes, flat darks are so close to bias in cooled cameras anyway, but I did purposely call them flat darks because I advocate not using bias frames at all unless absolutely necessary (bad pixel map or something). I'm instead for using large number of darks (hundreds) which is feasible when using CMOS and short exposure, not so much when using CCD .

Actually I like bias! I use them as darks alongside a bad pixel map, but that's for anther thread I guess.

Olly

[Luna-tic]

Wow.  Ask a simple question.......LOL. Thanks for the great info. Do flats need to be current with the images and darks (meaning taken relatively within a close timeframe)? Or can I take a series of flats and just use those whenever I need them for a processing session? Same would go for darks, I suppose.

[geoflewis]

5 minutes ago, Luna-tic said:

Do flats need to be current with the images

Yes, ideally flats should be taken immediately prior to or after shooting your lights, without changing focus, or removing or rotating the camera. By creating flats you are trying to replicate the exact conditions that your camera experienced when collecting the lights data, then any anomolies will be the same in both your flats and lights, so the flats can be used to correct your lights. If you are shooting through different filters using a filter wheel, then ideally you should also shoot different flats for each filter.

[Luna-tic]

Good to know, thanks much.

Next question. What's the best stacking program?  I see Registax recommended, and there are a couple of others I can't think of the names. I plan to use Lightroom to post-process the images.

 

By best, I mean mainly the easiest to use. Convoluted programs confuse me, I'm not the sharpest knife in the drawer when it comes to computers.

[Aidan]

Check out Deep Sky Stacker.  It's a very popular program for stacking and lots of info & tutorials out there.   

http://deepskystacker.free.fr/english/index.html

Select the link to download v 3.3.4

 

[geoflewis]

14 minutes ago, Luna-tic said:

What's the best stacking program?  I see Registax recommended, and there are a couple of others I can't think of the names.

I think that Registax is more appropriate for processing planetary images than DSOs. I use ImagesPlus, but PixInsight seems to be the dominant one now, with AstroPixelProcessor being the new kid on the block, but these are all commercial products that you'd have to purchase. Deep Sky Stacker (DSS) is available for free download and lots of folks use that, so I'd suggest you might start with that.

[Luna-tic]

Deep Sky Stacker was the one I couldn't think of; it should be sufficient for my needs, and I'll also download Registax for some planetary I plan to do. Free works for me very well.

Thanks for the guidance.

[CrashEd]

Great thread - some really useful info. 

[ollypenrice]

17 hours ago, Luna-tic said:

 Or can I take a series of flats and just use those whenever I need them for a processing session? Same would go for darks, I suppose.

Flats: it depends on whether or not you disturb your gear between images. My setups are observatory based and don't get changed or taken apart. Flats may work perfectly for 6 months, or I may get a new dust bunny at any time. I don't shoot them for the sake of it, I shoot them when the ones I have stop working!

Darks: many of the best DSLR imagers don't use them at all. They substitute a master bias for a dark and use a large dither (12 pixels and up) between subs. I don't use them any more with CCD either. I found they often did more harm than good. I use 'bias as dark' and bad pixel map.

AstroArt is a superb stacking programme, in my view far better than DSS, very transparent and logical to use, and good value. (But not free!)

Olly

[RayD]

13 minutes ago, ollypenrice said:

AstroArt is a superb stacking programme, in my view far better than DSS, very transparent and logical to use, and good value. (But not free!)

In what way(s) would you say it is better?  I ask as I use DSS and PI but having seen Carole using AstroArt I was impressed with the general layout etc. so may consider using it.

[geordie85]

23 minutes ago, ollypenrice said:

 

AstroArt is a superb stacking programme, in my view far better than DSS, very transparent and logical to use, and good value. (But not free!)

Olly

Does AstroArt resize and align binned 2x2 with unbinned subs? (I'm talking binned rgb and unbinned luminance of course)

[ollypenrice]

15 minutes ago, geordie85 said:

Does AstroArt resize and align binned 2x2 with unbinned subs? (I'm talking binned rgb and unbinned luminance of course)

It allows you to do it dead easily. You just go to Image-Resize and type in the unbinned pixel dimesions. Click, and the binned data is now the same size as the unbinned. Then you have just your R,G and B images open (and L if you have it) and goto Image-Align All. Choose Star Pattern, Translation and Rotation, click, and they are all aligned to each other.

Olly

[ollypenrice]

45 minutes ago, RayD said:

In what way(s) would you say it is better?  I ask as I use DSS and PI but having seen Carole using AstroArt I was impressed with the general layout etc. so may consider using it.

Firstly it's incredibly fast. This encourages you to try alternative stacking routines, darks, no darks, bias as dark, bad pixel map, sigma, average, etc etc etc to find out what works best for you. It supports bad pixel maps, it has sophisticated and adjustable hot pixel filtration. It has a great 'repair line' function which will zap aircraft trails which haven't be dealt with by Sigma (though the Sigma routine is excellent and has improved since AA4.) It has a good suite of post processing filters like DDP, Deconvolution, Unsharp Mask, Gradient Removal, etc. (Many of these I don't use because I have them elsewhere but I already have Pixinsight and PsCS3 etc.)

It calibrates and combines a stack of 25 full frame CCD lights using bias, defect map, Sigma Clip, Hot pixel filtration and column defect repair on my old Win7 machine in maybe forty seconds. And above all the results are clean.

Many people (I think Carole might be one of them) who see it in use here just buy it on the spot. I'm not on commission, nor do I get any freebies. How bad is that??

lly

PS If you can get PI to stack and calibrate it does it well, but two people have sat where I am now and said, 'I'll show you.' After a lot of clicking there's a long wait and then they say, 'Oh. Oh. I don't know what went wrong there then...' I creep quietly back to old AstroArt. But, of course, a lot of this is familiarity. I've been using AA for over ten years and that may account for much of my appreciation of it.