Any Thoughts on Synthetic Flats

[broadsword]

Hi All,

I'm fairly new to imaging, using a fairly basic set up, and whilst I'm fairly pleased with results so far I've been struggling with image processing due to the combined effects of vignetting, heavy LP, and not using flats. I know I'll have to succumb to the flats at some point, but at the moment with clear skies on work days I don't see daylight/twilight and I haven't rigged up a lighbox or anything.

Whilst surfing through Googled DSLR images I noticed one which mentioned the use of synthtic flats. I found a tutorial on doing this (basicially involving using the liight image itself as the flat after having blurred out and averaging all of the detail) and found a way of doing it in GIMP. So far it has produced some impressive results on star clusters, less good/useless with large objects like M33, but I suspect it might be viewed as heretical in some quarters.

Any thoughts? If there's much interest I'll post a couple of images with/without the synthetic flat.

[Gina]

I don't see how a flat produced that way could deal with dust bunnies or vignetting.

[kirkster501]

An idea I had but could be nonsense... So after finishing the image (and before doing the darks), T-shirt over the front of the scope and hold up my iPad with a totally white screen up to the front of scope (touching the cloth of T-Shirt).

Fire off half a dozen 2 sec exposures.

Would that do the business?

Only issue with this plan is it would wreck night vision

Rgds, Steve

[broadsword]

Gina,

Being a beginner, I haven't a clue how these synthetic flats can help with vignetting, but in practice I find that, by removing a lot of the light pollution in the centre of the light image I avoid lots of harsh level/curves stretching which quickly lose the details in the dark corners otherwise. I know that synthetic flats can't do anything for dust motes.

Steve,

Great idea, liked the 'touching the cloth' reference. Hope to get an iPAD for Xmas so will also give this a try. I had a similar idea recently involving a Plasma TV screen and a white powerpoint slide. Does white light produced in this way still work OK? Also, are 2 sec exposures enough for falts?As I said, I'm only a beginner....

[kirkster501]

.....liked the 'touching the cloth' reference....

Lol... It's all the curries I eat Neil. I can't help it..

[russellhq]

I had a similar idea recently involving a Plasma TV screen and a white powerpoint slide.

I've done this for my flats and it seemed to work quite well.

[broadsword]

But to return to the original topic, anyone any thoughts on the merits of synthetic flats? Do they have any place at all in astroimaging?

[dph1nm]

Yes - speaking with my professional astronomer hat on, flats based on median combining real on sky data are probably the best you can do. However, there is the issue of getting enough signal (i.e. more than in your lights!). I guess you can overcome this by smoothing, but then you lose the small scale variations in sensitivity which the flat can take out. Works best if you are taking many different targets during the night and you combine all of them. Having light pollution gradients may also mess things up.

However, if you don't care about photometry, you can also use these flats as sort of super darks and subtract them rather than divide. This can work very well if you have light pollution gradients.

One approach I have used as an amateur is to take offset night sky flats. i.e. move you target just out of shot and dither around taking lots of exposures of fairly random bits of sky. This will get you fantastically flat images, but at the expense of 'wasting' good dark time.

NigelM

[broadsword]

Oops, I had been subtracting them rather than dividing by them anyway. Guess that's why they seemd good at rmoning the LP.

Your second idea looked interesting. How would you use the set of random flats? Stack them and then divide by them, smooth/blur them out?

Thanks for any advice....

[MartinB]

The synthetic flats you describe can be used to correct vignetting and can also help with gradients which flats wont. It is quite a crude method though and won't address the problem of dust bunnies which are a real headache particularly with bigger stretches.

The thing that puts people off flats is the stringent requirements that people describe. Light boxes are a nightmare, t-shirt flats are ok but you need to have a camera that doesn't let in light or is well screened, sky flats are a real headache because the time available is very limited, twilight isn't usually practical because you won't be in focus and you don't necessarily want to stay up until dawn. I use an EL panel which works extremely well and Steve's Ipad suggestion with t-shirt is excellent for smaller scopes.

The other thing people make a big point of is insisting that the rotation of the camera and other bits for the flats and lights must match perfectly.This may be the case for some set ups and a good counsel for perfection but for some it isn't a practical reality. All is not lost. Vignetting is generally even enough for a difference in rotation not to be critical. The worst dust bunnies are those on or close to the chip. You can rotate the camera all you like and you wont change the orientation of the those bunnies. If you retain the orientation of your filters with your chip then rotation of the imaging kit within the scope isn't the end of the world. You can strip down at the end of your imaging run without disturbing the focus position and come back at a more convenient time ipad in had, set up again and get your flats at your convenience.

[dph1nm]

Your second idea looked interesting. How would you use the set of random flats? Stack them and then divide by them, smooth/blur them out?

You can median combine them is DSS just like ordinary flats. This should remove the stars and leave you with a nice image of just the background sky. To be honest, when I have used them it is to remove very bad light pollution gradients, so I just subtract. Very useful if you have a large target like M31 filling your FOV, where conventional software gradient removal simply doesn't work. Dividing might not work so well in this case. A compromise would be to use normal flats first, then create this master offset sky frame, smooth it, then subtract it.

NigelM

[broadsword]

Thanks Nigel, that sounds very interesting, I'll try it. One question, though, if the flats are of different random starscapes, will DSS be able to stack them - won't it say they can't be aligned?

[dph1nm]

Thanks Nigel, that sounds very interesting, I'll try it. One question, though, if the flats are of different random starscapes, will DSS be able to stack them - won't it say they can't be aligned?

Read them into DSS as flats, then use the median combine for stacking flats. It doesn't attempt to realign flats (nor should it). The one issue I can think of is that if there are very bright stars in the images DSS might not be able to normalise the image backgrounds correctly before median stacking. In which case I suspect you will be left with stars in the final skyflat image. I guess you just have to try it and see what happens. You could try clipping rather than median I guess.

NigelM

[dph1nm]

Actually, thinking about it, if you want to subtract them you might be better off reading them in to DSS as offset/bias frames, and using a median filter (or clipping) to produce the master bias.

NIgelM

[ollypenrice]

But to return to the original topic, anyone any thoughts on the merits of synthetic flats? Do they have any place at all in astroimaging?

No, none whatever!! Good, genuine flats are utterly essential and that's that. For reasons which we have never fully fathomed Yves and I struggled to get flats to work on his large reflector and we had to do messy work-arounds. Touch wood, we seem to have cracked it now. Processing the images without flats was a pain. Fortunately most of our images were of galaxies sitting in dark starfields so Dynamic Background Extraction in Pixinsight could operate as a surrogate flatfield correction. As Martin said, artificial flatfielding is crude. If you have a nebulosity-filled image there is no way you'l get the fullest possible stretch without them. And like Martin we use EL panels here.

Laptop screens are a bit unpredictable. Some are fine but others have gaps in the spectrum they produce. I had one which generated no blue and another with virtually nothing in Ha. If using a one shot colour camera this fact may be disguised but it means that, even though you have somethng which looks like a flat, one of the channels may in fact be flatless. Worth a try though. The key thing is to expose till the histogram peak is about a third of the way over to the right. Experiences with Yves reflector seem to suggest that, for us, a third of the way really does need to be the maximum. The refractors don't seem to care whether it's a third or two thirds. (This may have nothing to do with reflector/refractor, of course. It might be anything.)

Flats: just do it!!! I know you can think of better things to do at three in the morning but when you start to work on the image your flats will be pure gold.

Olly

[Gina]

Thank you for confirming those points Olly

I guess I ought to get round to making a light box, I have some white foamboard to make it from and a disc of white opal acrylic. I think to be sure of a continuous spectrum, incandescent bulbs would be best. I guess white LEDs couldn't be guaranteed to have a continuos spectrum. To date I have been using a white T shirt during the next day with daylight. This means that I park the scope so there is the possibility of dust moving about and maybe even some change in light path due to flexing, though I think this is very slight if any.

[allcart]

So how would you test the spectrum of light that a screen or lightpanel produce??

[ollypenrice]

So how would you test the spectrum of light that a screen or lightpanel produce??

With a one shot colour? Split the colour channels and look at the histogram of each channel. In fact Ps will show you the histogram for the three channels without your even bothering to split them. Just go to WIndow, Histogram, and choose All Channel View.

If using filters and CCD you see the histogram of the individual channels as you take each flat.

Olly

[MartinB]

I've got 2 EL panels, a Gerd Newman one and another that I made up myself. Gerd makes a big claim gor the spectral eveness of his panel whereas mine is visibly warmer, almost peachy. In practice it isn't an issue, you just need to use slightly different exposures for different filters. Using an OSC the calibration software applies a boxcar filter to the RAW flats which effectively makes them mono anyway so irregularities in the spectum aren't of any relevance.

I take flats through every filter I've used for the image but, with my set ups, the bunnies from the filters are extremely faint (they get fainter and larger the further they are away from the chip). If all else fails you can use just one filter for all your flats, provided it is the same diameter. It may not be perfect but it will sort out your vignetting and the dust bunnies from your chip and will be much better than a synthetic flat.

I guess most imagers have spent time using synthetic flats along with the various gradient removal tools in photo processing applications such as photoshop. You can waste a lot of your life this way before finally discovering Gradient exterminator and PixInight's Dynamic Background Extraction tool which will, with a bit of effort, sort out the vast majority of gradient problems.

[ollypenrice]

I've got 2 EL panels, a Gerd Newman one and another that I made up myself. Gerd makes a big claim gor the spectral eveness of his panel whereas mine is visibly warmer, almost peachy. In practice it isn't an issue, you just need to use slightly different exposures for different filters. Using an OSC the calibration software applies a boxcar filter to the RAW flats which effectively makes them mono anyway so irregularities in the spectum aren't of any relevance.

I take flats through every filter I've used for the image but, with my set ups, the bunnies from the filters are extremely faint (they get fainter and larger the further they are away from the chip). If all else fails you can use just one filter for all your flats, provided it is the same diameter. It may not be perfect but it will sort out your vignetting and the dust bunnies from your chip and will be much better than a synthetic flat.

I guess most imagers have spent time using synthetic flats along with the various gradient removal tools in photo processing applications such as photoshop. You can waste a lot of your life this way before finally discovering Gradient exterminator and PixInight's Dynamic Background Extraction tool which will, with a bit of effort, sort out the vast majority of gradient problems.

Ah yes, good point about the boxcar or debayering filter, Martin. I was overlooking that one.

Olly

[Gina]

I'm thinking of getting a Gerd Neumann EL panel for my Evostar 80 ED Pro for taking Flats. The dew shield on this is 115mm diameter so I presume a 100mm EL panel wouldn't be big enough and I should go for the next size up which is 160mm. Could someone either confirm or deny this please?

[MartinB]

Have a look at the Earlsmann panels. Easy to make up your own. I sandwiched mine between a couple of sheets of polar white perspex. Important to strengthen the cable where it joins the panel with plenty of duct tape. Quite a big saving on Gerd's panels.

[russellhq]

What about some of this stuff once it becomes available:

http://www.bbc.co.uk/news/science-environment-20553143

[Russe]

How about gradient removal on mono CCD images - is that a must before stretching the image?