Jump to content

Stargazers Lounge Uses Cookies

Like most websites, SGL uses cookies in order to deliver a secure, personalised service, to provide social media functions and to analyse our traffic. Continued use of SGL indicates your acceptance of our cookie policy.

Stub Mandrel

Help! Problem With Flat Frames

Recommended Posts

I'm trying to get my 'workflow' sorted out with some widefield shots before trying the DSLR on the scope or with a long lens.

For my first pictures I didn't use a flat and it was suggested that this would improve my results. For my second attempt i took ten flat frames and put them into DSS along with the lights, bias/offset and darks. DSS gave me three 'master' frames as a result.

The bias frame is pretty uniform, the dark frame highlights a good sprinkling of hot/warm pixels, which is what I expected.

The problem is the 'flat' frame. It looks how I would would expect, with slight vignetting at the corners:

MasterFlat ISO100

The problem is that instead of making the image more even, DSS seems to vastly over-compensate and burn out the corners. This is the result without the flat frame

hercules without flat

Now with the flat:

hercules with flat

What am I doing wrong? I am not doing ANY pre-processing of the RAW images before using DSS, and have not done anything to the master flat (except make a jpeg for posting it here).

Share this post


Link to post
Share on other sites
Hi,


I know DSS is a great program and widely used but I confess I do not it any more (having moved on to Nebulosity) since I felt it was rejecting subframes from the stack unnecessarily.


I wonder, are you using the same ISO setting for the flats as your lights? Also, is the (luminance) peak of your flats histogram (in camera) around the 50% mark? I think I found too low or too high (clipped in either R, G or B) gave inferior results.


Cheers


John

Share this post


Link to post
Share on other sites

The bias, lights and flats should have the same ISO - do they? If they do, I would leave out the darks and try again.

NigelM

Share this post


Link to post
Share on other sites

Ah... the flats were at 100 ISO, the bias at 800 ISO and lights and darks 3200 :eek:

More care needed ...

I also tried to take a flat at ISO 800 with auto exposure, oddly the frames came out in a range of different degrees of grey. It stacks better but shows a huge expanse of greeny light pollution at bottom right. That's in the 'right place' for LP and a differnet corner to where it appears for Cygnus, so I think I can be sure my problem is LP although I need to do better on the flats etc.

On the positive side, after looking for messier objects on my wide field pics and failing I can see M13 as a conspicuous blob that seems to be to large in proportion to its brightness compared to the stars. That's 3 down 107 to go :-)

Edited by Stub Mandrel

Share this post


Link to post
Share on other sites

Also someone, who I greatly respect their knowledge in these matters, pointed out you don't need bias frames in DSS if you are using Darks......I know I added bias frames and the results were definately worse off.

Remember the Darks should have same exposure lengths as well as ISO settings as your Lights.

Share this post


Link to post
Share on other sites

I really should pay attention. That's two people who said abandon bias, at least I didn't wait for a third.

It does seem to help, although I notice that after brutal stretching the historgram for this only has about 25-30 levels left:

hercules 6

Share this post


Link to post
Share on other sites

This over correction appears when 'darks for flats' are not applied to the flats or are wrong in some way. You do not need dedicated darks for your flats, however. A master bias or stack of bias will work perfectly well as a dark for flats. If using DSS you could put your set of bias or master bias in the darks for flat frames section rather than the bias section. Then DSS will just use them as darks for flats, which is what you want. I imagine that bias (darks for flats), with a DSLR, need to be at the same ISO as the flats they are going to calibrate.

Olly

  • Like 1

Share this post


Link to post
Share on other sites
That's two people who said abandon bias, at least I didn't wait for a third.

That's the wrong way round. You can abandon darks and DSS will work just fine, but if you abandon the bias, you will end up using the darks for the flats, which is incorrect (although if you have very low dark signal then you might not notice).

NigelM

Share this post


Link to post
Share on other sites

Lose DSS and buy the reasonably priced AstroArt which is deliciously transparent and makes it all so easy. I'm not on commisission, by the way, more's the pity!

I tried DSS but felt that life was too short.

Olly

http://ollypenrice.smugmug.com/Other/Best-of-Les-Granges/22435624_WLMPTM#!i=2266922474&k=Sc3kgzc

Share this post


Link to post
Share on other sites

Olly

I wouldn't dare take issue with you - I know how much more than me you know - but Astroart easy? I downloaded version 5 a little while ago and can't even get it to load two files at the same time, never mind make it stack them. Is there a secret handshake I'm missing?

I am not against spending money (you should see the purchases since we got back from Les Granges) and would, especially if I could get rid of DSS's white balancing, whether I want it or no (I don't because I use StarTools for post processing).

But seriously, is there an idiot's guide somewhere that doesn't involve a 15 minute tutorial on a subject I don't really need?

Share this post


Link to post
Share on other sites

Also someone, who I greatly respect their knowledge in these matters, pointed out you don't need bias frames in DSS if you are using Darks......I know I added bias frames and the results were definately worse off.

Remember the Darks should have same exposure lengths as well as ISO settings as your Lights.

I doubt very much if this statement is correct. If there is no Bias then the Flats can not be calibrated, nor the lights if there is no Dark frame either. The only way that Bias would  not be required is if the Flats are calibrated by Dark Flats and as the Dark Flats already contain Bias then an extra set of Bias is not required, however in this case you would have to take Darks to calibrate the Lights with. Bias is the most important of the calibration frames and the easiest to take . It is a lot easier and quicker to take a large number of Bias frames rather than Darks that in the case of a DSLR are almost always a mismatch or Dark Flats. Dark Flats are only really required if the duration of the Flat frames exposure is long enough to induce noise. This mostly happens if Flats are taken the good old fashion way by covering the scope with a uniform white material and pointing it to the twilight sky as it is supposed to be " flat ". A master Bias made up of about 200 Bias frames is good enough not to induce noise. As for the Flats something like 50 frames is sufficient but Flats have to be absolutely correct to work.

A.G

Edited by lensman57

Share this post


Link to post
Share on other sites

I have looked at all sorts of websites for advice, including several threads on this forum.

The only thing I can be sure of is that no two sources agree completely  and all disagree on at least two or three aspects of the process! I've checked out my master control frames from DSS, stretching their histograms and zooming in to 1600%.

FLATS

I can see that if vignetting is an issue flats become essential. Having stretched mine so the histogram goes from 0-100% I must have one of the cleanest sensors on the planet :-) one vaguely dark patch and one slight light patch, no vignetting at all (but this is a 0.625 of full frame sensor with a full frame 28mm lens so I wouldn't expect any).

My original problem stands - how is such an even 'flat frame' causing vignetting in the final pictures? Surely I must be doing something wrong?

BIAS

This is fascinating - unstretched, one of the colours in the Bayer matrix is slightly lighter than the other three pixels. Two (narrow, gaussian) peaks on the histogram at 6 and 16 out of 255. I can't see what would cause this unless the camera itself is boosting red or blue? There's one, solitary 'hot' pixel that looks to be at about 40-50 out of 255, but as its one pixel it doesn't show on the histogram!

DARKS

About 150 very bright pixels, these, I guess, are the 'hot' ones. Rather more 'warm' pixels, but the bayer pattern remains distinct, indeed the histogram looks almost identical to the bias one, which suggests that the sensor isn't a bad one. No sign of 'amplifier glow' no matter how hard I push the histogram

In the absence of any experience... my conclusion is that at my beginner's level and with my equipment, what is going to matter for me most are the dark frames, the flats and bias frames even when stretched drastically are not showing any significant artefacts. The thing that has surprised me is how clean the sensor appears to be - it isn't perfect, but I'm sure it is good.

I'm not pretending that as I get better I won't need to get more pernickity and critical and start worrying more about these, but for now I'll just try an build a small library of flats at different temperatures and set ups and a good master bias frame.

Edited by Stub Mandrel
  • Like 1

Share this post


Link to post
Share on other sites

" DARKS

About 150 very bright pixels, these, I guess, are the 'hot' ones. Rather more 'warm' pixels, but the bayer pattern remains distinct, indeed the histogram looks almost identical to the bias one, which suggests that the sensor isn't a bad one. No sign of 'amplifier glow' no matter how hard I push the histogram."

You seem to be fixated on the Darks. These are the least important of the calibration frames and I say again that with any imaging device without set point cooling Darks are irrelevant.

Darks are highly temperature dependant and your camera as fine as it maybe has no set point cooling so the darks taken and applied at any stage unless are a match in temperature to the Lights,  will cause all sorts of problems. If hot pixels are your problem the best way of dealing with them is to use High Dithering during capture and applying  Cosmetic Correction and Sigma Clipping during the stacking stage this will get rid of a lot of other nasties that your stretching may not show. DSS is capable of doing this but needs a minimum of  16 subs for it to work properly and you also need to be careful about setting the Threshold level for cosmetic correction so it wouldn't over correct.

As for calibration procedure and unless the mathematics have somehow changed the procedure starts with Bias then Master Bias and then the Flats are Master Bias subtracted and then it goes on. I suggest that if you have not done so already do some reading about the Stacking.

Regards,

A.G

Share this post


Link to post
Share on other sites

> I suggest that if you have not done so already do some reading about the Stacking.

I've done lots! Probably too much and I'm worrying abouyt detail instead of getting some experience. It's this damn cloud...

I'm grateful for all advice, but some of what you say - "without set point cooling Darks are irrelevant" - is totally at odds with what I am reading. If that's the case why do so many people who use DSLRs using darks? Is this not the point of taking the darks at the end of the imaging session? I get the feeling you may think I'm using a CCD camera not an out-of-the-ark DSLR?

 I've seen figures as low as five and as high as fifty or more for numbers of control frames - and all points in between. there is so much information out there, and everyone who has found amethod that works for them seems to think its the 'definitive' answer.

My approach is to try and learn by attacking my most obvious mistakes first - for me that appears to be teh question 'why do my flats cause dark vignetting', but I can't find an answer - I'm sure just being at the wrong ISO can't be the cause.

Share this post


Link to post
Share on other sites

Although Lensman is much more experienced, I've seen some improvement using darks within a range of temps. Not sure yet how broad that range is. I've heard 5c. A CCD and a DSLR are 2 different beasts. Without cooling, and at the same ambient, a DSLR will beat the CCD for noise.

As flats are difficult to get right, if LP is low and no vignetting, I believe I've seen as many ills from flats as cures. So if one doesn't use flats, no need for bias as darks include that data. I don't have the full Pixinsight, but mine has DBE and it does wonders for gradients.

In this subject area I think for a DSLR , dithering is a big benefit. Dithering for a DSLR needs to be large, on the order of 15 pixels to 2-3 star diameters.

Share this post


Link to post
Share on other sites

Flats are essential for a good image. If I were you I'd get flats nailed and come back to the darks and bias at a later date (you can't come back to flats).

At first it's easiest to take your flats immediately after taking your lights. Focus, ISO and camera orientation on the scope need to be the identical to your light frames otherwise they will cause problems like what you're seeing.

Temperature is not particularly important for flats unless it's vastly different causing a large focus shift.

Edited by wuthton

Share this post


Link to post
Share on other sites

how are you taking your flats by the way ?  I notice that you're shooting a widefield, so are you holding a laptop screen or such right up to the camera lens, or some other way ?

In your first post, that example of the flat frame, is that stretched to show the vignetting, or is that as-is ?  My flats when viewed without stretching look like pretty boring grey frames, and only once you stretch them can you see the vignetting and dust spots. 

Just wondering if the way you're taking them is introducing extra vignetting that isn't there in the lights, for example if you had the light source some way off, then the edges of the field of view as seen by the lens may be seeing less illumination or something ?

Maybe if you can post up dropbox links to your master flat, master bias, master dark and one of the light frames, then we can see how it calibrates in different programs ?

Share this post


Link to post
Share on other sites

I thi8nkl you may well be right that I am introducing more problems than I'm solving.

Maybe if you can post up dropbox links to your master flat, master bias, master dark and one of the light frames, then we can see how it calibrates in different programs ?

I think I've made too may mistakes with this set. When teh clouds break and I get a decent couple of hours, I will try and get the best set I can and do as you suggest.

Share this post


Link to post
Share on other sites

Also someone, who I greatly respect their knowledge in these matters, pointed out you don't need bias frames in DSS if you are using Darks......I know I added bias frames and the results were definately worse off.

Remember the Darks should have same exposure lengths as well as ISO settings as your Lights.

cough cough - I stand corrected I think !!  It is definately an age thing, in the above statement  replace "Bias" with "Dark Flats" - but ............ I am prepared to be shot down again.

I do use the T shirt technique to get my flats.

 I used my fridge to do my DSLR dark library (various exposure times and ISO settings)  - at +5C they should be good for +10C down to 0C and then used the ice box at 0C for another set of Darks which I found covers most of the temps I see around here at night. Using the fridge means better control and convenience as you can do this anytime and get a lot of Darks exposed.

Edited by adamsp123

Share this post


Link to post
Share on other sites

cough cough - I stand corrected I think !!  It is definately an age thing, in the above statement  replace "Bias" with "Dark Flats" - but ............ I am prepared to be shot down again.

I do use the T shirt technique to get my flats.

 I used my fridge to do my DSLR dark library (various exposure times and ISO settings)  - at +5C they should be good for +10C down to 0C and then used the ice box at 0C for another set of Darks which I found covers most of the temps I see around here at night. Using the fridge means better control and convenience as you can do this anytime and get a lot of Darks exposed.

Hi Pete,

I am so glad that you cleared the " confusion " of the OP. With regret I do not believe that a Delta T of 10 C is going to do your Dark subtraction any favours. I would strongly advise every one imaging with a DSLR to read the article in the following link. As for one our friends claiming that uncooled DSLR has less noise than an uncooled CCD I am afraid that this is not true yet again as simply the two can not be compared. The output of an Astronomical imaging CCD is linear even those with Anti Blooming Gates, at least extremely close to linear,  which simply encompasses all the cameras that are manufactured for hobby purposes and do not have a scientific grade sensor. As far back as 2012 Craig Stark ( author of PHD guiding and nebulosity ) in an article written for CN forums, discovered that at least Canon were applying on chip noise manipulation to long exposure data before these were written to file. This simply means that the data extracted from a CR2 file is not really " RAW ". I strongly believe that at least Nikon  and most probably others are following suit. The point that I am trying to make is that these cameras are not really manufactured for AP but can be used quite successfully if their limitations are kept in check but there is absolutely no point in people constantly trying to compare them to cooled CCDs and trying to challenge them. These are completely different animals to each other and can not be compared, the same goes for Mono CCD V OSC CCDs IMHO.

Regards,

A.G

http://www.mirametrics.com/tech_note_tempvar.htm

http://www.stark-labs.com/craig/resources/Articles-&-Reviews/CanonLinearity.pdf

Share this post


Link to post
Share on other sites

Hi Pete,

I am so glad that you cleared the " confusion " of the OP. With regret I do not believe that a Delta T of 10 C is going to do your Dark subtraction any favours. I would strongly advise every one imaging with a DSLR to read the article in the following link. As for one our friends claiming that uncooled DSLR has less noise than an uncooled CCD I am afraid that this is not true yet again as simply the two can not be compared. The output of an Astronomical imaging CCD is linear even those with Anti Blooming Gates, at least extremely close to linear,  which simply encompasses all the cameras that are manufactured for hobby purposes and do not have a scientific grade sensor. As far back as 2012 Craig Stark ( author of PHD guiding and nebulosity ) in an article written for CN forums, discovered that at least Canon were applying on chip noise manipulation to long exposure data before these were written to file. This simply means that the data extracted from a CR2 file is not really " RAW ". I strongly believe that at least Nikon  and most probably others are following suit. The point that I am trying to make is that these cameras are not really manufactured for AP but can be used quite successfully if their limitations are kept in check but there is absolutely no point in people constantly trying to compare them to cooled CCDs and trying to challenge them. These are completely different animals to each other and can not be compared, the same goes for Mono CCD V OSC CCDs IMHO.

Regards,

A.G

http://www.mirametrics.com/tech_note_tempvar.htm

http://www.stark-labs.com/craig/resources/Articles-&-Reviews/CanonLinearity.pdf

Thanks for making my point A.G. The charts show a geometric rise in dark current with CCDs above around 0C.

Stipulated that all DSLRs, unmoddified, fiddle with data.  Tests done by imagers do seem to show that at room temps, a CCD will be noisier than even a stock DSLR. Just because you see no point in comparing them, doesn't make the results untrue.

You stated "with any imaging device without set point cooling Darks are irrelevant." This flies in the face of the experience of many imagers, and your own statements that the devices cannot be compared.

Edited by kalasinman

Share this post


Link to post
Share on other sites

 I used my fridge to do my DSLR dark library (various exposure times and ISO settings)  - at +5C they should be good for +10C down to 0C and then used the ice box at 0C for another set of Darks which I found covers most of the temps I see around here at night. Using the fridge means better control and convenience as you can do this anytime and get a lot of Darks exposed.

my tired brain here misread that as you were taking pictures of your fridge !  Was thinking that you were using the fridge door for flats, or the lights when you open the door, or something.  dear oh dear, is it home-time yet ?

Share this post


Link to post
Share on other sites

Thanks for making my point A.G. The charts show a geometric rise in dark current with CCDs above around 0C.

Stipulated that all DSLRs, unmoddified, fiddle with data. Tests done by imagers do seem to show that at room temps, a CCD will be noisier than even a stock DSLR. Just because you see no point in comparing them, doesn't make the results untrue.

You stated "with any imaging device without set point cooling Darks are irrelevant." This flies in the face of the experience of many imagers, and your own statements that the devices cannot be compared.

As nearly all CCD cameras manufactured for AP have a cooler why would you want to use one at ambient?

Share this post


Link to post
Share on other sites

I've realised that this is all a lot more complex than I first thought.

Clearly my original ambition of making some rewardingly interesting pictures of globular clusters, nebulae and maybe the odd galaxy is a naive and foolish one!

I have realised my duty is to the cause of astrometry, and have ordered several thousand pounds worth of kit off ebay - I won't list it as I have no doubt that it will be deemed inadequate, but we all have to start somewhere.

My researches have uncovered that the three main cause of residual error, after processing a minimum of 512 darks, flats, dark flats, biased oblongs and trinagulated ovals are:

  1. Contamination by gamma rays.
  2. Stray radiation from the CERN, following the resumption of operations with the LHC.
  3. Brownian motion.

These can be addressed in the following ways:

  1. Cross correlating my results with information from http://aurorawatch.lancs.ac.uk/
  2. Close study of the CERN operating timetable https://espace.cern.ch/be-dep/BEDepartmentalDocuments/BE/LHC_Schedule_2015.pdf
  3. A complete ban on tea and warm beer and a strict 'cold lager only' rule

Unfortunately that last of these three actions is one I find compeletely unacceptable, and I fear I may have to give up on my quest to image M13, before it even starts.

Neil

Edited by Stub Mandrel
  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.