AstroPixelProcessor APP - flat-field calibration?

[BrendanC]

I'm getting a message I don't understand when calibrating frames:

CRITICAL WARNING: flat-field calibration can not be performed correctly!

To be able to perform accurate flat-field calibration, the provided flats need to be calibrated by either

- a MasterBias, of the same ISO/gain value as the flat frame, or
- a MasterDarkFlat of the same ISO/gain value and the same exposure time as the flat frame, or
- both

My lights and flats are definitely at the same ISO and exposure time.

However, I'm using a master bias, master dark and master dark flat from my library, for the ISO setting at which I took my lights and flats. My understanding is (or was!) that the bias should be taken at the shortest possible exposure; and that the dark and dark flat should be taken using my EOS1000D's AV setting (which Astrophotography Tool's plans actually suggest), which means auto-exposure.

So, my master bias, dark and dark flat will never have the same exposure time as my lights and flats. So, I don't understand why APP is telling me they should be, for the master dark flat. I've tried just using the master bias, and I get the same error.

Deep Sky Stacker doesn't alert me to this, so I'm not sure why APP is (which I'm trialling, which much frustration and scratching of head).

Please could someone suggest what's going wrong here? I'm probably doing something really dumb but I don't understand what.

Edited August 21, 2020 by BrendanC

[Kinch]

Just a question.....were the Masters made with different software?

If not then you can do one of two things

1) If you are 100% sure the calibration frames are OK - just ignore the warning....it will still produce a final image.

2) Since APP works through it all reasonably quickly.....I would put all the calibration subs back in at the start of process and let APP do its thing. It will make new Masters which you can keep & re-use. (I have gone this route on getting a similar warning).

[BrendanC]

They were made in DSS and I've since read that masters produced in DSS don't get along well with APP. 

Interesting that you got a similar message. 

I guess I'll just keep trying with APP and if I don't get anywhere I'll continue with DSS. It's always worth checking new things out from time to time though. 

Thanks for the help.

[Kinch]

I am not familiar with DSS (by that I mean that I don't use it). I do use APP all the time to do all my preprocessing and I find it great.  

[tooth_dr]

Your dark flat should be the same exposure as the flat.  You cant take darks or dark flats using the AV setting.  This is for the flats only.  The darks and dark flats are taken with the lens cap on and the exposure set to match lights (for darks) and flats (for dark flats)

[BrendanC]

Interesting - because APT's flats plan requires the AV setting, which I assumed meant dark flats as well as lights.

In fact, a while ago, on this very forum, I asked about calibration files, and asked if this made sense, assuming I was shooting at ISO800 for 30 second exposures, and the consensus was that broadly speaking, it was right (I tried looking for the original post but can't find it):

	Frames	ISO	Exp
Bias	50	800	1/4000
Darks	50	800	0:00:30
Dark flats	25	800	AV
Flats	25	800	AV

As AstroBackyard says: "Luckily for DSLR astrophotography shooters, there is a mode on the camera that is ideal for shooting flats. This mode is called “AV“, or Aperture Priority mode. This means that the camera will decide on the correct shutter speed to properly expose the image." - see https://astrobackyard.com/how-to-take-flat-frames/

So, is my summary table wrong? If so, how would I correct it? Because it has rather major implications for, well, everything I've done really. Seriously, if you can suggest a correction, I'd very much appreciate it. Is it a case, for example, of not relying on AV, being more mindful of the histogram, and ensuring that my flats and dark flats have the same exposure time?

I have to say, calibration files drive me nuts. I've never been able to nail down exactly what's required, after literally months of research.

Edited August 21, 2020 by BrendanC

[tooth_dr]

20 minutes ago, BrendanC said:

Interesting - because APT's flats plan requires the AV setting, which I assumed meant dark flats as well as lights.

In fact, a while ago, on this very forum, I asked about calibration files, and asked if this made sense, assuming I was shooting at ISO800 for 30 second exposures, and the consensus was that broadly speaking, it was right (I tried looking for the original post but can't find it):

	Frames	ISO	Exp
Bias	50	800	1/4000
Darks	50	800	0:00:30
Flat darks	25	800	AV
Flats	25	800	AV

Also, AstroBackyard says: "Luckily for DSLR astrophotography shooters, there is a mode on the camera that is ideal for shooting flats. This mode is called “AV“, or Aperture Priority mode. This means that the camera will decide on the correct shutter speed to properly expose the image." - see https://astrobackyard.com/how-to-take-flat-frames/

So, is my summary table wrong? If so, how would I correct it? Because it has rather major implications for, well, everything I've done really.

I have to say, calibration files drive me nuts. I've never been able to nail down exactly what's required, after literally months of research.

 

Flats - these are done with AV mode using your DSLR as you have correctly identified.   Dark flats are done with the lens cap on.  So whatever exposure time you use for your flats will be the length of time you need for your dark flats.   I dont know how this would work if you tried AV mode.  The point of dark flats is to calibrate your flats, so they need to be same exposure time and temperature.

Your flats with be very short (normally) so you could just not bother using dark flats, or maybe use a bias as a dark flat to calibrate your flat.  I've seen me using 15-20s flats so I would tend to use dark flats.

The only think wrong with that tablei s the AV setting for dark flats

 

 

[BrendanC]

I've been doing (even more!) research and according to this site, I should be using Manual mode: 'Set your camera to MANUAL mode, just like you would with light frames.' - see https://photographingspace.com/how-to-create-dslr-and-ccd-flat-frames-for-astrophotography/

So, you can see how confusing this all is.

Really appreciate that you've identified just the AV setting for dark flats being incorrect. So, is the correction that I should see what the AV exposure time was for the flats, and then use that time for the dark flats? For example, the last set of flats I took, timed at 1/100s. So, that implies I need to be in manual mode for my dark flats, set at 1/100s. Right?

In which case, the table now looks like this (apologies for formatting but despite having hard-coded html back in the day I cannot figure out how to make it work):

	Frames	ISO	Exp
Bias	50	800	1/4000
Darks	50	800	0:00:30
Dark flats	25	800	Manual, at  whatever AV mode chooses for flats
Flats	25	800	AV

Oh, and one more question that has always bugged me: do I even need bias frames if I have dark flats (and the other way around: do I need dark flats if I have bias)? I've read a whole ton of stuff about this too and never seem able to get a clear answer!

Edited August 21, 2020 by BrendanC

[BrendanC]

I think I just found the answer: using all calibration frames is the most accurate method -  http://www.covingtoninnovations.com/dslr/newdslr/#Calibration

Seeing as I already have a full Bias library, I may as well use it. 

OK, so I learned something new today (again) which may hopefully have helped me along the way.