The trouble with calibration & flats- joy of joy

[pyrasanth]

I use a flat panel for calibration. The first problem is that it was to bright for my Celestron RASA & Atik 11000 camera. I had to originally shoot UVCUT flats at 1/1000 of a second which was no good as shutter artifacts appeared on the sub. I fitted an ND 2.1 filter to the panel  & can now push exposures into the above 10 second range which the Atik 11000 appreciates more.

The problem is that I get a nice flat but it does not correct one of the corners of the light frame. This suggests that the flat is to bright (it averages 24,000 ADU)

I can correct the image using software tools like gradient exterminator but I prefer to try & use calibration first to get the image reasonable. I am using dark frames & bias correction as well.

My concern is that if I lower the exposure this could push the 3 corners that are not so affected into an affected state or should I not worry as much & see what happens.

Any advice would be appreciated- I mean shooting new flats is not going to make the telescope explode but its annoying that only one corner of the light is affected after calibration. There are lots of different opinions as to what the best ADU to shoot at is which does not help either.

[davew]

The advice in the old days used to be 50% of full well, or thereabouts, for flats in most cases. In the case of the 11002 the full well is around 60000 so if you aim for a start figure of 30000 and work from there. Way too low or high will affect your flats but somewhere in between should work.

I think the reason there are loads of different opinions is that the answers given are not normally for the chips asked about ie. why would an KAF 8300 chip be the same as an KAI 11002 chip or even Sony chip ? The recommended ADU value for a KAF chip is in fact around 70% full well. No idea what's recommended for the Kodak interline as I've never used one.

Speaking of which, why has your camera got a shutter ? Have you fitted one for taking darks ? 

How do you take these flats ? Is the panel close up to the back of the camera ? If so that may cause shadow spread. 

24000 doesn't sound too high but just keep trying different levels. You don't need to spend all night on it but just do a few of each setting that you chose, calibrate them with a Bias master ( Are you doing that ? ) and then you'll know what level to do. Then take loads so you'll keep the SNR high.

Dave.

[pyrasanth]

The advice in the old days used to be 50% of full well, or thereabouts, for flats in most cases. In the case of the 11002 the full well is around 60000 so if you aim for a start figure of 30000 and work from there. Way too low or high will affect your flats but somewhere in between should work.

I think the reason there are loads of different opinions is that the answers given are not normally for the chips asked about ie. why would an KAF 8300 chip be the same as an KAI 11002 chip or even Sony chip ? The recommended ADU value for a KAF chip is in fact around 70% full well. No idea what's recommended for the Kodak interline as I've never used one.

Speaking of which, why has your camera got a shutter ? Have you fitted one for taking darks ? 

How do you take these flats ? Is the panel close up to the back of the camera ? If so that may cause shadow spread. 

24000 doesn't sound too high but just keep trying different levels. You don't need to spend all night on it but just do a few of each setting that you chose, calibrate them with a Bias master ( Are you doing that ? ) and then you'll know what level to do. Then take loads so you'll keep the SNR high.

Dave.

Hi Dave,

Thanks for the advice.

I should have explained more about the shutter. This is the electronic interline shutter & it registers as a line on any exposure less than a second hence the need to use an ND filter inside the Gerd Neumann flat panel to significantly increase the required exposure to about 10 seconds when shooting L frames (UVCUT). Without the ND filter the CCD saturates at exposures of 1/1000 which is the minimum possible. However possible does not mean useful- I wish Atik had been a bit more forth coming in that respect as it could have saved me a lot of time!

The panel is simply placed on top of the dew shield so would be about 18" above the camera with the OTA pointing skyward.

The calibration error I'm getting is a dark triangle in the right hand lower corner of the light after calibration. I looked up the error & it is suggested that the calibration frame is over exposed which seems odd as it is under the 50% saturation suggested but I'm going to experiment with a few exposures bracketed to see if exposure changes the error on the sub.

I am using a  dark master set to scale in PIxInsight which consists of 23 dark frames of 1 hour duration each stacked to produce the master as well as a BIAS master of 100 frames stacked.

I'm not sure at this stage if the error is an optical aberration coming from the filter assembly which is a Starizona filter slide which could potentially be a source of vignetting but I would have expected this to be more symmetrical in presentation of the error & not confined to one corner- certainly the optics are not skewed.- only a bit more testing will tell. 

I will let you know what I deduce but I get a feeling that the error might not be so prevalent in either the RGB or narrow band calibration frames.

[Freddie]

Why are your dark frames 1 hour duration each when your lights are only a couple of mins each? They should each be the same duration.

[pyrasanth]

Why are your dark frames 1 hour duration each when your lights are only a couple of mins each? They should each be the same duration.

Hi Freddie,

It is standard practice to produce darks at least 5 times longer than the maximum exposure you would normally use. The darks are then auto-scaled with the calibration frames to produce the correct weighting. This effectively means you have darks which cover all your required exposures & don't need to worry remaking them as much.

[davew]

I'm amazed by two things now Pyrasanth. One is that I wasn't aware of Interline chips causing lines on short exposures. Is this an Atik or 11002 thing ? I saw two lines in your M42 image but assumed they were because the flat was produced by the camera before it was repaired ? I lead a sheltered life

The second thing is that not only do I think it's not standard practice to do longer darks, it's thought to be second best at least. If you want darks to work as they're supposed to, then do them at the same time and temperature as the lights. If you haven't got any of the correct length then you can scale longer ( Not shorter ) darks and hope they work. Can you link to some of the articles you've read. I'd be interested in seeing what's been said.

You aren't alone in taking flats with a RASA. I know there are a few people out there that have used the scope. I also assume that the Hyperstar system has a very similar problem. Staryzona's only published method for the Hyperstar is to do dusk / dawn flats. I know that some have had success with either a projector screen or white board. However, I think you could well get away with an 18" gap but I don't know. out of curiosity I looked at taking flats on as many sites as I could for you but could find almost nothing on the RASA and living nightmares, for some, with the Hyperstar. If Greg Parker is still a member here perhaps try PMing him and asking what he did.

Dave.

[Freddie]

Hi Freddie,

It is standard practice to produce darks at least 5 times longer than the maximum exposure you would normally use. The darks are then auto-scaled with the calibration frames to produce the correct weighting. This effectively means you have darks which cover all your required exposures & don't need to worry remaking them as much.

Thanks, that's a new one on me. I assume that's a PI thing.

[opticalpath]

Hi Freddie,

It is standard practice to produce darks at least 5 times longer than the maximum exposure you would normally use. The darks are then auto-scaled with the calibration frames to produce the correct weighting. This effectively means you have darks which cover all your required exposures & don't need to worry remaking them as much.

I have used scaled dark frames before (down, not up) but I always seemed to get better results with exactly matching exposure darks.  If you are using very long exposure master darks for scaling, watch out for saturated hot pixels .... which don't scale well.

I find that I have settled on just a few different exposure lengths that are optimal for my setup and, by and large, I stick to those, so it's not a hardship to build a set of exactly matching darks.

On a side note, if your EL panel runs on a 12V DC adapter, experiment with reducing the voltage.  I find that I can go down to 4.5 or 6V and get a much reduced brightness so that longer exposure flats are possible.

Adrian

[pyrasanth]

I'm amazed by two things now Pyrasanth. One is that I wasn't aware of Interline chips causing lines on short exposures. Is this an Atik or 11002 thing ? I saw two lines in your M42 image but assumed they were because the flat was produced by the camera before it was repaired ? I lead a sheltered life

The second thing is that not only do I think it's not standard practice to do longer darks, it's thought to be second best at least. If you want darks to work as they're supposed to, then do them at the same time and temperature as the lights. If you haven't got any of the correct length then you can scale longer ( Not shorter ) darks and hope they work. Can you link to some of the articles you've read. I'd be interested in seeing what's been said.

You aren't alone in taking flats with a RASA. I know there are a few people out there that have used the scope. I also assume that the Hyperstar system has a very similar problem. Staryzona's only published method for the Hyperstar is to do dusk / dawn flats. I know that some have had success with either a projector screen or white board. However, I think you could well get away with an 18" gap but I don't know. out of curiosity I looked at taking flats on as many sites as I could for you but could find almost nothing on the RASA and living nightmares, for some, with the Hyperstar. If Greg Parker is still a member here perhaps try PMing him and asking what he did.

Dave.

Hi Dave,

Here is the PixInsight link that I followed http://www.pixinsight.com/tutorials/master-frames/index.html

There are a lot of references to dark scaling on the net so I don't think the concept is in any way alien.

I was also surprised to learn of the shutter artifact on the Atik. I don't think it is the Atik but  just a characteristic of the chip- it does not like exposures shorter than 1 second or you get the interline shutter artifact.

GradientXterminator completely removes the right hand edge gradient so not too much of an issue. I will see if shooting the darks at the same exposure length makes a difference to the finished resut.

[pyrasanth]

I have used scaled dark frames before (down, not up) but I always seemed to get better results with exactly matching exposure darks.  If you are using very long exposure master darks for scaling, watch out for saturated hot pixels .... which don't scale well.

I find that I have settled on just a few different exposure lengths that are optimal for my setup and, by and large, I stick to those, so it's not a hardship to build a set of exactly matching darks.

On a side note, if your EL panel runs on a 12V DC adapter, experiment with reducing the voltage.  I find that I can go down to 4.5 or 6V and get a much reduced brightness so that longer exposure flats are possible.

Adrian

Hi Adrian,

I've not lowered the voltage on the panel. A 2.1 ND filter combination seems to work okay inside the panel. How do you drop the voltage on the 12V supply? so I can experiment.

The RASA is a pain for flats. It is full of varying intensity regions like sets of concentric rings. I think the basic Celestron adapters introduce vignetting. I might get a precision parts adapter to see if that makes a difference but at over £100 to get potentially the same result is worthy of some thought. I'm not sure if the Celestron adapter walls being conical tapered are having an effect vs just a straight pipe type adapter. I can't get my head around if there would be any difference between a conical section & a straight pipe- the clever forum guys might be able to answer that question without me spending £100.00 to find out.

[ollypenrice]

Phew, lots of new stuff here for me. I have used two Atik 11000s for quite some time and have never had any lines when taking flats at below a second. I think I only go over a second for the 3 nan Astrodon Ha filter. I'd worry that, at 10 seconds, I might be in the territory of shooting dedicated darks for flats rather than just using a master bias. As for the complexities and wonders of Pixinsight stacking, I glaze over. If their genius for communication matched their genius for being geniuses I might be tempted to get involved, but as I don't have a problem I'm not trying to solve it.

I think the problem is optical. You use a light panel on a very fast system. The panel is perched on top of a dew sheild. The dewsheild is more or less reflective. You have an overhanging central obstruction (the camera). The panel is not at infinity. Just where, exactly, is the light going? How is it reflecting? Where are the shadows? Would you expect this arrangeent to give you a flat incident beam?  Personally, I wouldn't. I've always found reflectors to be a hassle with flats, as has my collaborator Yves, and I suspect sky flats might be best with the RASA. Just a guess.

Olly

[pyrasanth]

Phew, lots of new stuff here for me. I have used two Atik 11000s for quite some time and have never had any lines when taking flats at below a second. I think I only go over a second for the 3 nan Astrodon Ha filter. I'd worry that, at 10 seconds, I might be in the territory of shooting dedicated darks for flats rather than just using a master bias. As for the complexities and wonders of Pixinsight stacking, I glaze over. If their genius for communication matched their genius for being geniuses I might be tempted to get involved, but as I don't have a problem I'm not trying to solve it.

I think the problem is optical. You use a light panel on a very fast system. The panel is perched on top of a dew sheild. The dewsheild is more or less reflective. You have an overhanging central obstruction (the camera). The panel is not at infinity. Just where, exactly, is the light going? How is it reflecting? Where are the shadows? Would you expect this arrangeent to give you a flat incident beam?  Personally, I wouldn't. I've always found reflectors to be a hassle with flats, as has my collaborator Yves, and I suspect sky flats might be best with the RASA. Just a guess.

Olly

Hi Olly,

Could to see you entering into the discussion. I wondered if the interline shutter showing up is because of the sheer amount of light being thrown off an F2.2 system. I doubt if I would get the same result on the Edge at F 6.3 (cant test as the Edge is in America at present being modified).

I tend to agree that the issue with the flats is optical. I get close with the panel but not as good as I'd like. I might try some good old tee shirt flats & see if that makes a difference- need to find an old pillow case or similar & give that a try. What is good is that I don't get many dust motes showing up so the tee shirt flats might be a good option but alas everyday it has rained so not good for outdoor flats.

[ollypenrice]

Yes, I think sky flats might be the way. Have you asked Harry how he does them? He'd be happy to tell you, I'm sure.

Flats can be a royal pain. One of the cameras here, not an Atik, seems to give entirely unpredictable results with the brightness values changing, as it seems to me, randomly. I'm struggling with it at the moment. It has to be the camera because the problem only affects one half of the tandem rig. The Atik half is fine.

Olly

[davew]

Hi Dave,

Here is the PixInsight link that I followed http://www.pixinsight.com/tutorials/master-frames/index.html

There are a lot of references to dark scaling on the net so I don't think the concept is in any way alien.

I was also surprised to learn of the shutter artifact on the Atik. I don't think it is the Atik but  just a characteristic of the chip- it does not like exposures shorter than 1 second or you get the interline shutter artifact.

GradientXterminator completely removes the right hand edge gradient so not too much of an issue. I will see if shooting the darks at the same exposure length makes a difference to the finished resut.

Sorry. Didn't make myself clear ( Once again ! ). While I know scaling or optimisation isn't alien, I mean, even DSS does it if necessary, I've never heard of doing darks 5 x the length of your normal lights. Given time I'm sure I'll find it so no bother. There are caveats to doing scaling but I'm sure they won't apply in your case

Just glad someone has turned up to sort your problems out.

Dave 

[opticalpath]

Hi Adrian,

I've not lowered the voltage on the panel. A 2.1 ND filter combination seems to work okay inside the panel. How do you drop the voltage on the 12V supply? so I can experiment.

The inverter that comes with the G. Neumann EL panels can be either 12 Volt DC supply or 110/ 220 Volt mains AC supply, depending on which version you ordered.  Reducing the voltage as I suggested is only practical if you have the inverter that is intended to run off your own 12 Volt supply; then it's simply a matter of powering it from an adapter that has switchable voltages such as 3/ 4.5/ 6/ 9/ 12V and experimenting with less than 12V input.  Although I have mains power in my obsy. I chose the 12V version specifically for this reason.

Adrian

[harry page]

Hi

There are lots of problems with flats and mirror systems / ccd cameras

Starting with my flat panel , its a home made jobbie that I originally made for my 14" newt and is about 600mm from the end of my scopes , I did find it to bright 

for the RASA and was worried about reflections , so a white sheet over the end diminishes the light  just fine and gives a very even illumination.

I work with about 1 -2 second exposure's ( sx h35 ) and agree very short exposures can cause problems

 I had written into the ccd firmware drivers and software   , multiple flushes to remove any excess charge between subs  and the start of a sub  , I do not know what is available with your camera.( the ascom driver I use

has user selectable flushes)   , I do not know what is available with your camera.

also as olly and me found out very small differences in the bias / darks do make a difference to flat correction , I would advise that you take correct matching darks and or try no darks at all ( bias only )

Perhaps because I am used to these mirror things , I have had no problems with flat correction.

Also please remember that most ccd cameras  have ABG and therefore start being non linear after about 50% of full well depth , hence the recommendation of about 50% for flats , but not average 50%   ---max 50%

I expsose about max 25000 adu , but this is not a universal number as each camera / chip has its own characteristics

Regards

Harry

[ollypenrice]

Great post, Harry. Thanks. Aha, now this multiple flush business might explain why flats are such a pain with the SXVH36. This one is running in Nebulosity on a Mac with standard drivers.

Olly

[harry page]

Great post, Harry. Thanks. Aha, now this multiple flush business might explain why flats are such a pain with the SXVH36. This one is running in Nebulosity on a Mac with standard drivers.

Olly

Hi Olly

I also got Fabio in astroart to add multiple flushes , and I forgot to say there are flushes before downloads as well this help stop any bleading from warm / hot pixels

Regards

Harry

[pyrasanth]

I have starting producing a new set of dark frames to stack from 1 to 10 minutes- 30 frames per set to produce a new set of dark masters. I'm not convinced that the scaling in PixInsight is the way to go. I tried a 3 minute dark & it gave a significant improvement over the scaled dark for the same exposure length stack. I will now stop being a smart a*** and use darks that match my subs.

I have also removed the shiny metal frame (a cake tin frame) that was being used to reinforce the end of my dew shield. I forgot that not painting this black would throw concentric rings around the CCD & on a bright night last night this is exactly what it did. The results looked so bad one could be forgiven for putting the whole telescope into the dustbin.

I now need to shoot a new set of flats- not sure If I should go panel or sky flat. If I go sky I might need an ND moon filter in the optical path as the image will be to bright without some form of attenuation. I don't know how well a piece of cloth attenuates the light.

Thanks Harry for the advice. I am milling over what I need to do. The ND filters are quite expensive at £42.00 each & I'm not sure if I need the 1.8 or 3 or a combination of some other values to reach a sensible exposure on the RASA.                                     

[ollypenrice]

It isn't hard to bring down the light levels using white fabric.  A piece of translucent perspex over the top helps keep fabric flat. If you use black paint to darken surfaces against reflection be sure to use barbecue or stove paints, which use pigments rather than dyes. Dyes, it transpires, can be reflective in IR and mess up flats. 

Olly