Over correcting flats, out of ideas.

[ollypenrice 28,421]

On 28/1/2016 at 18:18, opticalpath said:

Can't get rid of this box! Apologies to Opticalpath!!

Thanks to William for some excellent thoughts he gave me via PM.

Suppose the problem is this: at short exposures the camera responds non-linearly to different levels of signal, so in short, flat-length exposures it is less responsive to faint signal than to strong. This would produce flats which had too much contrast (more vignetting contrast than would be found in the full length lights.) This would produce the over-correction.

Perhaps I could find a way to measure the percentage vignetting at different exposure lengths. In fact just divide the max value by the min at different exposure lengths?

Olly

[harry page 464]

Hi

as you know I did used to have this problem

Personaly I think its down to flushing and giving time between exposures for this happen correctly

making sure my software has a double flush and leave a few seconds between exposures seems to work well for me

 

Harry 

 

 

[Oddsocks 733]

23 minutes ago, harry page said:

 

making sure my software has a double flush and leave a few seconds between exposures seems to work well for me

 

Harry 

 

 

Sounds a good idea, assume by double flush you mean program a couple of zero length (bias) exposures between each flat to fully clear the pixel wells and registers. Do you dark-flush or leave the flats lights source active?

William.

[harry page 464]

Hi

by flushing I mean that the registers have to be flushed from excess charge before exsposure and just before download , now as I have found some programs do this well already

maxim does it well without any user input , so does astro art ( after I got Fabio to write this in ) --neither of these programs AFAIK lets you adjust these settings  

I use a ASCOM driver with my sx camera with SGP  , again I was able to get bret ( ascom author) to put a option in for user selectable flushes

these type of flushes have nothing to do with light input !!!

Regards

 

Harry 

[Pompey Monkey 790]

1 hour ago, ollypenrice said:

Thanks to William for some excellent thoughts he gave me via PM.

Suppose the problem is this: at short exposures the camera responds non-linearly to different levels of signal, so in short, flat-length exposures it is less responsive to faint signal than to strong. This would produce flats which had too much contrast (more vignetting contrast than would be found in the full length lights.) This would produce the over-correction.

Perhaps I could find a way to measure the percentage vignetting at different exposure lengths. In fact just divide the max value by the min at different exposure lengths?

Olly

If you have eliminated the other factors like RBI (of which I admit to only having a basic understanding), to find the most linear range on your chip I would suggest the following:

Take a series of calibrated flats at set panel brightnesses and varying the exposures until you can populate a table of maximum - minimum ADU's like this:

		Panel brightness
		x	y	z	…
exposure	a	.	.	.	.
	b	.	.	.	.
	c	.	.	.	.
	…	.	.	.	.
Variation in ADUs

 

The exposures don't have to be the same for each panel brightness, look to getting values that give you "sensible" maximum ADU's. Then simply plot the variation in ADUs for the range of exposures at each panel brightness.

The curve with the least variation wins, and gives you the panel brightness that sits in a good linear part of the camera's response.

[Oddsocks 733]

I see, the flush routine is part of the software process but it is not clear if all camera control and acquisition software does this.

I have read elsewhere that programming a couple of zero length exposures between flats, lights and darks does the same thing but it was not clear if you could leave the detector exposed to light during the sequence, with a full frame CCD you can just keep the shutter closed or program the filter wheel to move to a blanked filter aperture but with an interline CCD there is no shutter so it makes it a little more awkward to have to cap the telescope for the zero time exposures.

[vlaiv 5,318]

2 hours ago, ollypenrice said:

Thanks to William for some excellent thoughts he gave me via PM.

Suppose the problem is this: at short exposures the camera responds non-linearly to different levels of signal, so in short, flat-length exposures it is less responsive to faint signal than to strong. This would produce flats which had too much contrast (more vignetting contrast than would be found in the full length lights.) This would produce the over-correction.

Perhaps I could find a way to measure the percentage vignetting at different exposure lengths. In fact just divide the max value by the min at different exposure lengths?

Olly

Easy to do, just take flats at different exposure levels (stack, calibrate and all). And then do cross correction between flats. If each flat is being perfectly corrected with any other, then nonlinearity is not an issue (for that range of exposures).

[steppenwolf 6,694]

I'm late to this thread but I have a similar problem with my QSI 683 WSG (KAF 8300 sensor) in that my Flats are over-correcting.

I have tried two different EL panels. Using exactly the same optical system and EL panels but swapping to my SXVF-M25C OSC CCD camera, my flats work perfectly.

My sensor, though, requires an external mechanical shutter.

I'd love to get to the bottom of this!

[ollypenrice 28,421]

Well, Steve, your technical knowledge is far more comprehensive than mine so if you're struggling... IIRC you're a Maxim man?

I'll try the suggestions above.

Harry, I did build in a three second delay but it didn't help. With the help of a second pair of hands I could try a couple of bias between lights. (Panel off, cap on, panel off etc. I'm expecting a couple of SGLers this afternoon. Hi Guys...

Olly

[ianaiken 389]

3 hours ago, ollypenrice said:

Thanks to William for some excellent thoughts he gave me via PM.

Suppose the problem is this: at short exposures the camera responds non-linearly to different levels of signal, so in short, flat-length exposures it is less responsive to faint signal than to strong. This would produce flats which had too much contrast (more vignetting contrast than would be found in the full length lights.) This would produce the over-correction.

Perhaps I could find a way to measure the percentage vignetting at different exposure lengths. In fact just divide the max value by the min at different exposure lengths?

Olly

I had the same problem and I would try increasing exposure time.  I resolved it by doing longer exposures with a dimmer panel (actually I now use monitor screen running white video from youtube around 80cm away from the C9).  Prior to that using a bright laptop screen up to the end of the scope with sub second exposures I was getting everything you described.  Now if I could only figure out how to take a flat for a 3nm Ha as 300s is a little too long.

[steppenwolf 6,694]

Quote

Well, Steve, your technical knowledge is far more comprehensive than mine so if you're struggling... IIRC you're a Maxim man?

Strangely, Olly, I have only recently discovered the issue and at first I thought it was the new EL panel that I use on my automated 'flat-flap' - I wasn't happy with the 'shape' of the brighter parts of the flat. I checked back on some flats that I had taken over a year earlier with another EL panel and discovered exactly the same pattern so it wasn't the panel per se.

The pattern shows a darker area to the left hand side of the frame and I did wonder if this was the mechanical shutter casting a shadow as my exposure lengths were pretty short (under a second). So, I increased exposure to about 8 seconds by placing sheets of paper between the telescope and the panel but keeping the ADU value at around 22000.

This made no difference to the the shape BUT all my experiments meant that I critically analysed my results and it was then that I discovered the over-correction!!

Is crochet easy to take up?

[ollypenrice 28,421]

1 hour ago, steppenwolf said:

 

Is crochet easy to take up?

I don't know about crochet but sometimes this game makes me think that parachutes are an over-rated accessory in base jumping...

Olly

[Starlight 1 1,368]

Have you tryed useing the Starlight camera softwear  when I said about Darks sub being a lot lighter on one side but when I use Starlight one its ok  not got round to trying flats

as camera not back from being repaired .

[harry page 464]

Hi

I am knowing that I am talking to the converted , but if you alter anything on your setup at all you must redo your calibration frames

change a cable , usb hub , software , drivers , power supply anything at all has a effect , sometimes a surprisingly large effect

Regards

Harry

[opticalpath 278]

On ‎29‎/‎01‎/‎2016 at 14:11, ollypenrice said:

 

Please ignore - there is some kind of bug in replying with Quote .... the new software is not properly flushing (sic!) older reply-quotes

Edited January 30, 2016 by opticalpath
Bug causing old abandoned qutes-replies not to clear

[opticalpath 278]

7 hours ago, harry page said:

Hi

as you know I did used to have this problem

Personaly I think its down to flushing and giving time between exposures for this happen correctly

making sure my software has a double flush and leave a few seconds between exposures seems to work well for me

 

Harry 

 

 

Harry, does this somehow apply to interline sensors too, like the KAI- sensor in Olly's camera?  I thought that they were not susceptible to RBI.  Are you describing a different effect?

Adrian

[opticalpath 278]

6 hours ago, harry page said:

Hi

by flushing I mean that the registers have to be flushed from excess charge before exsposure and just before download , now as I have found some programs do this well already

maxim does it well without any user input , so does astro art ( after I got Fabio to write this in ) --neither of these programs AFAIK lets you adjust these settings  

I use a ASCOM driver with my sx camera with SGP  , again I was able to get bret ( ascom author) to put a option in for user selectable flushes

these type of flushes have nothing to do with light input !!!

Regards

 

Harry 

So we're not talking about RBI ... which needs a bright full-saturation flash before flush to normalize the issue caused by trapped electrons?

[opticalpath 278]

4 hours ago, steppenwolf said:

I'm late to this thread but I have a similar problem with my QSI 683 WSG (KAF 8300 sensor) in that my Flats are over-correcting.

I have tried two different EL panels. Using exactly the same optical system and EL panels but swapping to my SXVF-M25C OSC CCD camera, my flats work perfectly.

My sensor, though, requires an external mechanical shutter.

I'd love to get to the bottom of this!

Steve, are you using the Maxim setting under Camera Setup/ Advanced where you can specify a normal or more aggressive flush between exposures?

Adrian

[steppenwolf 6,694]

Quote

Steve, are you using the Maxim setting under Camera Setup/ Advanced where you can specify a normal or more aggressive flush between exposures?

@opticalpath Hi Adrian, Thank you for your input. I have the following options which must have been automatically entered for me at installation time as I have to confess that I haven't visited this part of the setup until today! I Have the following options and my current setting is 'Normal' .........

Pre-Exposure Flush

None
Modest
Normal
Aggressive
Very Aggressive

I am hoping that you have good news for me

[harry page 464]

44 minutes ago, opticalpath said:

So we're not talking about RBI ... which needs a bright full-saturation flash before flush to normalize the issue caused by trapped electrons?

Hi

that's right nothing to do with RBI

 

Harry

[opticalpath 278]

28 minutes ago, steppenwolf said:

@opticalpath Hi Adrian, Thank you for your input. I have the following options which must have been automatically entered for me at installation time as I have to confess that I haven't visited this part of the setup until today! I Have the following options and my current setting is 'Normal' .........

Pre-Exposure Flush

None
Modest
Normal
Aggressive
Very Aggressive

I am hoping that you have good news for me

Well ..... tbh Steve, I have not had any particular flats problem with my older QSI 583 and I generally leave this set to 'Normal'. (in fact the earlier firmware did not even have this function setting at all)  Just thought it might be worth checking the setting and trying something more aggressive.   However ..... now I'm reading the QSI firmware info and it may be that these more aggressive settings are only supported by certain 500 series models with different sensors, while the 583 only supports Normal/ Modest/ None   It would be worth checking this on the QSI web site for your later 600 series camera, but I suspect the same may be true for it.  Sorry if I raised your hopes prematurely!

Adrian

Edited January 30, 2016 by opticalpath

[steppenwolf 6,694]

No problem, Adrian, it's early days with the problem and I WILL find a solution ......... hopefully ........ maybe .......... perhaps .......... PULL YERSELF TOGETHER MAN, OF COURSE THERE IS A SOLUTION ! I've always said that Flats are a pain - but a vital pain - and my words are coming back to haunt me!

[opticalpath 278]

On ‎30‎/‎01‎/‎2016 at 09:39, ollypenrice said:

Thanks to William for some excellent thoughts he gave me via PM.

Suppose the problem is this: at short exposures the camera responds non-linearly to different levels of signal, so in short, flat-length exposures it is less responsive to faint signal than to strong. This would produce flats which had too much contrast (more vignetting contrast than would be found in the full length lights.) This would produce the over-correction.

Perhaps I could find a way to measure the percentage vignetting at different exposure lengths. In fact just divide the max value by the min at different exposure lengths?

Olly

Olly, off topic, but referring to your comment earlier in this thread ... "Apologies ... Can't get rid of this box ..."

I've hit the same problem.  It happens if you start a reply to a post using Quote, but then abort before submitting the reply. Even if you have entered no text it appears impossible to escape and you can't delete the '3 hours ago so and so  said:' bit of header text; the system retains your unsent skeleton reply and presents it every time you try to reply to something else!  Once you start a reply-quote, you have to either submit the reply, or actively delete it entirely; otherwise it will come back and haunt you forever!  I got the answer from Grant.  If you want to abort a reply-quote, you must actively delete ALL the text by hovering over the top left corner of the text box and click on the 'move' icon that appears. This highlights an outer box that includes ALL the text, which can then be deleted.

Would be a lot simpler if you could just press ESC imo, but that's how it is.

Adrian

Edited January 31, 2016 by opticalpath

[narrowbandpaul 95]

Regarding RBI, it is not an issue for interline sensors as pointed out. Only full frame KAF type sensors.

In a nutshell, there are mini-potential wells at the epi-bulk interface which are handily placed to trap NIR photoelectrons (electrons generated by absorbing NIR light) during integration. Over time these trapped electrons will detrap just like dark current (as a result of their thermal energy). Point a KAF sensors (particulary the 16803 and 09000) at a bright star, then take a long dark....you'll see the star in your darks. This detrapping causes havoc for precise calibration of images, so the only way to deal with it is to fill all these traps prior to every exposure. Then all images start from the same condition and things calibrate nicely. The dark current is increased considerably as a result and deep cooling is essential. I have been testing a PL16803 that I have a share of and the rbi is significant, but its the only way. Some will say that a good work round is to run the sensor warmer than normal so that the traps empty faster, then continue imaging. How anyone thinks that is a hallmark of a good camera is beyond me. Its not solving the problem, its a work round that the manufacturer suggest because they cant be bothered to fix the problem. And these aren't small companies either.....

Anyway, i've mentioned a wandering bias level as a possible cause of inaccuracy in the flat process. This has been observed with the QSI683 where the offset changed with ambient (not ccd) temp. I can only suppose that the electronics are to blame, maybe a lot of use has taken its toll. Maybe send it to terry. He turned round a camera for Uni of Glasgow pretty fast. Until the results are repeatable its impossible to say what is the cause

 

 

[narrowbandpaul 95]

here is the cloudy nights thread about the ambient temp dependent bias frames for the QSI683. Cant help but think this should never have been an issue

http://www.cloudynights.com/topic/498053-qsi-683-vertical-banding-issues-solutions/