Flats issue from a newbie

[ACross]

Hi All, I am posting what I suspect is a newbie mistake question but hoping that someone can assist with the issue of flats.

Although I have been fumbling around the sky, taking snaps at leisure, recently I became serious.  I have read up about the different calibration files (flats, darks, bias) and they seemed to make sense; different ways to capture the image defects and extract those from the image of the sky. After a few weeks (months) of further fumbling I went back to the very first target to receive my attention, M42 Orion Nebula.

In short, I took 20x 30s exposures in LRGB  and ran these along with 20x LRGB each of darks, bias and flats. To obtain the flats I used a diffuse sheet of perspex (lightbox material) and an LED video lamp that has 180 white LEDs, turned to its lowest setting. Attached below is the stacked Luminance flat and the light image. In the lights I am getting very strong marks from dust and I had thought that the flats would subtract this but looking at the flats the marks are completely different shapes and do nothing to remove them from the lights.

The attached has been further stretched to show the issue. Now, I am obviously doing something wrong but I have no idea what, any pointers from the vast pool of knowledge will be much appreciated.

Thanks, Anthony

 

[PhotoGav]

The important thing with calibration frames is that settings should be matched to the lights they are going to be used with. In the case of flats it is essential that the optical train is not moved in any way, especially in terms of the camera rotating with respect to the optical tube. So, with your images above, were the flats taken with the camera and telescope in exactly the same alignment?

[Demonperformer]

The donut on the bottom and two on the left hand side appear to be in roughly the right places, but are quite a bit smaller than their equivalents on the light. So I'm wondering if there was some change in focus between the two? [there mustn't be]

[RayD]

To add to Gav's comment, you mention you have taken 20 x LRGB each dark and bias frames.  This isn't necessary as your darks and bias frames should be taken with the OTA covered to let in no light at all, so are not needed for each filer.  You just need to make sure they are taken at the same sensor temperature as your light frames.

I could be reading this wrong of course and you do mean just 20 of each and don't mean for each filter.

[david_taurus83]

If it was me, I'd be pre-emptive and get a camera sensor cleaning kit (blower, brushes, solution etc) and carefully clean the sensor window and also check the filters. Maybe give the filter wheel a good blow out as well. It's better to not have the dust donuts there in the first place as those ones look quite difficult to remove. Newts however are always open to contamination. Maybe a light pollution filter or clear filter in the nosepiece to 'seal' your imaging train?

[tomato]

Unless you have permenant set up, I would take the flats at the end of the imaging session, so the focus will be the same. As you point out, a set of flats for each filter should be taken as the dust motes will be different on each filter and these are pretty close to the camera sensor.

+1 for having your optical train as dust free as possible.

[ACross]

Thank you for your replies. After posting I realised I had missed some detail out so to fill in and answer a few questions posed. 

Firstly as to LRGB, I took 20x each for darks and lights and 50x each for bias. I knew this was overkill but the Pixinsight BatchPreprocessing script didn’t bat an eyelid and produces me a set of master calibration files (this was the first time I had used PI).

I have a permanent setup so the scope was left in place while the darks were taken, immediately after the lights using the same SGP sequence but with the end caps on and lights (and pc monitor) off. @PhotoGav & @tomatoThe flats were done the following day with absolutely nothing moved, including focus. I set up a lamp and diffuser and took the flats using the same camera temperature of -15 but the ambient would have been higher, not sure if this is enough to have made a difference.  Now here’s something: the exposure of the flats was only 40ms due to the amount of light from the lamp, could that be it? Should I be trying to get the light down to a level where 30s exposure is not blown out to white? 

After taking the flats I figured I had nothing much to lose and so experimented to find the source of the doughnuts. Image train is as follows: Into the 2” eyepiece I have a new filter wheel (and new filters). Connected to this is a Baader coma corrector that is screwed onto the front of the camera. Firstly, altering the focus did not alter the doughnut shape in any way. Nor did changing the filters or rotating the camera&wheel. With everything attached I then slightly unscrewed the camera from the corrector and still no change or rotation; this tells me the dust is on the camera. There is a glass screen over the sealed sensor although I cannot see anything on it I am hoping the debris is only on the outside as I do not relish the idea of opening up the sensor to the air.

Agreed that it is better to clean the imaging train and not have to correct for these and I like you suggestion @david_taurus83 of the light pollution filter, this could attach to the thread at the end of the focus tube and seal the whole thing. I am still puzzled as to the difference, as you say @Demonperformer the doughnuts do correspond in location and intensity to the marks in the lights but I am baffled as to why they are a different shape. In tapping this reply out I am beginning to think it is connected to the exposure and perhaps I should try exposing flats with a much lower light level and aim to get nearer to 30s exposure, perhaps a single LED And a couple of diffusion sheets. I would be interested to hear how others do this and what exposures.

Than you all for your comments. Will be spending some time today cleaning up the camera as this is the ultimate answer.

Anthony

 

[PhotoGav]

I am slightly confused about what calibration files you generated and which you actually needed to generate! Did you take darks and bias for each filter? If so, that is completely unnecessary. You only need one set of darks and one set of bias as no light is passing through the filters, so the filters are irrelevant. You do need flats for each filter used (some say you can get away with a ‘universal flat’, but I would not advocate that). Regarding light levels and exposures for flats... you should aim for about 30000 to 32000 ADU or just shy of half way along the histogram, so the exposure value itself is not important, just use a value and a light level that gives you that ADU result. 

I have no idea why your flats don’t work with your data though... from what you say, they should. All I can suggest is that it is down to the processing. Could you share the data set through Dropbox or something and I would be happy to have a go at processing it to see if the problem persists or not.

[david_taurus83]

Hi Anthony. On a separate issue, I'm a bit confused about your coma corrector spacing. If it's the Baader MPCC then it's optimised for 55mm from its flange to your sensor. Have you mounted this between your camera and filter wheel? I would have gone: Scope - MPCC - 16.5mm M42 spacer - 20mm filter wheel - ASI174MM 17.5mm from nosepiece to sensor. This gives you 54mm spacing which is pretty close in itself but glass adds back focus as well. So with your filters and the protective glass in front of your sensor you could assume the extra 1mm required for 55mm. Again, this is all assuming you have the MPCC. If so, the MPCC would act as the seal in the image train, keeping dust, pollen, bugs even out of system!

[ACross]

Hi Gavin.

Rather sheepishly yes, took 20x darks for each filter and also took 50x bias for each filter too!  Thought it might have been overkill but dont think it did any harm other than using up a chunk of my NAS drive space.  Is it ok to have a master bias and dark and how often should it be updated?

As regards flats, I had seen a Pixinsight tutorial which talked about a master or universal flat but was not entirely convinced; your comment is heeded, flats each time.  Your comment on exposure of the flats is encouraing, I had read about ADU and didnt understand it but selected an exposure that showed the peak approximately in the middle of the histogram. I will have a closer look at this before the next attempt.

I appreciate your offer to have a go at the processing and have copied the original capture data into Dropbox but caution that the folder is almost 1.4Gb. It can be access by this link: https://www.dropbox.com/sh/l6cydc3lirz083i/AAADMKS3AziIdqMUvIG9DChua?dl=0

Cheers, Anthony

 

 

[tomato]

Best not to have too short an exposure time for your flats, better to dim the light source by whatever means, the low tech solution of putting sheets of A4 printer paper in front of the light source works. Also if you have options for how the data is transferred off the chip, be sure to use the same one as you use for your lights, my software has a fast preview mode and a slower low noise mode, I had problems when I mixed the two.

[ACross]

Hi David

Your comment is extremely helpful as this is something I have struggled with a lot. Yes it is the MPCC, purchased nearly four years ago for my first VX8 on the advice of Orion Optics but could not understand where to fit it. Unfortunately not long after I had an accident and replaced the scope and mount as part of an insurance claim.  The second VX8 was purpose made (upgraded to 1/12 pv) and colimation seemed much better so stars were more round but still not ideal so I dug out the corrector again. I read about the 55mm spacing but it was not clear whether this was from the sensor surface, the glass front, flange or what.  I took it to be 55mm from lens to sensor and then guessed as to how it should fit.  

I am going outside to have a look at this and see if I have enough bits to make it work as you suggest.  

Cheers, Anthony

[ACross]

@tomato  Good point about the usb speed, I am not sure if I checked this and will have a look at the settings. The capture was done in SGPro and I think that the speed is setup in the equipment profile and this was used for both light and flat capture; worth checking again though.

[Demonperformer]

Master bias yes. Master dark as long as you have set point cooling on your camera - they need to be matched to the temperature you take the lights.

[ACross]

Thanks @Demonperformer 

Yes, have setpoint cooling and normally keep this at -15. Do you update the masters at all?  I guess over time there may be changes in the sensor such as dead pixels?

[PhotoGav]

Thanks for uploading the data Anthony. I will have a look as soon as I can. 

Regarding darks and bias, I update my masters annually. Flats are done for each project. 

[PhotoGav]

Well, I've had a play with the Luminance data and have obtained exactly the same result as you... which is good and bad! At least it's not an issue with processing (or perhaps we both have it wrong!). My best guess is that the mark left at the bottom of the frame is a drop of condensation on the chip window - it has a sort of thickness to the bottom of it as if a drop of water has gathered there. Have you had a chance to look and see what is on the chip window? Is there lots of rubbish on it?

Sorry I can't be of more help and good luck with sorting this one out...

[ACross]

Thank you Gavin, that is actually very encouraging; this is my first foray into "proper" LRGB imaging, first use of SGPro, first use of Pixinsight and one of the first uses of my re-configured  observatory too.  If we are both getting similar results that shows me my processing and basic understanding is good and that the problem lies somewhere physically in the system.

I have not yet had chance to take a detailed look at the camera but having done some elimination work I am certain that it is an issue with the camera, either on the surface of the glass or inside the sensor chamber - this would explain the droplet look to the small mark at the lower edge.  If the rain eases up a bit I will bring the camera in and have a good close look at it.  I remain baffled as to why the flats do not resemble the defects in the lights but the ultimate answer has to be to remove the defects in the first instance, before capturing anything and this has to be my focus (pun intended).

I appreciate all the comments and assistance posted in response, thank you.

Anthony

[PhotoGav]

My pleasure - I know how frustrating it is trying to get kit up and running and am constantly relying on other people to help with my own system, so it is good to be able to try and help others too.

I look forward to hearing what you find in the camera!

[ACross]

Well I have just had a good look into the sensor chamber with a strong light and a magnifying glass. There were dust motes on the outer surface of the glass and after blowing these away I could see a tiny speck on the sensor and another on the inside of the glass. Nothing for it but to open the chamber. Inside there are four desiccant tablets and it could be that the small specks came from those. After some careful use of the air pump (the tablets are very light) and reassembly it looks like I have eliminated most of the marks; certainly the prominent ones at least although there is a small one remaining in the middle of the image. 

Now I must wait for some dry weather to have another go at imaging and producing a new set of calibration files but I am much more optimistic, especially after Gavins efforts which confirmed I wasn’t doing everything wrong.

Cheers, 

Anthony

[ACross]

On 25/11/2018 at 11:53, david_taurus83 said:

Hi Anthony. On a separate issue, I'm a bit confused about your coma corrector spacing. If it's the Baader MPCC then it's optimised for 55mm from its flange to your sensor. Have you mounted this between your camera and filter wheel? I would have gone: Scope - MPCC - 16.5mm M42 spacer - 20mm filter wheel - ASI174MM 17.5mm from nosepiece to sensor. This gives you 54mm spacing which is pretty close in itself but glass adds back focus as well. So with your filters and the protective glass in front of your sensor you could assume the extra 1mm required for 55mm. Again, this is all assuming you have the MPCC. If so, the MPCC would act as the seal in the image train, keeping dust, pollen, bugs even out of system!

Hi David. Well I had a good look at the whole imaging train and also found a M42 male to male connector in the box of the filter wheel. This means that I can now fit MPCC directly to the filter wheel, the M42 connector and then the camera, giving me 54.5mm from the MPCC flange to the sensor.

Excited to try this out if only the weather would clear!!

Thanks for your help with this aspect, it has been a troubling point for some time.

 

[david_taurus83]

^^^ Glad I could help and I hope it improves things for you. Just to point out, the configuration you've quoted above. It may indeed be fine but it might be better to mount your camera as close to the filters as possible to avoid vignetting. So you could put the M42 connector on the MPCC and then the filter wheel/camera? It's dependant on your sensor size.

[ACross]

Aha, success!  With the very helpful advice on here I took the whole lot apart and started again. Starting with the scope collimation, re-positioning the coma corrector, cleaning sensor, cover, filters and corrector. After re-assembly and a lot of cloudy nights, I finally managed to get a good go at this last night and the results are so much better. 

 

I have almost eliminated false dark marks and those that remained were removed by the flats.  This is my first attempt at processing with Pixinsight and although it is not perfect, I am very pleased with it.

 

[PhotoGav]

Brilliant result well done. In a slightly masochistic way, sometimes the pain of it not working makes the pleasure of it finally working all the more enjoyable!

[ACross]

You are absolutely right. A few weeks ago I was getting so despondent, the whole lot nearly went onto AstroBuySell and my wife was starting to question my sanity. Then, while imaging this at the weekend, everything fell into place and it was thrilling to see it come together at the processing phase; very satisfying and reminded me why I get into astronomy in the first place.

Thank you to everyone for their help and comments.  Wishing you all clear, calm, dark nights and a Merry Christmas.