All Sky Camera/Weather Station V2.0

[angryowl]

This will be a thread detailing some of the changes and additions I will be doing to my ASC/Weather Station project. This is version 2.0 as I'll be making some very big changes from the initial project and I think continuing on in the existing thread would not have made much sense.

So, I still want to use an APS size sensor as after seeing the quality and light capturing capabilities of the now defunct Opticstar DS-616C XL camera and Meike lens I simply cannot go back to using a smaller lens/sensor combination. One thing is certain, I won't be paying £400 or potentially more for another APS astro sized camera so with that in mind I plan on heavily modifying a Nikon D50 DLSR and use the same lens. I chose the D50 primarily due to it having a CCD sensor (ICX453AQ) very close in specs to the one in the Opticstar (ICX413AQ) and the fact that I got a hold of a fully working body for £25.

Now there's a few issues with going down the DSLR route which I plan on addressing as follows:

• The oversized camera body can be stripped down to bare essentials and fitted in the existing case with some moving of parts around
• Uncooled, the sensor is quite noisy so to cool it I plan on using the existing Opticstar enclosure with the TEC and hopefully get it purged with Argon to avoid dew formation. Also, since the box will need to be completely sealed to achieve this, there's simply not enough room inside for the main board to which the sensor connects to. The only way around this is using an 39pin 150mm long FPC extension which I managed to find and will be arriving shortly. This means I can have the sensor completely sealed with enough slack in the connection to place the mainboard anywhere I want.
• The D50 uses the NEF file extension as a "RAW" file format but it's not truly RAW and a heavy median filter is applied to all long exposure images to smooth out the noise. It works great for day to day shots, but in an application such as mine it'll most probably eliminate or severely affect my stars as most of them at the FL I'll be using the camera at will be a few pixels across and the Nikon median filter is very aggressive with such small features. The way around this is what's commonly known as Mode 3 on Nikons. Nikons have a additional Noise Reduction mode which takes the long exposure light first then straight after an equal length dark with the shutter closed, then applies the dark on the light and you get a further noise reduced image which again works very well, but not so much for AP. With mode 3 you essentially have the NR feature on and take an exposure but then immediately shut down the camera after the light has finished exposing. What this does is it causes the camera to dump a REAL RAW image onto the SD card without applying the median filter OR the Noise Reduction process. This obviously results in a much noisier image as expected, but all the stars will still be there and the image in this way can then be dark-subtracted and processed to my liking. I'll post some test shots I've taken to illustrate this.
• The D50 uses a hybrid shutter, both the CCD electronic shutter and mechanical shutter are used depending I think on the exposure length. If a high enough exposure is used, from what I understand, one can use exclusively the electronic shutter, but for longer exposures the shutters work in conjunction. Now I know the ICX413AQ in the Opticstar is more than capable of taking long exposures solely with its electronic shutter despite the fact that in its datasheet they recommend a mechanical shutter for proper use. So, my thinking is since the D50's sensor is similar to the ICX413AQ the only thing preventing the camera from being able to take any exposure using exclusively the electronic shutter is that its mechanical shutter is in the way and I don't think that the camera would prevent the CCD electronic global shutter itself to still open and close when required. However, this is all a theory at the moment and the only way to confirm it is to test the camera with the sensor outside when the FPC cable arrives. More on this later...
• In terms of capture software available, the D50 is actually very poor and I could only get digiCamControl to see and control the camera via USB. But I won't be using this as when the camera is hooked up to the PC its SD card is identified as a storage drive and the camera can be used as it would normally with the images appearing on the drive after being written to the SD! Since I'm using my VB app to process the images I would just point the app to that folder and should work.

That's all I can think of for now but if and when new ones come up I'll add them here.

Next I'll be describing some of the other changes planned.

[Gina]

Sounds interesting - I shall look forward to further reports of progress.  Good luck.  I haven't decided yet what I shall be doing with my ASC.  I think the camera imaging board is unrepairable.

[angryowl]

I was planning on adding a rain and cloud sensor before the camera died and now that I'm using a DSLR, I'll be incorporating a proper LUX sensor for a very cool reason and another UVA/UVB sensor just because ?

I opted for a DIY rain sensor made out of nickel-coated stripboard to improve the copper's corrosion resistance. Well, that was the plan at least, but getting a proper smooth nickel finish is difficult if the surface to be plated is not completely smooth. So, some flaking and improperly coated areas were bound to appear, but for a first attempt I'll take it. An advantage over a commercial sensor such as this is the smaller distance between the conductive traces and larger surface area, therefore the ability to detect rain faster and smaller droplets. I had an LM393 comparator around and used this schematic which I think is exactly the same the commercial one uses. The circuit outputs both a digital value based on a threshold set via the potentiometer and an analog indicating the resistance of the traces. So, theoretically one can read the analog output and estimate the amount of droplets hitting the sensor.

For a cloud sensor I'll be using the popular MLX90614 IR sensor with a 90 degree FOV. The issue with this sensor is waterproofing it. It has an IR passband filter in front of it, but I doubt it's actually sealed against the sensor enclosure. I can't just throw a piece of transparent material over it and call it a day as the sensor works in the MWIR region, more specifically 5500 to 14000 nm. Most commonly available materials simply don't pass this wavelength range and there are certain glasses specifically designed for this such as germanium, silicon, sapphire and others but they are prohibitively expensive. As it turns out, natural colour HDPE does a decent job at passing these wavelengths and tested this with a common plastic bottle cap which I polished as smoothly as I could placed in front of the sensor and while it increases the measured temp by about 1.5 degrees, it responds almost exactly in the same fashion to temperature changes as without it. I think the increase can be compensated for in code but more research is needed on this.

The other two sensors are the VEML6075 UVA/UVB which also outputs an UV index as well and the TSL2591 LUX sensor. The LUX sensor has an incredible dynamic range, going from 188 uLux to 88000 LUX meaning it can tell the difference between a starlit, moonlit and cloudy night sky as well as between full sunlight and a sunny day. What I plan on doing with this, besides using it as an awesome lumen meter of course, is fully automate several aspects of the image capturing process.

The D50 doesn't have a simple way of attaching an external shutter other than an IR one, so I'll be taping into the camera's shutter button and have that be controlled by the Arduino via an optocoupler. This way I can leave the camera in bulb mode and depending on the LUX reading, the Arduino can automatically adjust the exposure time needed for a proper image. Furthermore, this'll also include the aperture stepper, so while in bulb mode alone with the Arduino I probably couldn't trigger the shutter fast enough to avoid overexposing during the day, I can simply turn up the aperture and control exposure this way. This means potentially having full automation 24/7 without any user interaction in camera software to change exposure times, which would have been the case with the Opticstar. Since, I'll no longer be using the crude LDR light meter, the TSL2591 will also be able to close and open the lens cover and increase aperture for protection during sunlight.

Sealing the LUX sensor will probably not be too difficult as even standard acrylic passes some IR and the visible range the sensor works in, but affordable UVA and UVB transmissive materials are not exactly easy to come by from what I've read so more research needed on this too.

Some pretty pictures

[angryowl]

To illustrate how the median denoising filter works, I've taken two 30s 400ISO exposures. The one on the left is true RAW without any denoising or dark frame removal applied at around 5 degrees temperature. The one on the right was shot at the same temperature and let the camera apply its median filter. Both stretched similarly in PI. Obviously, the true RAW image will be noisier and have more hot pixels which is clearly seen, but what is also clear is how harsh the filter really is. Looking at this side by side I doubt most of the smallest stars in the image would survive this.

Good thing is the Noise Reduction option needed in order to dump a true RAW unmodified frame on the SD card can be left on.

This way during long duration night time exposures the Arduino could set the exposure length, start it by triggering the shutter button, then a second or so after the exposure duration has elapsed, the camera on off button could then be triggered.

During any daytime exposures, I think it'd be easier to just let the camera apply both its median and NR process as the median pass wouldn't really destroy any data and the NR while it still seems to take place in the camera even at the shortest exposures possible in bulb mode, would not really do much at those exposure lengths, but letting the camera do its thing and keeping it on during the day I think would be best.

[angryowl]

The FPC extension arrived this morning and after some testing seems to work brilliantly. Opened the camera up, removed the sensor assembly and used a makeshift objective in front of the bare sensor. I was able to shoot from 1/4000 to 30 seconds just fine with only the electronic shutter and the pictures are free from any artefacts of any kind. Noise doesn’t appear to have increased and I can’t see any additional patterns of any kind on dark or bias frames. Good result and I can move on to getting the sensor assembly inside the sealed enclosure and the TEC mounted.

Over the course of last week more parts arrived.

The TSL2591 and VEML6075 light sensors. Tested them both and while the TSL2591 might work through acrylic, the VEML6075 definitely won’t. Looking at the datasheet, it might be possible to use 1mm or thinner PTFE as the UV coefficients for some thicknesses are given.

Got a TSL235R light to frequency converter as I’d like to incorporate a sky quality sensor. Don’t have a professional one to calibrate against, but I’ve read that the Dark Sky Pro app for the iPhone is rather good so will be borrowing a phone and calibrate this way.

If I’m implementing a rain and cloud sensor, I want to get notifications on my phone and while I could do that via local network, I really like the idea of having a GSM module hooked up to the Arduino for text notifications. I like the idea so much that I’ll be running the Arduino, cloud/rain sensors and GSM module off a DIY UPS. For this I’ll be using two 18650 batteries in parallel with a TP4056 charging/protection module and some boost converters for the required 12V (Arduino) and 5V (sensors and GSM module). Will cover the full circuit later on.

The Arduino still needs to arrive but I’ll be using the Mega 2560 Pro Embedded board rather than use the existing Mega. Reason behind this is the Pro is about half the size with the same number of IO and features. I’ll need every bit of extra space I can get as I don’t know how big the camera enclosure might get. I’ll try not making the same mistake as last time and minimize component count and get rid of anything taking up space unnecessarily.

The 1TB hard drive will go as I can use a micro SD card of a smaller capacity with the mini PC. I can afford to use a smaller capacity and still store weeks of video as I’ve now got a 5MB 12bit image compared to 17.5MB for the 16bit Opticstar.

Also, the VB image workflow can now be further improved due to full control and automation of exposure duration. The idea is that the app will use several processes to manipulate each image, resulting in a smaller image size. Then when the video for each day is assembled, there is no reason to keep the images.

[angryowl]

On 28/01/2019 at 21:39, angryowl said:

During any daytime exposures, I think it'd be easier to just let the camera apply both its median and NR process as the median pass wouldn't really destroy any data and the NR while it still seems to take place in the camera even at the shortest exposures possible in bulb mode, would not really do much at those exposure lengths, but letting the camera do its thing and keeping it on during the day I think would be best.

Yeah...This won't work as the sensor won't have the mechanical shutter in front of it to take the NR dark. The "dark" will be a light and probably result in the image being subtracted by more or less itself ?

Also found a way to control the NR and other options in digicamcontrol which is great as I'll have almost full control over USB if I ever needed to change anything.

[Gina]

Using a DSLR image sensor is very interesting.  I shall be interested to see if you can get the noise level low enough with cooling.  I'm still thinking about my ASC.  I can't afford a replacement astro camera at present.  Car failed MOT last month and the repairs emptied my bank balance!

[angryowl]

Been looking at dark frame subtraction in code and looks achievable in VB with something like imagemagik or other methods. I'll need to figure out a way to do this as I no longer have the luxury of letting the capture software take care of this.

[Gina]

If (when funds recover) I go for the ZWO ASI 385MC camera for my ASC, I might get away without cooling as it is twice as sensitive as the ASI 185MC it will replace.  That would save some work when I rebuild the ASC.

[angryowl]

5 minutes ago, Gina said:

If (when funds recover) I go for the ZWO ASI 385MC camera for my ASC, I might get away without cooling as it is twice as sensitive as the ASI 185MC it will replace.  That would save some work when I rebuild the ASC.

It's a shame the Fujinon lens doesn't produce a full circle on 1/3". If it did, you'd probably be able to use much more affordable cameras. Or you could compromise on sky coverage.

[Gina]

Yes indeed!

[Gina]

Been doing some calculations and image processing from one of my ASC images and the reduction in coverage with the ASI 120MC is far less than I thought.  This is due to the 120 having a squarer aspect ratio.  The 185/385 ratio is 1.77 whereas the 120 is 1.33 (4x3).  Here's a copy of a post I've put in my ASC Blog :-

22 minutes ago, Gina said:

ZWO ASI 185MC or 385MC have a coverage of 7.26mm x 4.11mm.  ASI 120MC has a coverage of 4.8mm x 3.6mm.  That is 87.6% of the height of the 185/385.  This is not as bad as the diagonal sizes would indicate as the 120 has a narrower X to Y ratio.  Here are a couple of screenshots of the two image sizes.  First is the image from the ASI 185MC and the second the same image cropped to the size of the ASI 120 image sensor with the original size (185 camera) shown as a dashed box.  The loss of coverage is far less that I anticipated and seems acceptable

[angryowl]

That doesn't look bad at all.

Have you considered further reducing the impact by rotating the sensor so that the landscape is roughly 45 degrees rotated clockwise? That would put the trees at the upper left and lower right in your 185MC image on the unused areas of the new chip. You'd still probably lose some sky at the very top but not as much.

[Gina]

Good idea, I'll give it some thought.  Only thing is I like to have north at the top.

[Gina]

Rotated the image then reduced canvas size and it seems to be worse but I may be getting tired and done it wrong...

[angryowl]

The original 185MC looks very eccentric to me and I think that would have to be fixed first (probably not easy) after which playing with the rotation might yield better results. Or better yet, the new 1/3" camera's sensor might be centered properly. Unless it's the lens that's off centre.

My Opticstar/Meike combination resulted in some slight eccentricity which cut some of the image at the bottom. Nowhere near as bad as yours, but still annoyed me greatly. Good thing is I can now tweak the sensor to get perfect framing.

[angryowl]

Decided on how the sensor will sit inside the sealed enclosure and for cooling two TEC1-03103 20mm by 20mm modules will be used. These are the same modules the Opticstar used and got the second one from Opticstar support as part of the spare parts they sent me. Will post some pictures when finished.

I think I’ll use the existing Opticstar TEC control boards and not bother with PWM or redoing the circuits (the TECs run on 3V rather than 12V) as I’ll mount a thermistor on the sensor back and will use temperature to control how many TECs will be on at any given time.

Also wondering how purging the enclosure might work. I think the easiest way would be to have a reasonably sealed box several times larger than the camera enclosure with two holes large enough to insert some nitrile gloves. The large box can then be purged with argon and then through the gloves seal the camera enclosure tightly with the 6 screws. I don’t see why this shouldn’t work but I may be wrong. I don’t think there are any small enough valves to mount on the camera enclosure walls and purge it that way so I think the first option is much simpler. Still thinking about this one, any suggestions welcome.

[Gina]

1 hour ago, angryowl said:

The original 185MC looks very eccentric to me and I think that would have to be fixed first (probably not easy) after which playing with the rotation might yield better results. Or better yet, the new 1/3" camera's sensor might be centered properly. Unless it's the lens that's off centre.

My Opticstar/Meike combination resulted in some slight eccentricity which cut some of the image at the bottom. Nowhere near as bad as yours, but still annoyed me greatly. Good thing is I can now tweak the sensor to get perfect framing.

The 185MC was off-centre a bit - I'm hoping a 120MC won't be.  At one stage I made a centring system but this proved insufficiently rigid so I went for the standard adapters and put up with the slightly off-centre image.  Some correction can be made by tilting the ASC as the lens gives 185 degrees coverage ie. below horizontal.  The actual image circle is about 5mm so bits were cut of with an image sensor 4mm high.

[angryowl]

Some good news, the sensor still works after being removed from its original aluminium mounting and assembled in the sealed enclosure. Looks like I managed to get the sensor spacing about right, but hopefully one of these clear nights I can do a proper test. I’m still waiting for a new Argon cylinder to arrive so I can purge the enclosure.

Wondering if sticking some desiccant in rather than purging the enclosure with Argon would work equally well at preventing condensation inside. Of course, this all depends on how airtight the final enclosure is.

Also, not sure exactly how I could address sensor non-orthogonality. I have four PTFE 0.3 mm thick spacers in between the enclosure and the aluminium cold finger on which the sensor sits. These are there to prevent heat from the front of the enclosure being transferred through the cold finger to the sensor. I had to use 0.3mm spacers so that the TEC aluminium bits sit nicely applying slight pressure on the aluminium cold finger when the enclosure was assembled. Therefore, these spacers can’t really get thicker, only thinner when trying to level the senor properly. Hopefully this won’t be an issue, but only a proper test will tell.

Haven’t tried to cool the sensor yet, but I know both TECs work, as I don’t want to run the risk of condensation on the sensor ruining things. The odd-looking mess encased in epoxy at the top of the cold finger is a TMP75 I2C temperature sensor with a blob of thermal paste between it and the cold finger.

Some pictures of the work:

[angryowl]

Getting the spacing between the TEC aluminium plates and the aluminium cold finger when the two halves of the enclosure were tightly pressed together by the 6 screws as needed to achieve an airtight seal was very difficult. The first few times the TEC plates pressed down on the cold finger so much so that the sensor would no longer output a correct image due to I think some of the sensor pins getting either shorted or some other reason.

What I had to do in the end was bend the corners of the aluminium cold finger so that the entire finger would mount closer to the front glass window, thus moving the sensor further up by a millimetre or so. This should be fine as I can adjust the lens mount to be positioned a bit higher to reach proper focus.

Anyway, being paranoid as I am, I decided that I couldn’t ensure that purging the enclosure with Argon, using the method I described in a earlier post, would go as planned so I added a small 3g sachet of fresh silica desiccant beads inside the enclosure before sealing it all up.

Also, I went about purging the enclosure differently by first fully screwing the four lateral screws that seal the enclosure then leaving the two end ones open. I then inserted an almost snug-fit hose piece down one of the remaining screw holes and started the Argon flow of gas. After a few seconds I partially screwed the other screw in, then removed the Argon hose and inserted the other screw and proceeded to tightening them both down completely.

I suspect there’s still some air left in there, but if I’m not mistaken the enclosure should be filled with mostly dry Argon gas and whatever humidity remained should hopefully be handled by the desiccant.

After giving the desiccant a few days to work, I did a cooling test out of curiosity and here are the results:

Starting cold finger temperature  22.1°C

Only one TECs on  13.3°C 

Both TECs on  7°C                

Left the TECs on for about an hour and no sign of dew or condensation apeared that I could see. The test was done with no heatsink on the back of the enclosure which was gettting rather warm to the touch and I plan on using one with a 90mm fan to keep temps low. So I think a delta of 15 without a proper heatsink and fan on the back should be good enough as I know the Sony ICX chips are rather noiseless and from what I read don’t benefit much from sub-zero temps.

[angryowl]

We’ve had plenty of clear nights lately but sadly not had a chance to try this out as I’ve been focusing on diagnosing my elongated stars in my main imaging setup. This has been going for some time now and driving me crazy and thought I’d make the most of the clear skies for troubleshooting this pesky issue.

The next chance I get I’ll try and get some light with this under a clear sky as I’m very curious the difference cooling will make.

[angryowl]

Heatsink fitted permanently and the fan temporarily for a quick cooling test and here are the results:

Starting cold finger temperature  22°C

Only one TEC on  11.8°C 

Both TECs on  2.6°C

That’s a maximum delta of 19.4°C and I’m more than happy with that and seems like the heatsink and fan help a fair bit in keeping the whole thing cooler. Again, this was left running for about an hour sitting at 2.6°C and no sign of dew whatsoever. I can only hope this holds true when the cold finger reaches sub-zero temperatures, but in theory it should.

The next test will hopefully be a proper one under a clear sky with both TECs running. However, I try not to connect/disconnect the sensitive FPC connector for testing the sensor too many times as I fear some small pin on the extension may get damaged with excessive use and render it useless.

[Gina]

I may have missed it but do you have any plans to protect the fan from the weather?

[angryowl]

1 minute ago, Gina said:

I may have missed it but do you have any plans to protect the fan from the weather?

This will all go back in the main large aluminium housing with the dome on top that was used before, so still all sealed up from the elements. Will also still use the cage fan I had for air circulation inside the main aluminium housing.

[angryowl]

By the way, really sorry to hear about your recent accident with the ASC. But very glad to hear you're still going ahead with it ?