DIY All SKy project

[purgitoria]

A few months ago i started the ball rolling on what will be my new robotic observatory (that's a whole other upcoming post) and wanting to have full automation and monitoring i decided i needed an All Sky Camera. I looked about for one available from various suppliers only to find some have been discontinued and other better ones are really really expensive and the ones i did not mind the price of didn't really seem to have any of the features i was looking for such as automated image stacking to AVI, uploading to my website via FTP, automation of frame grabbing after astronomical twilight etc. I thought to myself, surely it can't be that difficult to build a housing and find some software....
Building the housing was definitely the easy part as i got myself a big old lump of delrin and machined it down on the lathe and mill in my spare time at work and picked up an Oculus spare dome to fit to the top (about the only option for small housings that is already on the market as any other dome available was a minimum of 100mm without a skirt and was not of the best optical quality and would end up making my housing huge. When i get back to work in a months time i will be finishing off the housing and will put up some pictures of the completed product.
One thing that really annoyed me about most reasonable quality All Sky Cameras was that they were only available as a USB interface and after years in my current obs working with various USB equipment i have learned to hate how unreliable it all is and how prone to errors. For this reason i was adamant that the way to go would be a PoE (power over ethernet) camera that when connected to a managed POE hub (that i would use for my CCTV security system) could be remotely restarted and the connection reinitalized should there ever be an issue. This kind of limits your options to using something like a Point Grey or TIS (The Imaging Source) camera as they come with free SDK's (Just because it is a PoE camera it does not mean you can tap into an IP stream as these are GigE machine vision devices). In the end i decided to go for a TIS camera as i have had other Point Grey cameras in the past that have failed. I decided to code it with Visual Studio 2012 (it's free) and work in VB.NET as it seemed the easiest rout to follow and the least complicated structure from what i could see at first glance. It was a rather daunting start but once i got into it i found it was not as bad as i thought (i have never worked with software development in my life and knew nothing about any of this when i started) and after a week of teaching myself and some friendly pointers from people on http://www.vbforums.com i seem to be coming up with some ok results and what i have so far is this:

A basic user interface which automatically prompts you to connect to your imaging device when the program starts and set frame rate and other settings.
The UI starts and an image window is visible with a live feed from the camera initiated and displays some function buttons current computer time and camera serial number a checkbox and text boxes for an FTP transfer function

Function buttons:
A display button which allows you to open the device dialog you started on should you wish to change the settings
A settings button which gives you to capture settings dialog to set camera gain, exposure, set ROI and advanced trigger functions(not currently used)
An image sequence button to allow saving off a sequence of 25 images and write to disk to a specified location
Buttons to start and stop the live video feed display
An AVI capture button which opens a dialog allowing you to choose an available codec for video compression and allows you to select a filename and location for saving the video.

The FTP transfer function allows the user to automatically start capturing live images and saving them to a remote FTP server (such as the back end of your website to publish online)

In addition to main page there is a settings tab with the following features that i am currently developing:

text boxes for the user to enter their Latitude and Longitude
Display boxes indicating the current position of the sun relative to the horizon
Checkbox to allow the user to automatically start image capture on a timed basis when the sun is below -12 degrees relative to the horizon. Working out how to get the mathematics components of this to work was my biggest headache so far and although i found some freely available coding online for this it was in a completely different code language and i just kind of had to pull out the mathematics and write the rest of the code from scratch including correction for parallax. This means that it will capture all the images you need for time lapse or meteor hunting after Nautical Twilight and save them to a user defined location.
A disk clean up function checkbox which initiates a daily timer which will all for automatic deletion of files older than a user specified number of days to save on disk space.

The question now is, what else does it need? Are there any features that any of you use in AllSkyCam software that you find particularly useful or features that are missing but would like to see implemented?

[purgitoria]

Well after a couple of weeks in the sun somewhere a bit warmer it's back to the grind again and i am making some reasonable progress with the app and i must say the Stefan at TIS has been very helpful when i got stuck on some of the coding functions that were not described in their documentation. So far i have a simple interface with a main page and tabbed menu as below:
Sky Cam Main page normal mode
This main page contains the primary basic functions and information feeds
I also have a secondary options tab for user inputable variables and special functions. I decided that it was more convenient to use the interface in a reduced size normally as the program serves more of a secondary function to the all of my other systems but have added an option to pull it into full resolution mode as well should i wish and that mode is shown in the below picture:
Sky Cam Options page full res mode
I have been doing some experimenting under various conditions and have found that it is best to set the camera to a low frame rate (3.7FPS) when it is connected and initialized otherwise there is not enough ambient light in dusk conditions for the auto exposure function to work. To that end when it is set to auto capture mode it switches to maximum exposure (30 seconds with my camera) once the sun is below -12 degrees altitude (and also disables auto gain and exposure functions) but when the sun is back above -12 degrees it will switch back to a 0 gain and auto exposure setting and stop saving images. In this way the system can be left unattended for days at a time without anyone having to monitor it or make any adjustments. I may in the future add some extra functionality to gradiate the exposure setting in steps dependent on the suns altitude to maintain better coverage from actual sunset to full astronomical twilight but that will involve a lot of experimenting once the system is fully operational to determine the best settings for the varying conditions.

[purgitoria]

A little more progress and i am pretty happy with the way things are going so far even though i had a couple of days pulling my hair out trying to figure out some of the features.
I ?stupidly? decided to add some more advanced functionality to the application so that now when doing time lapse sequences in an unattended setting the application will now automatically create new folders named by date for each nights imaging which will not create the new folders until noon so that the nights imaging is always nicely organized and each image will save into it's correct evenings folder and be sequentially numbered for ease of post processing into time lapse. This was not the easiest function to create and involved a lot of head scratching (i may just have worn a bald spot.....) as the system for finding the correct folder was a bit complicated due to having this noon time for creating new folders. As well i decide to to a bit of cosmetic housekeeping to make the options interface a little clearer. I have also added in a few extra options boxes for user input so that the end user can select their own upload timers, auto cap folder and set their own solar altitude setpoint.

While doing all this it also dawned on me that if other people want to use the software they may well not have the exact same camera as myself and the sensor format may be completely different. So far i had only based it on my 1280x960 format sensor so if anyone else had used the software then they may well have found that the image was distorted on the screen due to the display formatting and i also had no screen stretch function other than full resolution mode which obviously just worked for my sensor.
This was time to get stuck in and add yet another tab with further options consisting of a series of radio buttons so that once the users camera is connected they can select an appropriate sensor format and the application window and video display will modify itself to fit the aspect ratio of that particular format (thankfully half the sensors had the same aspect ratio which made life a little easier). Now any sensor (i think i have all the currently available formats in there) can be selected and the application will provide the correct undistorted image. Each of these have a maximum and minimum windowed mode so that they should always be able to be sized onto almost any screen up to 1080 resolution (i did of course have to put limitations on the larger sensors as there is no way a full resolution image will fit onto the screen). Just to help the user if they don't remember their sensor format i have also added an info slider at the bottom which displays their camera name and the current video format they are using.

In addition to all that i have also spent some time compiling my HTML help file to distribute with the software.

So what's next on the list.......
Anyone any suggestions of what else it might require?
Perhaps a compass overlay?

[purgitoria]

And so the saga continues....
I was reasonably happy with the way things were going having solved issues with getting the system to display properly for different resolutions without distorting the image and having tested all of the other functions successfully. Then i went and stuck my camera on a long ethernet cable and put it out the window looking at the sky to take some proper subs and evaluate the kind of results i was getting. My first sub of 10 seconds looked like this:

I thought to myself "it's not winter, its the middle of summer, why is it snowing!?"
Obviously due to being a video camera and using POE there is a power transformer inside and that along with the constantly active sensor creates quite a bit of localized heat and this in turn was causing a lot of dark current to show up in the image which was really bad, especially when there is so much of it you can't differentiate between it and the stars (not that there were any showing when i took the picture). I used my save sequence function to capture a set of 25 dark frames and then did a quick master dark and then dark calibration of the image to get this:

A world of difference I'm sure you will agree as i can actually see the image instead of the dirty speckled haze. So this got me thinking, surely i must be able to automatically subtract darks from the images as they are captured? I know that the basic functions exist in the controller class so i decided to have a little play around and added in some script that will now automatically calibrate the images as they are written off from the image buffer and added some additional options tabs in there for activating the function (it will only run when the appropriate dark frames are available for the specified sub length) and will not do any dark subtracting when the sun is above the solar altitude setpoint.

[purgitoria]

Well i guess it is my own fault for not checking my drivers, lesson learned but not that it is a disadvantage for me with the effect it has had. I was having issues getting the software running on a different computer and never noticed that the driver i had installed was a much more up to date one than what i had installed on the computer i was using to develop my software. Not that it was my software causing the issue but actually an option during the driver install as it was an older laptop i was trying to test out the software on and as i am using a GigE camera i should not have installed a filter option which totally messed up all my network adapters as the laptop did not have a gigabit network adapter. When i did get it working however i noticed that i seemed to have a lot more options in the device properties dialog some of which i realized were of great benefit.
It seems that in a driver revision the dialog for device properties has been expanded to include additional slidebars and checkbox options for brightness, contrast and gamma which are more for the live viewing of the video and do not alter raw data and present no great benefit. What had appeared however was a most helpful setting which was for a maximum auto exposure limit. Previously in auto mode the maximum exposure setting was limited by the frame rate (fps) setting but this new feature allows the user to manually bypass this yet still allow the auto exposure up to your specified limit. I had previously been wondering how to gradiate the exposure during transition between daylight and night time but this little gem sees to have solved my problem for me .
This also got me thinking about another feature (there seems to be no end in sight just yet for what i am adding in) because it opened up easy possibilities for some additional auto capture and time lapse features. The main auto capture feature was designed specifically for unattended operation at night only so i decided to add in an additional feature with shorter time intervals so that the end user can now do easy time lapse of the daytime weather and the transitions from day to night as well as continuing all through the night if they wish, simply leaving the exposure in auto with the longest exposure length being dictated by the setting of the auto exposure maximum limit slider. This feature should also be great for capturing lightning storms, though it may be a while before i can see how effective that will be and will involve a lot of luck and timing: hopefully one day.
I thought the weather might have been a bit better tonight but turned out really overcast in the end so there was no full night of testing worth doing but at least i have my transition working from day to night and below is a time lapse of an initial test. The daytime part is pretty smooth but as the light fades the clouds look a bit stuttery as the exposure time ramps way up and i didn't alter the frame rate in the video to compensate as it was just a quick and dirty edit. I had it set to a maximum exposure of 15 seconds on this trial run and there was a total of 1200 frames captured over a period of just under 3 1/2 hours and the video was output to 25fps.

 

[steppenwolf]

This is an interesting project - the ASC on my automated observatory is a USB device and I have to say that I have been lucky with this connection type. On my elderly observatory PC (running Windows XP!) I have 5 USB ports on the motherboard, a 4 port PCI card and then my monitor has 4 ports on it as well (on which I run the non critical devices like the mouse and keyboard. I even have a non-active 5metre USB cable running to the all sky camera (a ZWO ASI 120mm). I dread that PC ever dying on me!

Good luck with the rest of your project.

[purgitoria]

Great thread on the robotic obs there Steve, i'm going down the same route myself and at present am only being held back by my observatory builder who is several weeks behind schedule. I work overseas for 6 months of the year and normally when the imaging conditions are at their best so always find it frustrating, in the past year i have done virtually nothing and am starting to forget how to process images properly. Now i will be heading back to work for a couple of months overseas again with my obs due to arrive a week or 2 after i have left so now it will be November before i really make any progress on it but at least once i am back at work i can finish my all sky camera housing. I have considered ACP but never sure it was worth the money and think i could make the system work with an integration of CCDAP and MaximDL, although i use TheSkyX for target planning as it is a really simple target list import into CCDAP. Bit of a mish mash of a build right now as i love to tinker (it keeps me entertained and out of trouble on my 2 months off work) but i enjoy putting all my own elements together. I will be using control elements from Scope Dome, with a custom built obs from Orbsey,  a Davis weather station, AAG Cloudwatcher Solo,  additional Firefly and Platypus controllers for peripherials, home built all sky and my own designed software all run on my home built computer system. I'm sure it will be up and running smoothly in a matter of days once i get started......... I had really hoped to put a 12" OS RiDK in there but they decided to discontinue all of their deep field instruments under 16" and have no stock materiel left for any more mirrors (i really tried to be the last one to get one built but they were adamant it was not going to happen). Not sure what i will go for now, maybe Orion ODK, Altair premium deep field or stick with a big frac like the Tak 150.

[purgitoria]

I now have a website associated with my software for anyone that is interested. http://sky-cam-astro.uk There is an up to date overview of the software on there as well as a few images of the current layout. If anyone is interested in trying out the software, especially anyone with a colour camera as i do not currently own one to test it on myself, then please get in touch via my website and i will send a link and download instructions for a beta version. The help file needs some updating as i have made quite a few changes in the past week and the latest help file will be in my next revision. The beta version is restricted to 12 hours use before the program closes but can be immediately restarted by the user and is offered free to anyone interested in testing it out.

[purgitoria]

So i promised to update on the physical build of my camera and now i am back at work i managed to complete the machining and have the finished product ready for some weather testing.

First of all it was machining down the big block of delrin i had into ll the individual parts i needed.These consisted of the main body, a base cover and a retaining ring for the dome.

Not the worlds largest or best lathe but it did the job. The hardest part was getting the block down to a manageable diameter as i had ordered a 135mm diameter block but what i received was a 170mm diameter block and the largest diameter my lathe chuck could handle was 138mm. There was a bit of sketchy drilling and tapping to put  some threaded bar in one end and jamming a rotating center into the other end so that i could whittle it down a little and even that was a struggle as i could barely get the tool out enough for that. I did manage eventually though.

The main body was bored out and a groove machined into the bast to take an o-ring seal as well as holes drilled and tapped for the bolts. A shoulder was machined all the way in where the camera would sit and bored all the way through just wide enough to take the neck of the camera. The idea was that the lens would screw in from the other side which had a stepped shoulder and would mean it would secure the camera in place without having to worry about any fancy mounting brackets or trying to thread holes inside the housing.

As i decided to go with a PoE camera for simplicity i had to mount an IP68 rated fitting into the base to allow me to connect a network cable.

I recessed it on the inside to allow as much space inside for the cable

On the retaining collar for the dome i recessed it for the shoulder on the dome and and then used a marine grade silicone to seal the dome in place and make a watertight seal. I did consider epoxy but should i ever need to replace the dome it is much easier to remove wth a silicone based product.

I then had all my parts ready for assembly

The camera was put inside and secured by screwing on the lens

Patch cable connected

And covers all bolted on