Raspberry Pi AllSky Camera

[Pete6]

It's been done several times before and this is my try at making one.  I built Thomas Jacquin's excellent project a couple of years ago and it worked very well.  However I found that the temperature inside the dome exceeded the working specifications of the ZWO camera.  Since the ASI224MC in question is also my guide camera I had a re-think.  I really wanted my own AllSky camera so I made my own.  You can see the end result of my efforts here http://75.10.164.105:8082/

Please remember that this is just a Raspberry Pi on the other end of your browser.  

The code is on GitHub here https://github.com/rederikus/AllSky-Camera-Raspberry-Pi

Initially I used a Pi 3B that was just lying on my desk.  Later I used a Pi 4B and even later I put in a Pi Zero W.  They all worked perfectly once I had made my software work.  The Pi Zero W only has WiFI but that does not stop it working.

The Pi Camera V2 is really the heart of this system.  I started out with a Pi V1 camera and it just did not have the low light performance I wanted.  The V2 camera sees down to about magnitude 3.5 from my light polluted back garden.  I intend taking it my observatory now it is stable and once I come out of self-isolation.  Then I can truly judge what it can see.  The Pi Camera V2 uses the Sony IMX219 sensor but the Pi camera version only has a fixed focus lens.  Arducam make a replacement sensor body with an M12 interchangeable screw lens.  Arducam also sell a variety of lenses so I acquired a 185 degree one and I was set.  I put in a cheap buck regulator board so I could feed 12VDC to the case and just derive 5V inside the case allowing a longer cable run

My software is just a bunch of bash scripts and html/php files but one file, webcam, is the core and it uses sunwait to obtain the daily sunrise, sunset and twilight times from latitude and longitude coordinates.  These times drive the camera and video modes.  

In use webcam runs itself at about 3AM daily and writes a daily crontab file.  In turn this captures a still image every 2 minutes during daytime and 4 minutes at night.  Every 3 minutes in copies the current still image to a directory and then every 10 minutes the stills are concatenated into either a day or night video.

The images gathered are kept for 24 hours and then erased as a new video is created.

I re-purposed a dome security camera case to house the device and I also included a DS18B20 to provide local temperature information.  I did think of using a BME280 but I had several DS18B20s in stock.

I eventually made two variants.  The first used a good quality but expensive security camera case and a Pi 3B inside.  That may be viewed here https://photos.app.goo.gl/kCZFW5AQyCtS55vf7  This cost just under $200.  I then decided to make it a cheap as I could whilst keeping the same specs.  For this I used a Pi Zero W, the same Pi V2 camera, a DS18B20  and a $7 dummy security camera from Amazon that was not waterproof until I made it so.  That's here https://photos.app.goo.gl/VJXURWkrzbK8dSfWA and it cost just over $50 using all new parts that runs off a 1A USB power adapter.

I am pretty happy with it and intend moving it to my permanent observatory once that is completed.  The software is available above and is obviously free.

I hope some of you enjoy it.

Pete

 

[Pete6]

One thing I forgot to mention is that if you omit the DS18B20 temperature probe then this project requires no soldering at all.  All you need to make this work is a Raspberry Pi and a Pi Camera V2 and, of course my free software.

There is a version of the files needed to run the software without the temperature probe in the GitHub repository. It is only the web pages that use the temperature probe.

I have never tried my software on anything earlier than a Raspberry Pi 3B so I cannot vouch functionality on the original Pi or the Pi 2.

[jiberjaber]

That's great - as a thought you could upload the resultant video to a google drive and then link back to that from the web page to take some of the load off the Pi perhaps?

Idea came from here:  https://github.com/ccrisan/motioneyeos/wiki/Features

 

[Pete6]

You are right.  Thank you.  I shall wait until the clouds go away and I get a clear night and grab the video.  The are about 5.5Mb each.  In the mean time you can have a look at the real thing here http://75.10.164.105:8082/ and see the stills, current day and night videos and the rest of my designed for old geezers with phones menu system.

[jiberjaber]

Hi Pete, Just going through your instructions to give this a try, couple of points I noted whilst running through the instructions and actions  

Could do with updating the formatting of the instructions, some of the commands to be executed are hidden in the instructions so might be hard for others to pull out what to do

Seems to be missing a few steps?

- I couldn't get the php test to work, I just get 404, perhaps it needs to be in the html directory?

- How to copy your files to the PI (git clone.... etc), and a few other commands required;

- copy webcam to /etc/init.d, also;

- some roge commands which don't seem to do anything (sudo chmod 755 /home/allsky/.sh )

- cp /home/allsky/sunwait/sunwait /usr/bin needs sudo in front of it

- I couldn't see which file I need to edit to change the sunwait location etc

I've got to the stage where I've rebooted and loaded the all-sky camera page but after 10 no images....  I think I might have found the problem, you have cron files but no instructions on what to put where?

 

[jiberjaber]

Right - I've found the  Raspbery\ Pi\ AllSky\ Camera.txt file which has better instructions in it, I was going off the instructions in the git page, just editing the webcam file now

 

[jiberjaber]

I get the following error:

 

Quote

pi@raspberrypi:/home/allsky $ /home/allsky/webcam
raspistill not running
/home/allsky/webcam: line 269: [: too many arguments
/home/allsky/webcam: line 273: [: too many arguments
/usr/bin/raspistill

E V E N T                                      HH:MM OFFSET
---------------------------------------------  ----- ------
Astronomical Rise (18 degrees below horizon):  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 01:41 -00:10
Astronomical Rise (Real)                    :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 01:51
Nautical Rise (12 degrees below horizon)    :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 03:18 -00:10
Nautical Rise (Real)                        :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 03:28
Civil Rise (6 degrees below horizon)        :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 04:17 -00:10
Civil Rise (Real)                           :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 04:27
CAMERA switches from Night to Day           :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 04:19 -00:08
VIDEO switches from Night to Day            :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 04:24 -00:03
Sunrise (0 degrees below horizon)           :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 05:08 00:00
Sunrise (Real)                              :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 05:08
Sunset (0 degrees below horizon)            :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 20:52 00:00
Sunset (Real)                               :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 20:52
Civil Set (6 degrees below horizon)         :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 21:32 -00:00
Civil Set (Real)                            :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 21:32
CAMERA switches from Day to Night           :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 21:41 -00:09
VIDEO switches from Day to Night            :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 21:44 -00:12
Nautical Set (12 degrees below horizon)     :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 22:32 -00:00
Nautical Set (Real)                         :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 22:32
Astronomical Set (18 degrees below horizon) :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 00:09 -00:00
Astronomical Set (Real)                     :  Error: Unknown command-line argument: 0.47527778 Error: Unknown command-line argument: 51.71972222 00:09
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ===
Authentication is required to stop 'cron.service'.
Multiple identities can be used for authentication:
 1.  ,,, (pi)
 2.  root
Choose identity to authenticate as (1-2): 1
Password:
==== AUTHENTICATION COMPLETE ===
rm: cannot remove '/var/spool/cron/crontabs/root': Permission denied
cp: failed to access '/var/spool/cron/crontabs/root': Permission denied
chmod: cannot access '/var/spool/cron/crontabs/root': Permission denied
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ===
Authentication is required to start 'cron.service'.
Multiple identities can be used for authentication:
 1.  ,,, (pi)
 2.  root
Choose identity to authenticate as (1-2): 1
Password:
==== AUTHENTICATION COMPLETE ===
Wed 13 May 20:43:24 BST 2020
 

 

[jiberjaber]

So I have sort of managed to get this working. I needed to add to the webcam script 'sudo'  to allow the crontab commands to work. Some parts are not functioning but I am able to get an image (using a V2 noir camera)

 

I am using the no temp sensor html directory version, the about page doesn't work. The errors shown above are also present in the sun times page. (on both versions of html)

 

 

I also had to remove the -hf flag to remove the horizontal flip on the camera image. 

 

This has a lot of potential if I can get the rest of it working it working 👍

Edited May 14, 2020 by jiberjaber

[jiberjaber]

Not sure if anyone is following this, but just in case, I now have the sun times rectified.  The Lat/Lon in the webcam script must be in the exact format specified, that is 2 digits left of the decimal point and 6 digits right of the decimal point then the direction eg: 51.123456N any deviation from this seemed to stop the script from working.

 

I've reverted back to the with sensor html files as the about page failed with the no sensor files, just need to resolve the lack of seconds till next capture timer now

Edited May 14, 2020 by jiberjaber

[DaveL59]

getting there, tho the FoV may be a bit narrow unless your sky view is limited by the location/obstructions. How's the night/sky view without the lighting?

[jiberjaber]

Just now, DaveL59 said:

getting there, tho the FoV may be a bit narrow unless your sky view is limited by the location/obstructions. How's the night/sky view without the lighting?

Very dark! Though as dawn was happening as I got ready to take down the scope I could see stars, given this is just the standard lens and through a double glazed window I was quite happy about that, unfortunately I didn't save this image but here's a copy of what I posted to facebook this morning... you can just make out a star between the temperature 'C' and the trees. My sky view is limited so a wide FOV might not be worth it.  I might try and put it out on the patio with the scope tonight if I remember.  Certainly has potential - I might look for where the files are deleted and comment that out, there are other ways to manage storage space within linux which might allow a round-robin buffer of images to be present subject to a setting and disk space  

[Pete6]

Oh wow!  I had stopped looking at this thread.  I thought it had died.  I am so sorry.  I have been quietly updating and submitting bug fixes and enhancements to Github as I have found tham.

I will give any help I can to anyone who wants it.   I am sure that, since I wrote this, I may well have skipped some parts of the instructions.  I also need to repeat that I am NOT a programmer.  I just wrote stuff and changed it until it did what I wanted.

I only just saw these posts.  I shall now read them properly and will reply once I have digested their contents.  Again, I am sorry I was remiss in monitoring this thread.

[Pete6]

@jiberjaber,

I think I know why you are seeing errors.  I run my AllSky Camera in root.  It was easier for me to do this and since the device sits up on the roof and was not used to run any other jobs I figured it would be okay with a strong password.  I made some effort not to use root and after three days I failed. Please try running in root and see if this resolves the condition.  It should.  If you really do not like running in root then I can go back and have a another try at the problem.  If you know how to do this then please let me know.

Somewhat a non-answer I know but this is what I did.

Secondly, regarding your most recent post, please let me know which files you need to find and I'll tell you where they are.  In brief, it works like this.

- raspistill takes s still image every 2 minutes by day and 4 minutes by night.  Each still image is written to /run/shm/webcam.jpg at 800x600 pixels

- every 3 minutes, day or night /home/allsy/pics/copypic.sh copies /run/hm/webcam.jpg to /home/allsky/pics/webcam-$(date +'%s').jpg  The $(date +'%s') bit writes the date as linux epoch format which is seconds elapsed since 1 January 1970.

- every 10 minutes /home/allsky/pics/concatday.sh or concatnight.sh concatonates these still images into a 5 frames a second video that builds over the period.  The webcam-ddd.jpg files are copied (not deleted yet) to /home/allsky/pics/day or night directories.  They stay there until the next day or night so there is one full day of history.

- at day/night changeover (there are two one for the camera day/night and one 3 minutes later for the day/night video) /home/allsky/pics/newdaymovie.sh or newnightmocie.sh runs to make a new period video.  Once the new movie is completed newday(night)movie.sh deletes the previous day or night directory content making ready for new still image history files.

That's how it works at an image level.

/home/allsky/webcam is the key file.  It runs everything.  webcam's job it to get the day's suntimes via sunwait and then write out a crontab file to /var/spool/cron/crontab/root.  This may be viewed by typing crontab -e Not that crontab runs webcam each day at about 3AM to generate a new daily crontab file.  I have commented the webcam file fairly well but PLEASE come back and ask question that I may or may not be able to answer.

If I have left anything out or if any part of this is unclear to your just ask  and I will spill the beans.

I want my little project to be accepted 'cuz I think it's good and cheap so I'll help.

Pete

Edited May 14, 2020 by Pete6
more info added

[jiberjaber]

OK managed to get the movies created I think.  Needed to create a couple of directories 

 

/home/allsky/pics/day

/home/allsky/pics/night

 

[jiberjaber]

2 minutes ago, Pete6 said:

@jiberjaber,

I think I know why you are seeing errors.  I run my AllSky Camera in root.  It was easier for me to do this and since the device sits up on the roof and was not used to run any other jobs I figured it would be okay with a strong password.  I made some effort not to use root and after three days I failed. Please try running in root and see if this resolves the condition.  It should.  If you really do not like running in root then I can go back and have a another try at the problem.  If you know how to do this then please let me know.

Somewhat a non-answer I know but this is what I did.

Hi Pete, You've done a great job I think it's mostly the instructions that need neatening up to make it easier for others to install (I'm no programmer either, so just debugging what I can )

It is running as root I think as the files it creates (stills & movies) are owned by root.  

I think most of the issues I've got to the bottom of now.  Only thing left to do is solder up a sensor to confirm that is working for me.  The next image in X seconds seems to have totally disappeared since I adjusted the concat and exposure times to make them shorter so I could see what was going on.  I just get a row of black coloured full stops on the image now (I've highlighted them for clarity in the picture below

 

[Pete6]

- Yes, I do need to revise the instructions..  I will start that today as I have a clean install to do so I'll write down stuff as I do it.

- I guess if it was created by root, it'll be owned by root.  For me this was not an issue since I just logged in as root all the time.  Is this a problem for you?  I would actually like to fix this but it just took so long and other stuff was waiting.

- The dots.  Well html is a pig.  It does not like blank space.  I use the approved method of &nbsp; but that did not work so... I used " ....... " and made them black, the same as my background.  I suspect that if you edited the file it is possible that you somehow lost the black.  html needs <font color="86878a"> infront of the dots and </font> after it.  If this fails, please go back to the original downloaded files and try again with that.  If that fails, post it here and I will try to fix it for you.

- The DS18B20 temperature probe is easy to wire up.  Here is a very crude wiring diagram

     GPIO
  3.3V 1 o--+--------- red ------
                   |  
   nc  2 o      \
                    /4K7
   nc  3 o      \
                    /                    to DS18B20 probe
                    | 
 probe 4 o--+--------- yellow ---
 
 Gnd  5 o-------------black -----

You will need to replace some of the html and php files to accomodate the DS18B20.  The fikes are the same I just commented out the sections for the DS18B20.

Anything else you need, just ask.

Edited May 14, 2020 by Pete6

[jiberjaber]

Thanks Pete, I'm just making a quick 3d print to mount the camera in the end of a coke bottle so I can mount it outside - not the best of solutions but the quickest one I could come up with this afternoon! Next problem is how to get power to it, which might dictate where it gets mounted lol

 

I've not touched the html files but just puzzled why the time remaining to next image has disappeared, only thing I've altered is the concat and exposure timers - I'll be putting them back shortly so will see if that brings back the info on the web page.

 

 

[Pete6]

Jason,  Be a little cautious of reducing the times.  raspistill takes 1m 40s to complete a night image. This was timed Pi prompt to Pi prompt.  I also found, using htop, that a large number of raspistill instances were present (and it fell over) if I reduced the times to much.  Honestly I would leave it as I set it until you have had it running for a few days then make changes cautiously and one at a time.  I have 4 of them running including one Pi Zero W.  They have all run for about 2 weeks without falling over.  

I like the 3D print.  I was going to do that but I got stuck with the transparent part so I re-purposed an old security camera case.  

Powering the device can be done via the USB port of course but that rather limits the length of cable you can use.  If you are handy with a soldering iron then you could put a cheap 12 to 5Volt converter inside the box.  This has two benefits.  

1. You can feed 12V DC a longer distance

2. There is slightly more heat inside the case to help with dew and condensation removal.

Lastly I do not know where you are from but editing /var/www/html/suntimes.php allows you to change the satellite images for those from your geo-location.

Eumetsat: https://eumetview.eumetsat.int/static-images/latestImages/

GOES/NOAA: https://www.nhc.noaa.gov/satellite.php

JMASAT: https://www.data.jma.go.jp/mscweb/data/himawari/index.html

Not every image will work however.  Some are just the wrong size.

Edited May 14, 2020 by Pete6

[jiberjaber]

Thanks Pete.  I only changed the day exposure time to suit testing as I couldn't work out what was working with such a long time between events.  I left the night time settings as was.  I've just set them back to 2 mins for daytime and 5 mins for the video.

The about none sensor version doesn't work, its not coming through as an html file, if I copy the about from the sensor version that works, not sure exactly what's wrong there

It's mounted outside now, I had to reposition if as the link wasn't good and the webpage with the image was timing out (oddly though the video page worked!) After a shift of another 1 meter it's much better. I've got a 5m USB hub which I used to use for the telescope back when I did this in 2013, it seems to be working well to power the Pi Zero W.

I've not got a wide angle lens for this and it's the no-ir version so colours are a little off (a blue sky looks like clouds!) but it's out there and working at the moment, will be interesting to see what night looks like from this position!

 

 

That lovely blue sky looks quite grey in the camera! Perhaps there is a better AWB setting for the no-ir version? (I couldn't find anything with a quick google)

 

 

[Pete6]

52 minutes ago, jiberjaber said:

Thanks Pete.  I only changed the day exposure time to suit testing as I couldn't work out what was working with such a long time between events.  I left the night time settings as was.  I've just set them back to 2 mins for daytime and 5 mins for the video.

The about none sensor version doesn't work, its not coming through as an html file, if I copy the about from the sensor version that works, not sure exactly what's wrong there

It's mounted outside now, I had to reposition if as the link wasn't good and the webpage with the image was timing out (oddly though the video page worked!) After a shift of another 1 meter it's much better. I've got a 5m USB hub which I used to use for the telescope back when I did this in 2013, it seems to be working well to power the Pi Zero W.

I've not got a wide angle lens for this and it's the no-ir version so colours are a little off (a blue sky looks like clouds!) but it's out there and working at the moment, will be interesting to see what night looks like from this position!

Right, Jason.  Let us see if we can make this work.  Before that though.  I love your pics.  They are just great.

The temperature probe files your actually need are:  webcam.php, index.php and about.php.  Of these three webcam.php is the most complicated so we'll leave that till the end.

about.php only differs by having the temperature probe section commented out.  IF you just take the version from the html section of the Gihub zip file and it should start working IF and ony if the temperature probe is working.  This is true for each of the 3 files.

In the directory /home/allsky/tools there is a file called DS18B20-test.sh once you set chmod 755 DS18B20-test.sh this file should check that the DS18B20 is working.  I am sure you have enabled 1-Wire in raspi-config.  If not now would be a good time.

index.php is the same as about.php.  Just replace the file with the one from the Github zip file.  The same rules apply if the DS18B20 ain't working then neither will the file.  

webcam.php is the same again but with the added complication of the line that displays the temperature in the still image.  However so long as (you guessed it) the probe works just replacing the file should work.

All the above has actually worked for me a few times.  From looking at your pics I think it may be worth downloading the latest version that I uploaded.  about.php should diplay V1.09  Get back to me if this does not work as I said.

The NOIR camera does not seem to matter.  I have had both types and sure the colours are a bit off but you can adjust for that in the webcam file.  If you are going to play with the exposure settings it may be worth checking out the raspistill settings page https://www.raspberrypi.org/documentation/raspbian/applications/camera.md   I designed this device to work at night and what I got in the day was a big plus but I didn't spend a lot of time adjusting colour temeratures

The wide angle lens does not matter.  Obviously the more glass you have to let more photons hit the sensor the better.  If it works for you, great.

You may be a little confused about my timezone and, at times my spelling.  Well, I am a Brit, born in London, living in Charlotte North Carolina.  I can tell you live in the UK because of the pic of your guttering and your spelling of colour and your locatoon of Essex - clue there mate.

Pete

 

Edited May 14, 2020 by Pete6

[jiberjaber]

haha yes Pete, sorry I forgot to answer your question earlier  Yes currently residing in Sunny Essex

So at the moment there isn't any glass  on the camera, just cut up coke bottle plastic though it seems to be OK (at the moment, suspect it wont withstand the first hot and wet cycle due to the duct tape holding it all together!) I've got some camera lenses and holders I might have a fettle with later in the week - I think I might have a 2.8mm lens somewhere in that lot.

I haven't bothered with the 18B20, I was in too much of a rush to get it out into the wild to get the soldering iron out lol. 

I've another 10 mins before it switches to night mode, it's quite exciting.  Image is pretty black at the moment so time to check the polar alignment on the scope ready for tonight

 

[Pete6]

That's my favourite part when it switches.  Yeah, I programmed but there is a kid like joy in watching it happen.  I'm 74 years old and 12 mentally.

Not having glass does not matter.  I used a 5 quid dummy camera case and it distorted the optics summat rotten.  However the images are fine.  What you are doing is exactly what I intended although I did not envisage a drinks bottle.  However it works great.  Getting something like this working for under 20 quid is what it is about.  

If you do not have a DS18B20 then unless your open up the files and comment out the modules that run the probe they will not work.  Thanks Linux.  You can however enable the 1-Wire interface in raspi-config and then have a look at my text file to see if raspi-config did actually make the required changes you should be ready to install the temperature probe.

[jiberjaber]

Probably need to adjust the cut over time or phase the ramp of exposure, it seems too bright at the moment now...  

That said, it looks like the camera has caught the handle of the plough!  

[jiberjaber]

OK, very happy with this! Good work Pete!

 

[DaveL59]

46 minutes ago, Pete6 said:

I used a 5 quid dummy camera case and it distorted the optics summat rotten.  However the images are fine. 

My first test with a cctv module I used one of those, but since repurposed a dud dome PTZ that I got for a few quid off the bay and adapted to run upside down (dome up). So long as you can get a good seal (thats the tricky bit) it works fine and gives a clear view with no distortion.