Another All Sky Camera

[jiberjaber]

Will do.

 

In between the rain here we had a bit of clear sky so managed to see some stars! Focus not as bad as I was expecting, so I think I will leave that alone! I seem to recall it being a bit of a pain with such a fine threaded small lens! I think there must be more heat kicked off the 3B+ as there is no sign of condensation like there was before, but perhaps the more exposed location is allowing more air over the dome too which helps.

[Mognet]

Focus looks pretty good there. I know with mine I ended up settling for a close enough focus set in daytime

The Pi Zero in my conservatory had a CPU temp of 25 at the time your image was taken, and the spare room one recorded 29 degrees. Looks like the Zeroes run cooler than the other models, so that might account for the change in condensation

[Mognet]

A few little defects fixed and tested, and I've added a version history too. I've called it version 1.2 as I hadn't really recorded the updates before

• Fix history creation where an unused period containing only the initial files was being archived in the history section
• Fix minor defect in autodeploy and update that would display an error if enter was pressed with a blank entry instead of a response
• Change autodeploy to only copy the necessary files
• Change autodeploy so that the gps file is created in the execution directory
• Change update to copy LICENSE and README.md too
• Add a link back to the GitHub page and a this version history
• Tidy up on sun times page

And I'll put Hardware Watchdog on all mine soon as a router reboot ten minutes ago has just caused one of them to lose connection 😠

[Mognet]

With some help from Bob (jmrp) on GitHub who discovered the right settings, the code has been changed so that it can capture images every minute at night too. Only problem is that not all cameras can do it. It works with my Pi V2.1 NoIR camera modules and should do with the Pi HQ camera too, but it doesn't and does work with the WaveShare camera module. The solution lay in using --exposure off instead of --exposure night in the nighttime capture. There is a note in the raspistill documentation to say that whether it works or not depends on camera tuning

Edited June 5, 2021 by Mognet
Update with new knowledge

[jiberjaber]

Is there an easy guide to upgrading from the GIT without having to reenter long/lat etc?

[Mognet]

22 minutes ago, jiberjaber said:

Is there an easy guide to upgrading from the GIT without having to reenter long/lat etc?

Hi Jason,

Yes there is. There's a bit about updating here https://github.com/MarkGrimwood/Mognet-All-Sky-Camera-install/wiki#updating First bit is just the plain update, second bit is just to update GPS coordinates

And by coincidence, I made some bug fixes today too. Just minor stuff this time

Edited June 18, 2021 by Mognet

[jiberjaber]

On 18/06/2021 at 18:41, Mognet said:

Hi Jason,

Yes there is. There's a bit about updating here https://github.com/MarkGrimwood/Mognet-All-Sky-Camera-install/wiki#updating First bit is just the plain update, second bit is just to update GPS coordinates

And by coincidence, I made some bug fixes today too. Just minor stuff this time

It's raining Sunday so I'll give it a go then I think - thanks!

 

[jiberjaber]

On 25/06/2021 at 23:33, jiberjaber said:

It's raining Sunday so I'll give it a go then I think - thanks!

 

Finally got round to updating, very easy to do... the SD had ran out of space which prompted me to ssh on and do something! 

Did you add a version number to the about page, seems to be missing on my updated version, just wondering if it got missed on the update?

[Mognet]

Hi Jason.

The latest version is v1.2.4, and that should be visible on the about page. There was a fix a little while ago that would stop an occaisional problem with space clearance. Looks like you might have run into that if your card ran out of space

[jiberjaber]

Yep - I think mine hasn't actually updated based on that!

I've managed to break the main page, the SD ran out of space so I did a "sudo find /var/www/html/history/ -mindepth 1  -mtime +42 -delete" which now means it reports a missing directory and no about link is there anymore.. 

 

I'll give the update another go

 

[jiberjaber]

Think I might have spotted an issue or 2?

Looks like the new about.html doesn't get copied as part of the update?    Fixed with a reboot!

 I also note "no crontab for root" which might be why the file clearup isn't happening?

Y/N is case sensitive so needs a lower case 'y' to work

My missing directory issue is still there though, so the actual about link is lost at the moment (see above post for image)

Quote

This is the update script for Mognet's All Sky Camera
=====================================================
This script updates the code for the all sky camera. It will preserve the current and historical images and videos captured, but will overwrite all the driving code
Note: Any customisations that have been made to the code since the install will be lost!
If you are certain that you want to proceed then answer Y to continue, otherwise N or Ctrl-C to exit
Y
Please enter only Y or N
y
Stopping the cron service
Copying the back end code
'/home/pi/Mognet-All-Sky-Camera-install-main/back-end/dailyupdate.sh' -> '/home/pi/Mognet-All-Sky-Camera/back-end/dailyupdate.sh'
'/home/pi/Mognet-All-Sky-Camera-install-main/back-end/gps' -> '/home/pi/Mognet-All-Sky-Camera/back-end/gps'
'/home/pi/Mognet-All-Sky-Camera-install-main/back-end/mgasc.sh' -> '/home/pi/Mognet-All-Sky-Camera/back-end/mgasc.sh'
'/home/pi/Mognet-All-Sky-Camera-install-main/back-end/pics/capture.sh' -> '/home/pi/Mognet-All-Sky-Camera/back-end/pics/capture.sh'
'/home/pi/Mognet-All-Sky-Camera-install-main/back-end/pics/newmovie.sh' -> '/home/pi/Mognet-All-Sky-Camera/back-end/pics/newmovie.sh'
'/home/pi/Mognet-All-Sky-Camera-install-main/back-end/pics/clearspace.sh' -> '/home/pi/Mognet-All-Sky-Camera/back-end/pics/clearspace.sh'
'/home/pi/Mognet-All-Sky-Camera-install-main/back-end/sunwait/LICENSE' -> '/home/pi/Mognet-All-Sky-Camera/back-end/sunwait/LICENSE'
'/home/pi/Mognet-All-Sky-Camera-install-main/back-end/sunwait/README.md' -> '/home/pi/Mognet-All-Sky-Camera/back-end/sunwait/README.md'
'/home/pi/Mognet-All-Sky-Camera-install-main/back-end/sunwait/USAGE.txt' -> '/home/pi/Mognet-All-Sky-Camera/back-end/sunwait/USAGE.txt'
'/home/pi/Mognet-All-Sky-Camera-install-main/back-end/sunwait/sunwait' -> '/home/pi/Mognet-All-Sky-Camera/back-end/sunwait/sunwait'
'/home/pi/Mognet-All-Sky-Camera-install-main/gpsupdate' -> '/home/pi/Mognet-All-Sky-Camera/gpsupdate'
'/home/pi/Mognet-All-Sky-Camera-install-main/LICENSE' -> '/home/pi/Mognet-All-Sky-Camera/LICENSE'
'/home/pi/Mognet-All-Sky-Camera-install-main/README.md' -> '/home/pi/Mognet-All-Sky-Camera/README.md'
Copying the front end code
'/home/pi/Mognet-All-Sky-Camera-install-main/front-end/about.html' -> '/var/www/html/about.html'
'/home/pi/Mognet-All-Sky-Camera-install-main/front-end/arrowLeftDisabled.gif' -> '/var/www/html/arrowLeftDisabled.gif'
'/home/pi/Mognet-All-Sky-Camera-install-main/front-end/arrowLeftEnabled.gif' -> '/var/www/html/arrowLeftEnabled.gif'
'/home/pi/Mognet-All-Sky-Camera-install-main/front-end/arrowRightDisabled.gif' -> '/var/www/html/arrowRightDisabled.gif'
'/home/pi/Mognet-All-Sky-Camera-install-main/front-end/arrowRightEnabled.gif' -> '/var/www/html/arrowRightEnabled.gif'
'/home/pi/Mognet-All-Sky-Camera-install-main/front-end/currentview.html' -> '/var/www/html/currentview.html'
'/home/pi/Mognet-All-Sky-Camera-install-main/front-end/fileread.php' -> '/var/www/html/fileread.php'
'/home/pi/Mognet-All-Sky-Camera-install-main/front-end/imagedisplay.php' -> '/var/www/html/imagedisplay.php'
'/home/pi/Mognet-All-Sky-Camera-install-main/front-end/index.php' -> '/var/www/html/index.php'
'/home/pi/Mognet-All-Sky-Camera-install-main/front-end/liveview.html' -> '/var/www/html/liveview.html'
'/home/pi/Mognet-All-Sky-Camera-install-main/front-end/mgasc.css' -> '/var/www/html/mgasc.css'
'/home/pi/Mognet-All-Sky-Camera-install-main/front-end/shotlist.php' -> '/var/www/html/shotlist.php'
'/home/pi/Mognet-All-Sky-Camera-install-main/front-end/suntimes.php' -> '/var/www/html/suntimes.php'
'/home/pi/Mognet-All-Sky-Camera-install-main/front-end/webcam.php' -> '/var/www/html/webcam.php'
Recalculating times and restarting the all sky camera

E V E N T                                      HH:MM OFFSET
---------------------------------------------  ----- ------
Astronomical Rise (18 degrees below horizon):  --:-- (Midnight sun) -00:10
Astronomical Rise (Real)                    :  --:-- (Midnight sun)
Nautical Rise (12 degrees below horizon)    :  03:07 -00:10
Nautical Rise (Real)                        :  03:17
Civil Rise (6 degrees below horizon)        :  04:09 -00:10
Civil Rise (Real)                           :  04:19
CAMERA switches from Night to Day           :  04:11 -00:08
VIDEO switches from Night to Day            :  04:16 -00:03
Sunrise (0 degrees below horizon)           :  05:01 00:00
Sunrise (Real)                              :  05:01
Sunset (0 degrees below horizon)            :  21:07 00:00
Sunset (Real)                               :  21:07
Civil Set (6 degrees below horizon)         :  21:49 -00:00
Civil Set (Real)                            :  21:49
CAMERA switches from Day to Night           :  21:58 -00:09
VIDEO switches from Day to Night            :  22:01 -00:12
Nautical Set (12 degrees below horizon)     :  22:51 -00:00
Nautical Set (Real)                         :  22:51
Astronomical Set (18 degrees below horizon) :  --:-- (Midnight sun) -00:00
Astronomical Set (Real)                     :  --:-- (Midnight sun)

Wed 21 Jul 00:38:26 BST 2021
no crontab for root
pi@allsky:~/Mognet-All-Sky-Camera-install-main 

 

Edited July 20, 2021 by jiberjaber
updated after a reboot...

[Mognet]

1 hour ago, jiberjaber said:

Yep - I think mine hasn't actually updated based on that!

I've managed to break the main page, the SD ran out of space so I did a "sudo find /var/www/html/history/ -mindepth 1  -mtime +42 -delete" which now means it reports a missing directory and no about link is there anymore.. 

 

I'll give the update another go

 

The error trapping on the main page isn't great, so the message of "couldn't open info file" means it's looking in a history subdirectory and not finding that file. I wouldn't be surprised if the directory is empty too, but the directory itself hasn't been removed. It was a problem I was seeing on one of mine before some of the recent fixes (v1.2.2 I think fixed it). The directory in question looks like it's /var/www/html/history/1623186661-night/ There might be others too, and they will probably be the oldest ones

The find command is new to me. I'll have a play with that tomorrow in between job hunting and learning Python

57 minutes ago, jiberjaber said:

Think I might have spotted an issue or 2?

Looks like the new about.html doesn't get copied as part of the update?

 I also note "no crontab for root" which might be why the file clearup isn't happening?

Y/N is case sensitive so needs a lower case 'y' to work 

Just checked the update script on GItHub (https://github.com/MarkGrimwood/Mognet-All-Sky-Camera-install/blob/main/update) and it should copy the entire contents of the front-end directory to /var/www/html/ while preserving the history. (Just seen your update) It should handle Y or y too

Not sure about the "No crontab for root" message. I think I've seen it on some of the clean installs

 

I realise that 'works for me' doesn't mean it works for everyone else. Might need to see what messages are actually coming out of the update script. Might also be a good idea to add version number reporting to the update and autodeploy scripts too in the next update

Edited July 21, 2021 by Mognet
Update based on updated info

[jiberjaber]

Yep - that's got it, found about 4 empty directories (I think I was using find but only reporting files initially rather than files & directories...)

index page back to normal now, will see how the file space goes when I get close to the 2GB mark again.

I found out why I wasn't updating, each time I wget the file I hadn't removed the old zip file so it was just downloading a main(1) ... main(2) etc and then when I unzipped I was just unzipping the oldest file called main.zip rather than the new download. Might be worth adding a step to the update instructions to make sure there isn't an already existing main.zip before wget'ing the next one.

Anyway I am now on v1.2.4

Good luck on the job hunting...

[Mognet]

That's brilliant.

The space clearance/empty directory problem should be fixed in this version along with a little problem in creating new periods that one of mine was having issues with.  It looked like it was just a timing thing between the capture and new movie scripts. Space clearance now happens three times a day; twice for the new periods, and a third at noon when the timings are regenerated. And that third one also reports to update.log in ~/Mognet-All-Sky-Camera/back-end/ This latest version should be stable. I've had it running for about three weeks and haven't managed to break it yet

I'll update I've updated the instructions so that removal of the downloaded files and installer directory are included, and added version messages to the autodeploy and update scripts too along with a version history in the README.md too

2 hours ago, jiberjaber said:

Good luck on the job hunting...

And thanks. Job hunting is tedious. Twenty years experience in IT; 3 as a dev and 17 as an automated software tester, and I've only managed to get one interview out of 80+ applications so far.

Edited July 21, 2021 by Mognet
Have done the code update now

[jiberjaber]

On 21/07/2021 at 14:45, Mognet said:

That's brilliant.

The space clearance/empty directory problem should be fixed in this version along with a little problem in creating new periods that one of mine was having issues with.  It looked like it was just a timing thing between the capture and new movie scripts. Space clearance now happens three times a day; twice for the new periods, and a third at noon when the timings are regenerated. And that third one also reports to update.log in ~/Mognet-All-Sky-Camera/back-end/ This latest version should be stable. I've had it running for about three weeks and haven't managed to break it yet

I'll update I've updated the instructions so that removal of the downloaded files and installer directory are included, and added version messages to the autodeploy and update scripts too along with a version history in the README.md too

And thanks. Job hunting is tedious. Twenty years experience in IT; 3 as a dev and 17 as an automated software tester, and I've only managed to get one interview out of 80+ applications so far.

Yeah - I had similar issues, mostly computer says no - it's quite different to when I was last looking for a job! 21 years with the same company in consultancy, redundancy and then the pandemic put my job hunting on hold and I did a 90 degrees shift to teach people how to cycle, most rewarding job I've had, but it doesn't pay the bills so I suspect I'll have to get a "proper" job at some point.

 

Anyway, back to the all-sky... I noticed the version history is't up to date after install of the latest - I checked the git hub and the version text isn't aligned with the current version (it only goes up to 1.2.4) - no big problem but thought I'd let you know

I've made some progress of late whilst I've been sitting around recovering from an operation. I've not got a sky temperature on my weather station with a view to cloud detection and thought it would be good to get it on to the all-sky so I could compare temperatures with the view. Works well so far, data arrives by MQTT to the all-sky and then I add it on to the banner text so I can then review the movie to see the temp vrs view. I have contemplated putting some sort of grid overlay on so I can try and calibrate the view to temperature and then use the numbers to map to cloud cover. Not progressed that much as I keep jumping from project to project LOL - dew heater for the all-sky is the next on the list! 

Shame it was a full moon last night, such clear skies! 

 

 

[Mognet]

Looks good. Glad you're getting on with it and you got your MQTT input sorted

I know what you mean about the clear skies and full moon. Went out to check the skies last night and Jupiter was sparking, but the full moon washed everything else out. Tonight had been promising until about ten minutes ago when clouds rolled in again

6 hours ago, jiberjaber said:

I checked the git hub and the version text isn't aligned with the current version (it only goes up to 1.2.4)

I didn't bother updating the about page with the last change as v1.2.5 only affected the install scripts. They'll be realigned with the next change

[Mognet]

Have made a minor update to take it to v1.2.6

This time I found that surplus jpg files were not being deleted before new movies were created. I'd turned off the power here without shutting down the cameras cleanly first and a jpg file wasn't added into the daytime movie that day, but was left hanging around. That meant it kept getting added in at the start of every daytime movie as that part uses a wildcard add rather than named file. The code to clear the surplus files relied on a wildcard which doesn't seem to work with a file delete in Bash, even though it does from the command line

[Mognet]

The v3 Raspberry Pi camera has just been announced, which needs libcamera instead of raspistill. I have a v3 camera on order so I can investigate and update the code as required. I knew raspistill was being deprecated so it's time to fix things

Also currently investigating a slowdown in capture that's led to images being skipped on the Pi Zero when previously they had been fine. It possibly stems from copying the videos using the cp command, but has happened on new cards a week after initial install

[Mognet]

I'm testing an update that handles the new versions of Raspberry OS and the v3 camera, and can hopefully sort out some code that will handle the older cameras with no additional user intervention. In the meantime I'm trying to find the source of an intermittent bug that causes the generation of a new video to fail leading to a 404 error when trying to view the video.

If you can't wait until then, the test code, which uses libcamera-still instead of raspistill, is in the update_for_v3camera branch on GitHub.