Raspberry Pi AllSky Camera

[jiberjaber]

2 minutes ago, DaveL59 said:

My first test with a cctv module I used one of those, but since repurposed a dud dome PTZ that I got for a few quid off the bay and adapted to run upside down (dome up). So long as you can get a good seal (thats the tricky bit) it works fine and gives a clear view with no distortion.

Yep thats my plan, I have asked a mate who runs a CCTV company if he had any spare but I just missed him chucking some out last week! Never mind.  

 

[Pete6]

Hey, that looks fantastic.  It is pretty much about what I get but where I live is very light polluted.  Once I can get out of the house (COVID) I shall take mine down to the observatory in darkest South Carolina http://charlotteastronomers.org/

I was aiming to get down to mag 3.5 and from your pic it looks like you are slightly better than that.  I see one star Taiyangshou  in UMa that is mag 3.65.

[DaveL59]

1 minute ago, jiberjaber said:

Yep thats my plan, I have asked a mate who runs a CCTV company if he had any spare but I just missed him chucking some out last week! Never mind.  

 

doh, always the way huh. Don't go for a smoked dome though, but then you can always get a clear replacement which is what I did.

If it helps this is what I did with my cheapo build

 

[jiberjaber]

19 minutes ago, Pete6 said:

Hey, that looks fantastic.  It is pretty much about what I get but where I live is very light polluted.  Once I can get out of the house (COVID) I shall take mine down to the observatory in darkest South Carolina http://charlotteastronomers.org/

I was aiming to get down to mag 3.5 and from your pic it looks like you are slightly better than that.  I see one star Taiyangshou  in UMa that is mag 3.65.

It's an unbelievable clear night tonight! Not the usual skies  

I was expecting to have to fettle with the focus but it seems fairly good..

[Pete6]

For me the stumbling block was always the case.  When my son replaced one of his old analogue security cameras I got the case.  I had already built Thomas Jacquin's Allsky Camera that used a ZWO ASI225MC camera but that was too much money to sink into such a project.  Anyway it was my guide camera.  This project fell out of that exercise.

[jiberjaber]

Ah - so the webcam part of capturing and showing on the web page working fine, but the concat and night movie stopped at 22:20 which is the time for Nautical Set.  I've rebooted but no webcam jpg appearing in the pics directory? Seems the webcam.jpg file is being created in the /run/shm/ but is 45k in size and seems to be copied and then deleted by the copy script??

[Pete6]

Not sure exactly what is happening but, newnightmovie.sh will only run once automatically when crontab tells it to.  In order to stop white flashes getting into the videos I use concat.sh and newday(night)video.sh to reject any incoming webcam-tttt.jpg file that is less than 110Kb in size.  If there is no files greater than 110Kb it will never start the video.  I am seeing exactly the same thing on my Pi Zero W system but not on the Pi 3 or Pi 4 systems.  Currently I have no idea why this should be.  I will find out but it may take me a few days as I can only check it at Civil Sunset when it triggers.  I only discovered the problem last night.  I thought it might have been an anomaly.  It did it again on just the Pi Zero W again an hour ago today and now you are also reporting in.

I need this fixed as soon as I can.

Edited May 15, 2020 by Pete6

[jiberjaber]

Given the file which is 45k still has a valid image in it, perhaps the low total light is resulting in a smaller file? 

[Pete6]

Not sure that is the problem.  If a jpg is all one colour then it will be a very small file due to the way the compression algorithm works.  I was seeing a number of 100% white images, mainly at changeover from day to night that were causing annoying white flashes in the videos so I took them out.  However I have seen some evidence that the Pi Zero W, which has a single core CPU and is somewhat slower than the Pi 3 and 4 may need more time to complete a night image.  I have increased the variable " shotN " from 4 to 5 minutes which will give the Pi Zero more time to complete its image.  My explanation for this is, with shotN=4 and using htop I see multiple instances of raspistill indicating that it is starting a new image before it has completed the current one.  This seems unique to the Pi Zero.  If I increase the shotN time to 5 (minutes between shots) then this does not seem to happen.

I believe this to be a uniquely Pi Zero problem.  During development I obviously wanted the fastest image rate I could get and on my development Pi 3B I found that 4 minutes was the lowest I could reliably get shotN.

Pleased leave this with me for a day or so as I wish to test what I have done by just leaving the Pi Zero W running without me fiddling with it.  This last has been a feature of development.  Just leaving it alone.  This is hard for me.  I just want it to work.

If you wished to edit /home/allsky/webcam and set shotN to 5 then it may fix your problem too.  Obviously, it is up to you.

[DaveL59]

the other fun thing you'll be having to deal with, keeping the dome clean for the best image and of course

the bugs that think it's a great place to set up a web etc

[jiberjaber]

7 hours ago, Pete6 said:

Not sure that is the problem.  If a jpg is all one colour then it will be a very small file due to the way the compression algorithm works.  I was seeing a number of 100% white images, mainly at changeover from day to night that were causing annoying white flashes in the videos so I took them out.  However I have seen some evidence that the Pi Zero W, which has a single core CPU and is somewhat slower than the Pi 3 and 4 may need more time to complete a night image.  I have increased the variable " shotN " from 4 to 5 minutes which will give the Pi Zero more time to complete its image.  My explanation for this is, with shotN=4 and using htop I see multiple instances of raspistill indicating that it is starting a new image before it has completed the current one.  This seems unique to the Pi Zero.  If I increase the shotN time to 5 (minutes between shots) then this does not seem to happen.

I believe this to be a uniquely Pi Zero problem.  During development I obviously wanted the fastest image rate I could get and on my development Pi 3B I found that 4 minutes was the lowest I could reliably get shotN.

Pleased leave this with me for a day or so as I wish to test what I have done by just leaving the Pi Zero W running without me fiddling with it.  This last has been a feature of development.  Just leaving it alone.  This is hard for me.  I just want it to work.

If you wished to edit /home/allsky/webcam and set shotN to 5 then it may fix your problem too.  Obviously, it is up to you.

No probs, happy to up my night time shot time as well to test, I'm just playing with a backup script to copy the files to a date based directory and then rsync it over my network - probably needs running before the files are deleted for the previous session (i.e. on the transition from night to day mode).  Where would be a good place to grab the old data before it is removed?

39 minutes ago, DaveL59 said:

the other fun thing you'll be having to deal with, keeping the dome clean for the best image and of course

the bugs that think it's a great place to set up a web etc

Yes - I have the same problem with my security camera, it often has white streaks across it where the IR bounces back off spider webs! Luckily the camera can be reached from one of our windows with a duster on a stick! Might be more of a challenge with the all sky camera location though!

 

[DaveL59]

Just now, jiberjaber said:

 

Yes - I have the same problem with my security camera, it often has white streaks across it where the IR bounces back off spider webs! Luckily the camera can be reached from one of our windows with a duster on a stick! Might be more of a challenge with the all sky camera location though!

 

yeah a common problem with general CCTV if the IR is built into the camera but handy you can reach them easily. I've only 2 that have that issue as the others I have separate IR which avoids that, tho occasionally you get a spooky mega spider shadow being cast  

My skycam's are on the shed up by the roof, so a 2-step gets me high enough to give them a wipe over and I tend to give them a RainX finish to help the rain slip away from the view area. Sort of works, in the main.

[jiberjaber]

3 minutes ago, DaveL59 said:

yeah a common problem with general CCTV if the IR is built into the camera but handy you can reach them easily. I've only 2 that have that issue as the others I have separate IR which avoids that, tho occasionally you get a spooky mega spider shadow being cast  

My skycam's are on the shed up by the roof, so a 2-step gets me high enough to give them a wipe over and I tend to give them a RainX finish to help the rain slip away from the view area. Sort of works, in the main.

Yeah - I am not sure final location at the moment, will depend on how wide a lens I can get working, current location is good as I can just see the TV aerial which gives me an indication it's still working. With a wider lens I might need to relocate.  Current idea is on top of the weather station but thats on the end of a scaff pole so less easy to get to to maintain, plus side is it might be self cleaning from the weather! Obv subject to final enclosure too!  

[jiberjaber]

I've been playing with the all sky cam and was interested to see if changing the capture resolution increased the FoV as I had noticed that on other software.  The bad news is that it seems to stay the same when I switch up to the largest resolution that still supports 2x2 binning. I also adjusted the capture jpg quality to 100% - it's not like I am short of storage space on my 64GB card that's in it and everything is now copied off to a network drive anyhow.

I've exposed it on to the internet for a short while if anyone is interested in taking a look

http://jiberjaber.dyndns.org/index.html

 

[Pete6]

I found during development that increasing the resolution slowed down night time image processing.  That is why I left it at 800x600 and 4 minute interval exposures.  Initially I  was reluctant to leave maximum resolution as it meant dropping pixels but it kept failing so I left it at 800x600 and it worked.  I intend to try using a Pi 4 with higher res but the thing does what I need it to do.

I am VERY interested to hear the experiences of others who use the software.  

[jiberjaber]

I've reverted back to 800x600 but left the quality at 100. Also added a small bit to the about page to give an indication of load

Quote

$loads=sys_getloadavg();
echo "Current Load $loads[0], $loads[1]";

I've also inserted my backup scripts in to the newdaymovie/newnightmovie files, one for day and one for night.

Here's an example of the night one, it seems to work when run itself, interesting to see if it all works OK when called from the newdaymovie/newnightmovie.  I've inserted it inbetween the move of new MP4 to replace the old and the clean up for next time parts

#backup DAY files
/bin/sh /home/allsky/backupnight.sh
 

 

Quote

#!/bin/bash
#backupnight.sh
#creates a directory with the current date in YYMMDD format with session type (D/N) appended
#edit HOME variable with path to backup directory location
#copies all jpg and movies relating to that session
#optional rsync to remote location
#Config Variables
EXE="/usr/bin/$BIN" # Directory of camera control executable
HOME="/home/allsky" # Working directory
backupDIR="backup"
remoteDIR="/mnt/astro/allskyimages"
#-------------------------------------------------------------------
#
# "$HOME/$backupDIR/$(date +%y%m%d)_N"
#create backup location ignore any errors
echo "Creating backup directory, $HOME/$backupDIR/$(date +%y%m%d)_N"
/bin/mkdir -p "$HOME/$backupDIR"
/bin/mkdir -p "$HOME/$backupDIR/$(date +%y%m%d)_N"
#copy files
echo "Copying files"
cp -r "$HOME/pics/night/"*.jpg "$HOME/$backupDIR/$(date +%y%m%d)_N/"
cp -r "$HOME/pics/movienight.mp4" "$HOME/$backupDIR/$(date +%y%m%d)_N/$(date +%y%m%d_%H%M%S)_movienight.mp4"
#remote copy update
#/bin/rsync
rsync -avzh --progress --recursive --ignore-existing "$HOME/$backupDIR" $remoteDIR

# end
 

 

 

Looking forward to seeing if the increase from 4 to 5 gives an all night movie.

I was thinking, whilst watching the day movie, it would be nice to have some idea of time as the sky moves above, perhaps the banner that is in the webpage image.  I think when I looked at the webpage, the time banner image is created on the fly and not saved?

[Pete6]

Nice mods.  I like it.  May I please incorporate this into the Github files for distribution? 

I have set my Pi Zero W to take images at 5 minute intervals at night too.

I too would like to incorporate the time into the videos.  I am not sure how to do it though.  Obviously I have the time of each image but I am not sure how to get that neatly and without jumping into the image or possibly under it whilst still on screen.

The other thing I'd like is to display the time in minutes and seconds until the next still image is displayed.  I have all the data available in index.html.  I even put the text below the image.  I have not the faintest idea how to accomplish this.

Edited May 15, 2020 by Pete6

[jiberjaber]

haha - thats a great image, I think if it tried that on mine the camera housing would fall apart!

It looks like the the image with the banner is made as the page is created so might be able to use that as the source for the webcam files?

More than happy for you to incorporate the changes, It looks like the backup script didn't work at this end on the newnightmovie.sh execution, suspect it is a permissions issue. Might have a look tomorrow! 

Another great clear night here!

[Pete6]

I can tell you that I a glad that damn great bird landed on the camera housing and not on my bald head. lol.

The text is written into the image in webcam.php.  You should see the bits where I have changed it to accomodate  Outside Temperature and CPU Temp

Thank you I shall change about.php accordingly

[jiberjaber]

13 hours ago, Pete6 said:

I can tell you that I a glad that damn great bird landed on the camera housing and not on my bald head. lol.

The text is written into the image in webcam.php.  You should see the bits where I have changed it to accomodate  Outside Temperature and CPU Temp

Thank you I shall change about.php accordingly

Might want to add a link to the github page as well?  

I'm going to give a M12 lens a go, I thought taking one off an existing board camera and repurposing it might work but the mounting centers are different. However, I have come across a 3d printed M12 mount for the V2 camera which I will give a go, I was hoping someone might have already done a 3D printable version, so happy to try it out  

 

https://www.raspberrypi.org/forums/viewtopic.php?t=233466

 

[Pete6]

I posted the Github link in my OP but that has probably not going to get read that much.  You are right.  Here it is again https://github.com/rederikus/AllSky-Camera-Raspberry-Pi

For my camera I used a normal Raspberry Pi Camera V2 with the fixed focus lens. I replaced that with an Arducam device that has the same IMX219 sensor but comes with an M12 lens mount and a 160 degree lens.  I then replaced the lens on that with a 185 degree M12 lens.

I was an expensive route with lots of parts bought but unused.  However it did work in the end.

[jiberjaber]

58 minutes ago, Pete6 said:

I posted the Github link in my OP but that has probably not going to get read that much.  You are right.  Here it is again https://github.com/rederikus/AllSky-Camera-Raspberry-Pi

For my camera I used a normal Raspberry Pi Camera V2 with the fixed focus lens. I replaced that with an Arducam device that has the same IMX219 sensor but comes with an M12 lens mount and a 160 degree lens.  I then replaced the lens on that with a 185 degree M12 lens.

I was an expensive route with lots of parts bought but unused.  However it did work in the end.

Hi Pete - sorry I was referring to the about page - add a link there  

[jiberjaber]

I've found a wide angle lens in my stock of old first person view cameras I used to fly with. Not 180 FoV but wider than the stock lens. The M12 lens mount worked so I am printing an end for the bottle camera. Just need to work out how I am going to focus it, suspect I'll have to disconnect the camera and bring it in the house, hook it up on HDMI and then twiddle whilst looking at some clouds LOL  I'll have to remove it to fit the new end anyway, so shouldn't be too much of a problem. More of a problem is a screen for the lens, suspect I might just leave this one with a smear of grease around the end of the lens thread.

Will know more in an hours time when the print has finished  

[Pete6]

Well, I made your changes and I have put the new about page up on Github with all the rest of my files.  Was this not what you meant?

[Pete6]

1 minute ago, jiberjaber said:

I've found a wide angle lens in my stock of old first person view cameras I used to fly with. Not 180 FoV but wider than the stock lens. The M12 lens mount worked so I am printing an end for the bottle camera. Just need to work out how I am going to focus it, suspect I'll have to disconnect the camera and bring it in the house, hook it up on HDMI and then twiddle whilst looking at some clouds LOL  I'll have to remove it to fit the new end anyway, so shouldn't be too much of a problem. More of a problem is a screen for the lens, suspect I might just leave this one with a smear of grease around the end of the lens thread.

Will know more in an hours time when the print has finished  

That sounds good Jason.  My son does o lot of FP flying.  He usually comes home with a pile of wrecked planes and drones.

To focus a lens mounted on the camera I have found the easiest way is to connect the console to the mini-hdmi port, shut down cron with

service cron stop

and then use

raspistill -k

to focus my camera. when you are done restart cron or reboot the Pi.