Raspberry Pi AllSky Camera

[wormix]

Just now, discardedastro said:

Oh, that's neat. I've got some aluminium tape which I'm using to wrap my enclosure (aside from the home) to help reject solar load - it'll also completely reject UV. Could layer that and/or black flocking over the PLA and it'd be fine for extended use I'm sure.

That’s a good shout - I have some very thick black felt and aluminium tape that may work. 
 

Although is the issue more heat rather than direct sunlight, as looking at this thread the internals can get pretty toasty! 

[discardedastro]

1 hour ago, wormix said:

That’s a good shout - I have some very thick black felt and aluminium tape that may work. 
 

Although is the issue more heat rather than direct sunlight, as looking at this thread the internals can get pretty toasty! 

And on that note, I've literally just finished making v2, which has some active cooling. Metal fan grilles - this is mostly to keep small crawling things out but might help with dust somewhat, I might add some polypropelene dust filter material to the input port but we'll see how it goes.

The port on the left is the inlet - I wanted to use the rubber mounting supplied with the Noctua fan to minimise vibrations (though in practice they're not really significant enough to worry about much) and so the metal filter is on the inside; on the right it's on the outside to make life easier for screw mounting.

I rejigged the internals somewhat - added some 3M adhesive foam blocks to hold the electronics in place. The Wago blocks could do with a better mounting scheme but it's good enough for now - it isn't exactly moving around much but this keeps most things out of sitting in any water.

The fan driver I'm using is a dumb-as-chocolate little board from Amazon with a thermistor which is sort of hanging out in the middle of the box. This PWM modulates the little Noctua A4x20 fan which can deliver quite a lot of air at full pelt while being quite quiet.

I've covered the top in alu foil - I wanted to check this was all going to work OK before I dismantle the dome again to paint the surface under the dome black. I actually cracked the dome at the edge when assembling it, so might seek to get another dome of the same design (or two, for spares) before I take it off and muck about re-siliconing.

 

[Mognet]

I've had an oddity occur with a Pi Noir webcam (this one) I got it last week and it's all been working fine, then a couple of nights ago it developed a white bloom across the image at night. It survived tinkering with the capture setting, numerous reboots, several power downs, and cable checks. And a case redesign to make sure it wasn't stray light from the Pi Zero inside. It has even made it onto the videos. Then in right clicking and viewing the image prior to posting here, it 'fixed' itself.

Can't see a reason for it in the code either

 

[jiberjaber]

How odd! Mines still going OK

[Mognet]

The mystery white bloom reappeared again tonight. Wondering if it's from auto ISO and exposure time in darker conditions, so I've changed the night and day arguments to a fixed ISO value.

Daytime="-ISO 100 -awb  greyworld -n -w 820 -h 616 -ex auto -o /run/shm/$SCR.jpg"
Nighttime="-ISO 3200 -co 70 -awb greyworld -n -w 820 -h 616 -ex night -ag 9.0 -dg 2.0  -ss 6000000  -o /run/shm/$SCR.jpg"

It's back to behaving how it should with only a little glow on that edge again

Edited August 24, 2020 by Mognet

[wormix]

Finally mounted outside, but with a temporary fix for power for testing whilst I sort the permanent solution. 
 

 

[Mognet]

The mystery white bloom has suddenly returned again, so I'm going to order another Pi Noir camera to see if that's the problem. I've ruled pretty much everything else out except that and the Pi Zero I'm using

[pete_l]

11 hours ago, Mognet said:

The mystery white bloom has suddenly returned again, so I'm going to order another Pi Noir camera to see if that's the problem. I've ruled pretty much everything else out except that and the Pi Zero I'm using

If you're up for a little experimentation I can send you the python code I use to handle my v2 camera on a pi-zero.
This is an example of the sort of image it produces.

It doesn't look great here, but the uncompressed image gets down to about Mag 6.
The bloom at the top left is due to the Moon (north is to the top). Though I am not too happy about all the other edges.

 

[Mognet]

5 hours ago, pete_l said:

If you're up for a little experimentation I can send you the python code I use to handle my v2 camera on a pi-zero.

Thanks. That could be useful as I was looking at writing my own ASC software as I need the programming practice. Just read up on the basics of PHP for the front end and was about to start looking at Python for the back end

[pete_l]

36 minutes ago, Mognet said:

Thanks. That could be useful as I was looking at writing my own ASC software as I need the programming practice. Just read up on the basics of PHP for the front end and was about to start looking at Python for the back end

#!/usr/bin/python3

from __future__ import print_function

import sys
import picamera
from picamera import mmal, mmalobj, exc
from picamera.mmalobj import to_rational
import time
from fractions import Fraction
import datetime as dt


MMAL_PARAMETER_ANALOG_GAIN = mmal.MMAL_PARAMETER_GROUP_CAMERA + 0x59
MMAL_PARAMETER_DIGITAL_GAIN = mmal.MMAL_PARAMETER_GROUP_CAMERA + 0x5A

def set_gain(camera, gain, value):
    """Set the analog gain of a PiCamera.
    
    camera: the picamera.PiCamera() instance you are configuring
    gain: either MMAL_PARAMETER_ANALOG_GAIN or MMAL_PARAMETER_DIGITAL_GAIN
    value: a numeric value that can be converted to a rational number.
    """
    if gain not in [MMAL_PARAMETER_ANALOG_GAIN, MMAL_PARAMETER_DIGITAL_GAIN]:
        raise ValueError("The gain parameter was not valid")
    ret = mmal.mmal_port_parameter_set_rational(cam._camera.control._port, 
                                                    gain,
                                                    to_rational(value))
    if ret == 4:
        raise exc.PiCameraMMALError(ret, "Are you running the latest version of the userland libraries? Gain setting was introduced in late 2017.")
    elif ret != 0:
        raise exc.PiCameraMMALError(ret)

def set_analog_gain(camera, value):
    """Set the gain of a PiCamera object to a given value."""
    set_gain(camera, MMAL_PARAMETER_ANALOG_GAIN, value)

def set_digital_gain(camera, value):
    """Set the digital gain of a PiCamera object to a given value."""
    set_gain(camera, MMAL_PARAMETER_DIGITAL_GAIN, value)

####################### settable values here #######################################

started = dt.datetime.now() # current date and time
date_time = started.strftime("%d%b%Y_%H%M%S")
print("Started at:",date_time)

endtime = dt.time( 7,0,0 )   # end imaging at 7 a.m.
lasttime = dt.datetime.now().time()

# gains settings of (12,4) produce an ISO of 2500 in the exif data
# analog gain, values over 12.0 get set to 12
a_gain = 12

# digital gain, use values of 4 or less, 2 is best for me
d_gain = 2

exposure_us = 10100000    # just over 10 seconds (max)

def fnames():
  global lasttime

  frame_num = 0
  while True:
    ts = dt.datetime.now()
    timestamp = "%04d_%s" % (frame_num,ts.strftime("%d%b%Y_%H%M%S"))
# t  = '%s_%04d' % (timestamp, count)

    thistime = dt.datetime.now().time()

    if lasttime < endtime and thistime >= endtime:
      print("Ending at {} after {} images".format(ts.strftime("%d-%b-%Y %H:%M:%S"), frame_num))
      break

    lasttime = thistime
    yield "long2_%s.png" % timestamp
    frame_num += 1

####### main code starts here ##################

if __name__ == "__main__":
#    with picamera.PiCamera(sensor_mode=4, resolution=(1640,1232),framerate = Fraction(1,10)) as cam:
    with picamera.PiCamera(sensor_mode=3, resolution=(2592,1952),framerate = Fraction(1,11)) as cam:

        cam.color_effects = (128,128)   # monochrome / greyscale

        cam.exposure_mode = 'night'
        cam.drc_strength = 'low'

        cam.shutter_speed = exposure_us

        set_analog_gain(cam, a_gain)
        set_digital_gain(cam, d_gain)      # it is said that values > 4 saturate the image (?)

# use stdout here to get output immediately
        sys.stdout.write("Waiting for gain to be set (Ctrl-C to stop) ")
        sys.stdout.flush()

        try:
            while cam.digital_gain < d_gain:
                sys.stdout.write(".")
                sys.stdout.flush()
                time.sleep(2)
        except KeyboardInterrupt:
            print("Stopping...")

# fix the auto white balance gains at their current values
        g = cam.awb_gains
        cam.awb_mode = "off"
        cam.awb_gains = g
        print("\nAWB gain: ", g)

# fix the analog and digital gains so they won't change during the shots
        cam.exposure_mode = 'off'

        print("Set a/d gains: {}, {}".format(cam.analog_gain, cam.digital_gain))
        print("Starting sequence until ...".format(endtime))
        s1 = time.time()

# without burst = True, 30 frames takes 661 seconds, with it 318s

        cam.capture_sequence(fnames(), use_video_port=False, burst=True, format='png')
#        cam.capture_sequence(fnames(), use_video_port=False, burst=True)
        s2 = time.time()
        print("Finished in  {:.3f}s".format(s2-s1))
        exit

long_shot.py

Run from the terminal as ./long_shot.py This takes back-to-back 10s exposures with almost no gap in between. The exposures are lossless and about 5.2MB each.

Cancel either with Ctrl-C or it ends automatically at 7:00 a.m.

The first image takes up to a minute due to some silliness inside the Pi camera (I'm sure my phone doesn't wait a minute before taking a photo and the IMX219 is a phone camera ;( )
I have tried using the full resolution of the v2 camera (3280x2464) but the software falls over with an out of memory error. Although the raspistill software manages it with ease. This seems to me to be a crock in the picamera module.

Almost none of this is my original work. I am no expert in Python so will not be able to answer any difficult questions.

Edited November 3, 2020 by pete_l

[Mognet]

Replacement camera has been in for a couple of days now and there hasn't been any return of the mystery white bloom, so it looks like it was a dodgy camera module

Now to learn some Python so I can work on my own version. And to source a dome so I can mount it outside instead of pointing it through the conservatory double glazing

 

[jiberjaber]

Well after a while I forgot about my all-sky camera and the SD card was full... so I took it inside to resuscitate is and it never made it back out until just now!  I had taken a backup of home, etc and www but I had to piece together what I had done in the meantime to get it running... I suppose I should really document it but it helped being able to scroll through this thread as an aide memoire !

I need to sort out some internal heating for dew control and also a 12V feed then looking at the FoV now, I think it should be ready to move on to a pole at the end of the garden away from the house....  I'll need to design and print a bracket for that but that's not too hard a task to do I think (though I have seen some designs for pole mounted 3d printed enclosures which might cut down on the weight a bit) 

 

 

[Mognet]

I had to refresh the card in mine after it stopped talking to the network for no apparent reason, and since then I've made the capture code a bit more robust. I wasn't certain that it was adding every frame to the video before, but I'm pretty certain of it now. And I think I've cured the problem where it would create videos of several days or nights run together. It can now safely capture every minute in daytime and two minutes at night on a Pi Zero without getting tangled

Next job is to update the web interface so that every frame can be viewed, and several days (or more) history can be stored. The pages will also benefit from some common PHP and CSS code, which I think is tonight's job

[jiberjaber]

2 minutes ago, Mognet said:

I had to refresh the card in mine after it stopped talking to the network for no apparent reason, and since then I've made the capture code a bit more robust. I wasn't certain that it was adding every frame to the video before, but I'm pretty certain of it now. And I think I've cured the problem where it would create videos of several days or nights run together. It can now safely capture every minute in daytime and two minutes at night on a Pi Zero without getting tangled

Next job is to update the web interface so that every frame can be viewed, and several days (or more) history can be stored. The pages will also benefit from some common PHP and CSS code, which I think is tonight's job

Looking forward to seeing that in action.  I can't remember where I got up to in tinkering but from the video in the back up I used looks like mine last worked 4th Oct before I took it down.

 

[Mognet]

36 minutes ago, jiberjaber said:

Looking forward to seeing that in action

Thanks. I'll publish it once I'm happy with it. The plan is to include an auto deploy script so it can be a single command install too, along with all the other improvements

[jiberjaber]

3 minutes ago, Mognet said:

Thanks. I'll publish it once I'm happy with it. The plan is to include an auto deploy script so it can be a single command install too, along with all the other improvements

That will certainly make life easier!

Edited January 2, 2021 by jiberjaber

[jiberjaber]

On 02/01/2021 at 20:42, Mognet said:

Thanks. I'll publish it once I'm happy with it. The plan is to include an auto deploy script so it can be a single command install too, along with all the other improvements

How are you getting along with your new version? 

 

I'm itching to give it a try - the current one is OK but suffering from a few gremlins that I am reluctant to investigate and fix

 

 

[Mognet]

11 minutes ago, jiberjaber said:

How are you getting along with your new version? 

 

I'm itching to give it a try - the current one is OK but suffering from a few gremlins that I am reluctant to investigate and fix

 

 

I've been tinkering and letting it run to show any gremlins. Have caught a few that might not have shown from standard testing. And found that memory card speed and quality is important! Three rebuilds have happened thanks to rubbish cards

I've got image and video history working nicely. Videos are now smooth, and on a Pi Zero it will successfully capture once per minute daytime and once every two minutes nighttime. I've also taken some stuff from the W3Schools tutorials to handle the differences between PC and mobile displays, and modified some of the image displays. There are other plenty of other changes too. I'll document them and the gotchas when I upload it. That will probably be tomorrow as I just need to finish adjusting the user/owner of the web pages and images in /var/www/html/. I'm changing them from root to nobody.

And I've written an autodeployer! It does almost everything for the setup Have to say that working with Bash is..umm...interesting 🤣

[Mognet]

@jiberjaber Hi Jason. I've uploaded my version onto GitHub, and because there are so many changes I've started a separate thread for it