jiberjaber

I think I was looking more at what format the data needed to be in order to get it in to WeeWX.  Looks like there are two approaches, a serial way (the arduino-pws way) and a flatfile way (your solar powered example link)

If the flat file works, then you could implement how you get your data from the ESP to the WeeWX any way you fancied...  just need to be sure it doesn't cause problems with getting the data for your proposed wall dials.

I know I am going to sound like a stuck record here, but MQTT would allow the real time data to be used by multiple devices, you could have a simple MQTT client in python creating a flat file for WeeWX and your wall dials could subscribe to the topics that interest them and get the data real time.

For example, on my all-sky camera PI I have a script that gets the weather data and puts it in a file for use by the camera to create the header...

Quote

#!/bin/bash
#mqttwx.sh
#launches mosquitto_sub to listen for temperature, wind and direction for 1 time
#this is run by cron every 2 mins

#Config Variables
EXE="/usr/bin/$BIN" # Directory of camera control executable
HOME="/home/allsky" # Working directory
#-------------------------------------------------------------------
#
# Listen for temperature
mosquitto_sub -h 192.168.1.6 -t home/garden/TempC -C 1 > "$HOME/wxtemp.tmp" &
# Listen for wind speed
mosquitto_sub -h 192.168.1.6 -t home/garden/wind/speed -C 1 > "$HOME/wxspeed.tmp" &
# Listen for wind direction
mosquitto_sub -h 192.168.1.6 -t home/garden/wind/dirC -C 1 > "$HOME/wxdir.tmp"

# Copy temp files over existing ones
mv "$HOME/wxtemp.tmp"  "$HOME/wxtemp"
mv "$HOME/wxspeed.tmp"  "$HOME/wxspeed"
mv "$HOME/wxdir.tmp"  "$HOME/wxdir"

# Log time of update to wxupdate when debuging
#
echo "$(date +%y)-$(date +%m)-$(date +%d) $(date +%H):$(date +%M)" >"$HOME/wxupdate"
# end

28 minutes ago, Gina said:

I think I prefer the WeeWX approach but I don't know if it is possible to make the ESP look like one of the commercial stations. 

I have yet to find something which will work as I would like.  The ESP as web server is interesting though complicated and would require quite a lot of work to add the wind sensors and rain gauge.

Another problem may be that I was thinking of having 2 ESPs in different places to cover all the WS sensors.

Maybe I need to rethink the whole thing.

A bit of a quick google on Arduino and Weeex gives quite a few options Gina. Here's one, though you might want to look through the google results incase there's one that suits you better (this is a few years since last updated)

https://github.com/rkaczorek/arduino-pws

2 minutes ago, Gina said:

I certainly think MQTT is OTT for what I want.

I agree that I could probably use KStars/Ekos/INDI but that doesn't really fit the bill as I see it.  It's great for astro but my thinking is that a weather station is rather different.  If linking the weather to obsy roof control I can see the advantage but I don't think I shall be doing that - just a rain sensor will take care of closing the roof if it rains.  I know I could use an RPi (maybe plus ESP to give enough IO pins) and get the data indoors that way.

I thought I read that one or two on here had weather station software running on their indoor computers that didn't use Ekos etc.

Maybe a look through weeex ?

There's also this if it's not all about indi, though you might have to reverse engineer how the sensors are connected to the application and then replicate that on you ESP station implementation..
http://www.weewx.com/

Just now, JamesF said:

I can see the benefits of MQTT etc., but I think it's possibly over the top for what you want, Gina.  Where it would probably come in really useful is if you have all sorts of things that need to happen based on the data from the weather station.

As you say though, there must be some relatively simple software that reads the sensors and either writes it to some application running on the network elsewhere, or makes it available for applications to read over the network.  I wonder if Wim's weather station project might give you a good head start:

https://stargazerslounge.com/topic/345153-indi-weather-station/

His software is on github.

James

Yes - if it's just for integration in to Ekos etc, then I agree certainly.  Not withstanding the limitations on space inside the ESP and any time constraints, there probably no reason why you couldn't incorporate this and if at a later date needed more than just sending stuff to indi, then the extra could be added (eg: I have MQTT, Twitter and Wunderground uploads from my ESP8266 weather station - also toying with indi uplink as well)

No - it's probably my bad way of explaining one of many ways to get the data off the ESP to somewhere else!  

I went down the MQTT route because so much stuff uses it today for IOT so thought I might get some advantage out of using the same approach, this works well as my house lighting uses MQTT also.

So here's how mine works:

Sensors ---ESP -.-.-.-.-.-.-.-.-.-.-Wifi AccessPoint -.-.-.-.-.-.-.-.-.-.- Linux Box

On the Linux Box I run a MQTT broker (Mosquitto), NodeRed and an Influx database. 

The ESP read and processes the sensors and at certain times (every 5s or every minute etc) sends a MQTT message which identifies where the data reading is from, what the data reading is and also it's value. so that might be:

where = home/garden/    what=wind/speed_2m = value=8.44

on MQTT this is known as a 'topic' that looks like home/garden/wind/speed_2m and will have a value 8.44

In addition to the MQTT the ESP also sends a tweet every couple of hours and uploads to wunderground every couple of minutes.

https://www.wunderground.com/dashboard/pws/ICHELMSF35

The broker on the linux box receives all the broadcasts

Now say I am a display in my study, I am interested in the wind speed every 2 mins, I would connect to the broker, and subscribe to the 'topic' home/garden/wind/speed_2m Each time there is a publish from the wxstation of that topic, it will also then be sent by the broker to the display in the study which will then act upon it.

NodeRed is like a graphical programming language which I use to do some other stuff around the house, this connects to the broker using the same subscribe approach for various topics and acts on the topics when received, for example my Nodered subscribes to all the wxstation's topics and stores them in a database when received.  Because of the way the topics are structured, you can subscribe using a wildcard to pick up all of them, so for example "home/garden/wind/*" will pick up all garden wind values (so speed, gust, direction etc). There is no specification of what you use for your topics but using something that makes sense and allows for future groupings is a good idea.

Once the data is in the database (Influxdb) I can then also use their tools to create a dashboard showing the weather data (for this I use Grafana). This could probably be embedded in a web page somehow... Here's what that looks like:

There is probably some software to do this for just a weather station - it probably uses similar components - I'm just a bad hacker pulling together bits with a little understanding, I am sure a proper developer would know exactly how to describe and do better what I have

So I guess having a think about what data, where you want it to be and how you want to use it (now and in the future) would be a good start to work out the best approach for you, however I can recommend MQTT in some form as it is well supported for 'IOT devices' which the weather station is to some degree.. 

Here's some more stuff on MQTT

https://www.hivemq.com/blog/mqtt-essentials-part-1-introducing-mqtt/

Also this chap is quite useful for info on more than just MQTT (also devices around the house etc) https://tech.scargill.net/

Another option might be looking at some of the home automation software which might give you a way of sending the data (in their format) and using/presenting it.  Home assistant or openhab etc... 

To receive the data from the wx station?

If so, then it also depends on what you plan to send from the WX station.

Here, I use something called MQTT which is a protocol that allows me to describe what and where and the value eg: home/garden/rainhourly = 0.1mm or home/garden/temp=22 or home/bedroom/temp = 24 etc 

It's a publish and subscribe model which needs something called a broker, this runs on one of my linux boxes and is a hub that all devices broadcast to and that other things can subscribe through, so my little weather display here in the study subscribes to all the home/garden stuff and also the home/*rooms* stuff so it can display all the weather plus temp/humidity in the rooms etc...

Is that what you were after?

I am not up to speed with any dedicated weather station receiving programmes though I am sure there might be one somewhere?

6 minutes ago, Gina said:

I have the ESP8266 NodeMCU V1.0 ESP-12E WiFI Module on a breadboard ready to connect to for testing.  I have the information on programming this so that's alright, I think.

Now to the nitty-gritty... Can someone please tell me how to read the WiFi signal from my computer (Linux Mint).  For astro I use KStars/Ekos but the far end uses RPi with INDI - a totally different system.  I know several of you use these ESP devices in your weather stations so I would appreciate help here please and not try to "reinvent the wheel".

Not sure I understand what you are asking Gina??

Do you want to know what settings you are using on the linux machine?

Wanting to connect the ESP to a wifi access point?

I can probably help on both... 

12 minutes ago, Gina said:

Thanks Jason 😀  Looks like I can use that module then.

Well certainly as a proof of concept, I'd probably give it a try on breadboard to ensure as some of the pins might still be reserved or operate in an odd way which might not be obvious from a quick read of the pin outs... I've never used that many IO on one before but I have had issues with shifting around one-wire or SPI stuff away from the dedicated pins... 

I'd avoid using GPIO1 & 3 (Serial port) and 16, 15 and 2 as they are used for sleep mode wake up/boot/reset

So that leaves 6,7,11,8,9,10,12,13,14,0,4,5 as potential, this again is limited further if you need to also use hardware PWM or the SPI bus.

I decided to make the jump from skychart today as it doesn't seem to be updated any more and was looking at SkySafari so this was very timely! Thanks....

I couldn't quiet see the need for the pro version (I usually just use these apps to confirm what I'm looking at with my eyes ) so I went with the Plus version for just over £6  

When you get happy with the scope location, it will help to put some marks on the ground where the tripod stands to help get the same orientation when setting out.  On my patio I used a drill to put some indents in, I had the same on my decking but it was more sensitive to movement on there. I found it helped a lot... 

6 hours ago, Mognet said:

I took a different approach and replaced the fread usage with a series of fgets. Might be my history of processing sequential files on a mainframe showing though!

An example from webcam.php. Similar code changes are required in suntimes.php, index.php and about.php. Index.html also contains the fread code, but it looks like the php section in there isn't used, so I've commented that out

// Get the sunset time from the daily file
$myfile = fopen("/home/allsky/daily", "r") or die("Unable to open file!");

$ADawn=substr(fgets($myfile),0,5);
$NDawn=substr(fgets($myfile),0,5);
$CDawn=substr(fgets($myfile),0,5);
$SRise=substr(fgets($myfile),0,5);
$SSet=substr(fgets($myfile),0,5);
$CSet=substr(fgets($myfile),0,5);
$NSet=substr(fgets($myfile),0,5);
$ASet=substr(fgets($myfile),0,5);
$sunrise ='Sunrise: '. $SRise;
$sunset = 'Sunset: '. $SSet;
fclose($myfile);

I've added the sunrise time into the banner text too

The odd thing with the last hour of night images didn't reoccur last night, so I'm not sure what happened there. Could be a timing thing again

I can't quite remember now but I think the approach I took was to only have to change the code in a single location... but it's been a while since I looked at it   

I can't fit the sunrise on mine as I have outdoor temp and wind speed/direction on it but I can certainly see the appeal for anyone who doesn't have weather station data.

See if this works for you..

4 hours ago, Mognet said:

I suspect at least one problem could be down to the file /home/allsky/daily having the text (midnight sun) in it. That's the cause of the sunset time in the image displaying something like 01VT04 instead of the proper sunset time

The daytime movie problem I think was purely down to timing and the change over being on the hour. I notice that when it resolved itself was when the daytime video was due to start at 04:02. And that means it's likely to happen again whenever the start is on the hour

Having the image time on the video could be useful if anything interesting is caught. Last night I got a bit of an ISS pass in the frame

Yep -I captured ISS too. 

I think I did a fix for the issue regarding the strings being wrong out of sunwait when there is no astro dark... should be in this thread....

(perhaps I should get round to cloning the github and adding my alterations to it just in case I lose the device)

7 minutes ago, Mognet said:

Has anyone seen an issue where no images are taken for the last hour of night but the last taken image is still being added to the night video? I have a number of duplicated night images from 3am to 4am. It's the second night it's happened that I know of. Saturday night/sunday morning the image didn't change after 3am but was working again in the morning

I'm going to have a couple of debugging sessions. The day video not clearing and the sunset time display issue are on my list already. I know the basic causes, just need to identify fixes now

I've just looked and I have the same issue, last image before 4AM was at 02:55 then this is added until another image is taken at 03:59 and the final frame is 04:02.  This is one of the reasons I switched to using the banner image.  I haven't been looking at the movies much so hadn't really noticed the issue at the end of the night movie.

The very long day video seems to have rectified itself now - so must have been a timing issue.

1 hour ago, wormix said:

Quick update - still running fine, videos being deleted as appropriate and rerunning ok. 
 

Worth noting I have reverted back to the original code where the video is made up from the image file rather than the buffer (ie I don’t now have the text banner displayed in the video) 

 

Also to update - this project has just got far more expensive as I bought a 3D printer!  The friend I had making my camera holder mentioned he was considering selling his Ender 3 Pro so I enquired and at £80 I thought why not!

Slippery slope!  

3 hours ago, Pybo said:

Haha....nor, not Nunthorpe.. 

Acklam, just behind A174 fence

 

Are you around boro?

Marton originally, Sis lives just up from you near Hall Drive.  I guess you must be in the new houses adjacent to A174?

10 minutes ago, Pybo said:

Cheers! 

Luckily we have just moved house to the outskirts of southern Boro... 

Our back garden faces directly south and we have an almost unrestricted view from east to west.. i think its as gud as it gets for dark skies up here... we are not overlooked out back so no street lights. Just loads and loads of clouds...

Southern part of the Boro is the best part (unless it Nunthorpe - there a bit too posh there  )

Welcome Dave, is there any dark sky in the Boro? 

Well I looked through the scripts and nothing has changed other than the commented out bit to delete short clips.  I think this is either:

1. something to do with the cron sequence interaction (i.e. something is also happening at the same time as the cron is executing the newmovie script)

2. a permissions issue where something isn't getting actioned (perhaps removal of the old movie before concatenation?) 

Given it's been working OK for sometime and now started a common issue (3 of us reporting the same issue now) I don't think its permissions - more likely 1, to be a clash of timing somewhere ?  

Hmmm - <insert Big Brother geordie voice over> "day 3 in the lockdown household of Jiberjaber..." and we still have a concatenated daytime movie! Where as the night time video is still working as expected... will have to investigate I think..

34 minutes ago, wormix said:

Mine has returned to normal now - did have an issue with file size as I was using the buffer grabs so I had the banner text in video - those files are significantly smaller so have now reverted back to the original image 

File size as in teh movie size or the deleting based on file size? (that rm statement is littered through all the scripts) 

On 05/07/2020 at 15:44, Mognet said:

Had a little oddity happen overnight. The daytime images from yesterday didn't get deleted so the daytime video covers both days. First time I've seen that happen, so must just be a little timing issue

Same here - yesterday's (Monday) and today's. Not checked Sunday yet (I back up all the pics and videos, not sure why I think I had some grand plan for a historic viewing capability!) 

ETA - it's been a month since I last rsync'ed the camera's backup, this might take a while LOL

21 minutes ago, wormix said:

That’s better!

Hmmm - must be when I copy the image from the browser rather than use the snipping tool...