Jump to content

sgl_imaging_challenge_2021_3.thumb.jpg.30e9b298c34c80517e8b443ce153fce3.jpg

Setting up an MQTT system for Weather Sensing and Astro Control


Recommended Posts

Yeah emmc is certainly better than SD based storage. I'm just a bit obsessed with RPi so haven't looked at many other SBC's ūüôā¬†I've only just started playing with the Wemo D1 Mini's, I really like their form factor. I've got some ESP32's in the D1 Mini form factor coming to have a play with as well. Just need to find the time to actually complete some projects now!

Link to post
Share on other sites
  • Replies 326
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Now have the Wind sensors working.

New reconnect code. Changed to if (client.connect("indoorClient"))  { void reconnect() { // Loop until we're reconnected while (!client.connected()) { Serial.print("Attempting MQTT connect

Posted Images

I'm still having a problem displaying the dashboard.  I'm wondering if I'm trying to display too much information.  Or too much changing information.  Most of my sensors are sending messages every 5s and several in quick succession.  It really isn't necessary to have the data updated this often and generally the actual values don't change that quickly.  The only reason for the rapidly changing data is noise.  The only truly rapidly changing data is wind gusts and even that doesn't want displaying every few seconds.  I think it would be better to average the data and send MQTT messages far less often.

Or am I barking up the wrong tree here?

Edited by Gina
Link to post
Share on other sites

You might also want to check that you are NOT sending random "Nan" values from the ESP - I have found that some devices/sensors provide rubbish non numeric values (hence NAN - not a number) which if not filtered out somewhere(best done in the ESP) before displaying the actual data create problems.

I agree that there seems very little reason to provide data from most devices every 5s - obviously any rain detection needs to send its started raining ASAP however the timings can and should be controlled ,IMHO, from within the ESP.

I doubt you "large" display is causing your issues as many Home Automation systems have far more complicated display layouts. If you still suspect that is the issue you can of course split the display into tabbed windows easily in Node-red Dashboard and see if that is really the issue.

But I go with Wimvb first try the 60s refresh/send loop.

  • Thanks 1
Link to post
Share on other sites

I have now applied averaging in all the currently running clients.  Taking readings every 5s as before but accumulating data over 12 intervals then dividing by 12, sending the messages to MQTT and resetting ready for the next 12 intervals.  I have also reset the RPi to clear all old data and started again.  Looking better so far but it's "early days"!

Link to post
Share on other sites

I'm still displaying readings to the default 2 decimal places when the readings are nothing like that accurate.  In fact integer values would suit the reading accuracy better.  I see no point in showing the atmospheric pressure more accurately than integer values but I may round temperature and humidity to tenths.

Edited by Gina
Link to post
Share on other sites

Since the atmospheric pressure often changes very slowly I've decided to round to one decimal place.  Same for temperatures but pressures I've set to zero decimal places.  Just a matter of setting the appropriate number of decimal places in the dtostrf function which converts numbers to a char array for MQTT.

Link to post
Share on other sites

This is the dashboard display.  The pressure shows where I had rounding to integer (which seemed unsuitable).  The wind sensor is awaiting development.

848028321_Screenshotfrom2020-08-2614-06-05.thumb.png.4396461796ab04dabe95edeefbc3f23f.png

Link to post
Share on other sites

Dashboard display still fine.  Hoping to make progress on the sketch for the rain sensors today - lots of data processing.

1754992411_Screenshotfrom2020-08-2709-45-07.thumb.png.1f587de72254e689ad263199b4ad251f.png

  • Like 1
Link to post
Share on other sites

I like this system for displaying current conditions and recent history.  Of course, standard weather data logging and display is another matter.  That may come later as it's something I've always been interested in but there's lots to do before that.  There are more sensors and data processing to add to the current system plus the ROR control.  Although in some ways a weather station and control of the observatory roof are two distinct systems, they do relate to each other and covering both in one MQTT system makes sense.

Link to post
Share on other sites

The automatic Y range of the history graphs works quite well for the pressure but less well for temperatures but with the Node-RED system being so easy to use, it's easy to alter the ranges to suit current conditions manually, unlike physical gauges. 

Here is a new display with more suitable temperature ranges IMO.

828958893_Screenshotfrom2020-08-2714-33-09.thumb.png.09f38f731114d39e428d5431718dc2b2.png

Edited by Gina
Link to post
Share on other sites

Here's the sketch I put on my sensors that's been running beautifully for several years at this stage.

What's good about it is that it limits the frequency of posting to a max of once per second, it will only post if there's a specified difference between the last post and the current reading, and if it's unfeasibly calm, it'll force a posting every 5 minutes.

I just did some chopping and changing, all credit should go to the original author of course. I was using a Wemos D1 Mini (ESP8266) so pins may need to be updated.

ESP_BME280_MQTT_LWAT.ino

  • Thanks 1
Link to post
Share on other sites

I'm getting a strange problem with the wind speed.  Whether this is anything to do with MQTT or just the sketch in the wind ESP32 I don't know.  Until today the speed calculations have been fine but now the mean speed keeps resetting then building up again to the correct value.  These are enlargements of the mean speed history in the dashboard and the MQTT Explorer.   I'm wondering if the stronger breeze today could be anything to do with it. 

Unfortunately, to upload a new sketch to the ESP32 in the wind client means taking the wind sensor mast and unit down and connecting the ESP32 to the Mint box USB.  I seem to remember reading something about uploading a new Arduino sketch over WiFi but can't find it now.

1793347282_Screenshotfrom2020-09-0218-58-34.png.914361683966b2b9ce5245748f918fec.png  377560496_Screenshotfrom2020-09-0218-59-26.png.030fb1b9913a6c30b97b9da267f09107.png

Link to post
Share on other sites
4 hours ago, Gina said:

I'm getting a strange problem with the wind speed.  Whether this is anything to do with MQTT or just the sketch in the wind ESP32 I don't know.  Until today the speed calculations have been fine but now the mean speed keeps resetting then building up again to the correct value.  These are enlargements of the mean speed history in the dashboard and the MQTT Explorer.   I'm wondering if the stronger breeze today could be anything to do with it. 

Unfortunately, to upload a new sketch to the ESP32 in the wind client means taking the wind sensor mast and unit down and connecting the ESP32 to the Mint box USB.  I seem to remember reading something about uploading a new Arduino sketch over WiFi but can't find it now.

Depending on how you're doing your averaging, could be hitting data type limits and wrapping to a negative? If you're averaging on device by doing (x+y+...+z)/n, depending on n the sum of xyzetc could exceed a short/small int.

Link to post
Share on other sites
28 minutes ago, discardedastro said:

Depending on how you're doing your averaging, could be hitting data type limits and wrapping to a negative? If you're averaging on device by doing (x+y+...+z)/n, depending on n the sum of xyzetc could exceed a short/small int.

I was wondering that too, but I don't know enough about the platform to comment really.

I've just this evening tracked down an oacapture bug that only showed up on the RPi and only for certain types of camera, and went away if I changed some code completely unrelated to what I eventually discovered (after quite a few days of looking) was the actual problem.  Having fixed it, I have no idea why it didn't fail all the time everywhere.  Debugging can be so much fun :D

James

  • Like 1
Link to post
Share on other sites
1 minute ago, JamesF said:

I was wondering that too, but I don't know enough about the platform to comment really.

I've just this evening tracked down an oacapture bug that only showed up on the RPi and only for certain types of camera, and went away if I changed some code completely unrelated to what I eventually discovered (after quite a few days of looking) was the actual problem.  Having fixed it, I have no idea why it didn't fail all the time everywhere.  Debugging can be so much fun :D

Those bugs are the best sort of bugs!

This is why I really love working with Rust - while it's a bit slower to write than C and friends, it's super easy to debug and very hard to write code that breaks in odd ways - the compiler doesn't let you. I also had some fun with rust-fuzz to fuzz a binary parser I wrote a few weeks back - throwing automatically-generated random data that attempts to probe absolutely every code path and potential error can throw up errors very easily (especially ones you didn't even consider possible!) when you're dealing with third party input. I keep meaning to have a go with Rust on the ESP32 - there's now a good uC toolchain developing for a (pretty broad) subset of Rust.

Link to post
Share on other sites

Without wishing to derail Gina's thread, Rust (along with a few others) is a language I've noted as something to look into at some point.  I am a little wary of new languages however.  I reckon they take about twenty years to properly "bed in" and become stable enough to be reliably usable.  Some newer ones that have quite vocal followings can be an absolute nightmare when you want to run multiple applications, often ending up in "Dependency Hell" when one needs specific versions of certain packages/libraries whilst another requires often conflicting/incompatible versions of the same.  Few things make me feel more like giving the developers a good slap :)

James

Edited by JamesF
  • Like 2
Link to post
Share on other sites
2 minutes ago, JamesF said:

Without wishing to derail Gina's thread, Rust (along with a few others) is a language I've noted as something to look into at some point.  I am a little wary of new languages however.  I reckon they take about twenty years to properly "bed in" and become stable enough to be reliably usable.  Some newer ones that have quite vocal followings can be an absolute nightmare when you want to run multiple applications, often ending up in "Dependency Hell" when one needs specific versions of certain packages/libraries whilst another requires often conflicting/incompatible versions of the same.  Few things make me feel more like giving the developers a good slap :)

And there's definitely still a bit of that in Rust, though it has at least got a proper package manager (glares at Go and Elixir) baked in pretty well so managing versions of stuff is at least trivial. I think Rust's one of the more promising ones for doing astro coding in - it's designed specifically to focus on systems programming and so lends itself well to doing "always on" applications and that fuzzy area between "safety critical" and "really just don't want this to fall over at 3am and leave the observatory roof open..." where you don't need formal, provable correct code but do want a bit more compile-time scrutiny ūüôā I can heartily recommend giving it a go. But yes, let's stop derailing Gina's thread!

  • Haha 1
Link to post
Share on other sites

An interesting departure from topic ūüėĀ¬† Though not entirely unrelated.

The problem has been there overnight and still bad.  I have been through the calculations and can't find any "overloaded" numeric types.  I plan to set up another ESP32 for testing as the running one is up inside the wind unit and would require everything taking down and bringing the"works" indoors to update or swap ESP32s.

 

Screenshot from 2020-09-03 07-43-50.png

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By wimvb
      I recently posted my design for a weather station in this section.
      https://stargazerslounge.com/topic/345153-indi-weather-station/
      As I had bought several pressure/humidity sensors, as well as ir temperature sensors and ESP32 development boards, I wondered how small a weather monitor could get. The sensors are quite small, and so is the micro controller. Such a weather monitor wouldn't incorporate wind speed measurement nor a rain detector, since these take up more space. But otoh, there is seldom rain without clouds, so if you detect clouds, you should be safe.
      Here it is, a miniature (9.8 x 5.9 x 2.7 cm) weather monitor. The device has built in wifi, is powered from a micro usb contact and is compatible with the INDI Weather Watcher driver.
      The parts:

      (the mat underneath has a 1 inch grid pattern)
      BOM:
      plastic box 9.8 x 5.8 x 2.7 cm a piece of V-board, in my case with copper islands rather than strips ESP32 development board with male headers MLX90614 ir temperature sensor with I2C interface BME280 environmental sensor with I2C interface micro usb cable and power adapter, or a powerbank for wireless operation Assembly is really easy and involves drilling a hole in the box, soldering the components in place and wiring to the ESP.
      The finished monitor in place. As this is a box with a click lid, I used silicone to seal it. The holes on the sides and bottom are drilled at an angle to keep rain out.

      As I built it, the electronics will heat the BME slightly, and because it is mounted inside the casing, it will be slow to reach ambient temperature should this change abruptly. Adding more holes near the ESP would take the inside temperature down. Otoh, temperature readings don't have to be that accurate, and you could use the MLX ambient reading for more accuracy.
      Here's the INDI control panel for the weather monitor

      (Wind and rain are simulated, because I was testing the driver when I took the screen shot)
      Here is how it looks in Ekos scheduler. The red marker indicates that weather conditions are bad. In this case clouds = 100 %. If the tickbox next to "Weather" is checked, Ekos will allow weather conditions to control an imaging sequence.

      And in the ROR driver

      (I know it says Dome, but the ROR driver is derived from the dome driver, and it's still under development. Besides, this is the simulator driver.)
      The code for the esp is on my github page:
      https://github.com/wberlo/indi_meteostation
      You need the files:
      bme280.py mlx90614.py boot.py (replace the ssid and password with your own, or comment/uncomment lines to create an access point) main_mini.py (which you will have to rename to main.py before uploading to the esp board)  
    • By wimvb
      For quite some time now I've been working on a small weather station. The idea is to add automation to an obsy I'm building. The weather station as is measures temperature, humidity, pressure and presence of cloud. I also have the components to build a rain detector, but haven't implemented that yet.
      First the box:
      Made from vent covers and pieces of aluminium

      All put together

      With sensors. The IR sensor is in the far back (top of the housing) and the BME280 sensor is the free hanging red pcb. The other pcb holds a few pull-up resistors and a capacitor for the I2C wires. Insulation added to keep critters out.

      First I thought of using an Arduino to control and collect the data, but recently I started tinkering with ESP32 wifi boards. These are a lot faster and have more memory than Arduinos, and I wanted to use microPython.

      The ESP32 board is also smaller than an Arduino UNO (about the size of an Arduino nano) and has built in wifi, which means I just need power to the device.
      I uploaded the code to GitHub
      https://github.com/wberlo/indi_meteostation
      Output is as a HTML file (index.html) which presents data in a simple way. The INDI driver 'Weather Watcher' can read this file.
      For now, temperature is mapped to temperature (of course), and 'clouds' is mapped to 'forecast/Weather' in INDI. 'Clouds' is a parameter that varies from 0 to 1. I used the same criteria as the AAG Cloud Watcher device: if sky temperature is less than -8 degrees, the sky is assumed cloud free. For a sky temperature between -8 and 0 degrees, there is partial cloud, and for a sky temperature higher than 0 degrees, the sky is overcast. This seems very arbitrary, and I expect to have to adjust this piece of code later.

      Major update:
      The weather station now also reads and reports the SQM values from my wifi-SQM.¬†ūüė謆See further down this thread for details.
       
    • By wimvb
      Just finished work on a sky quality meter with built in wifi. The device is based on the ambient light sensor TSL2591 and the wifi board ESP32. Communication between the two boards is through I2C. The device has a 40 degrees lens.
      The light sensor is programmable, which means you can set integration time (from 100 to 600 ms) and gain (from 1 to almost 5000, in 4 steps). I implemented automatic adjustment of these parameters to allow for the highest dynamic range (600M:1 accoring to the spec sheet).
      The device shows Sky readings as a web page. It is connected to a local wifi network, although it could also create its own access point. So far I haven't been able to calibrate the sqm yet, partly due to eternal cloud cover. But it should only require one parameter to be adjusted.
      The code is available on GitHub. Sky-Quality-Meter
      Here are som pictures.
      The components:

      The parts connected:

      The finished device:

      This is how output is presented:

    • By x6gas
      Astroberry (strictly speaking Astroberry Server) is a fantastic operating system for the Raspberry Pi that allows control of your astromony kit and even better it's free!
      However, while there is a lot of useful information on SGL and elsewhere on the web, I had some trouble understanding how to set everything up and I couldn't find a beginners step-by-step guide.  I don't have much experience of the RPi or Linux or indeed any operating systems other than windows but after some trial and error I've got things working so I thought it might be useful to chronical the steps that hopefully will get you up and running.
      Astroberry uses INDI Library - an Open Source Architecture for Control & Automation of Astronomical Devices - you can think of this a bit like ASCOM.  Astroberry is also really flexible and there are multiple ways to do most things so what follows is just ONE way to get you up and running.
      So let's get started.  When I say 'computer' I mean your main computer and I use RPi when referring to the Raspberry Pi (that's a computer too, of course, but just to differentiate between the two).
      The Astroberry homepage is at https://github.com/rkaczorek/astroberry-server.  You'll need a Raspberry Pi, of course, (apparently Astroberry works with any RPi; I was using an RPi 3), an SD card of at least 16GB, and a computer with a suitable SD card slot (the RPi 3 needs a microSD card; most microSD cards come with an adapter that allows you to use a standard SD slot in your computer), and access to the internet.
      Firstly download the Astroberry Server image file from https://www.astroberry.io/distro/ (the image file is the operating system that will run on your RPi).

      Unzip this file into a folder on your computer.
      Then download balenaEtcher from https://www.balena.io/etcher/ - you'll use this to write the image file of the Astroberry operating system to your SD card; this process is known as 'flashing'.
      Once you've installed balenaEtcher, run it and select the Astroberry Server image file (when I did this the file was called astroberry-server_2.0.0.img) from the folder where you unzipped it.  Insert your SD card into the SD card slot on your computer, select this card from the 'Select target' button on balenaEtcher and then select 'Flash!'.  The process takes a little while but will show progress as the file is copied and then verified.  Make sure the flashing process has completely finished before removing the SD card from your computer.
      [Note, as the author of Astroberry @RadekK states in a comment below it's actually possible to set everything up without a monitor, mouse or keyboard.  To do that, insert the newly flashed SD card into your RPi and power it on.  After a few moments a wifi network 'astroberry' should be available.  Connect your computer to that network and point your browser to http://astroberry.local or http://10.42.0.1 (which is the default IP address assigned by Astroberry).  You should be able to everything via this remote connection.  Astroberry is also able to use a remote desktop app called VNC (icon is in the top right) so you can play with that too once you're well acquainted with Astroberry.] 
      Insert the newly flashed SD card into you RPi, connect a display, keyboard and mouse to your RPi and power it up.  You should see the Astroberry operating system load up.  Answer the questions and set your localisation options.
      Astroberry will create its own wifi network called 'astroberry' that you can use to connect to your RPi (very useful for use 'in the field') but this won't be connected to the internet.  We're not going to use the astroberry network for now.  Instead we are just going to have your RPi connect to your home network / internet.  To do this, click on the icon in the top left corner of the screen, select 'Preferences' and then 'Advanced Network Configuration'.

      Use this to add your wired or wifi network. 
      When you boot up your RPi, Astroberry should now connect it to your home network in preference to the Astroberry HotSpot.  If for some reason that doesn't work and Astroberry is connecting to it's HotSpot instead then you can do the following:
      Click on the icon in the top left corner of the screen, select 'Preferences' and then 'Advanced Network Configuration', select your home wifi network from the list and then click on the cog icon in the bottom left of the Network Connections window.   Click on the 'General' tab ensure that the 'Connect automatically with priority' has a tick next to it, and set the value to 1.  Close the editing window.# Then select 'Astroberry HotSpot' from the list, click on the cog icon in the bottom left of the Network Connections window again this time to edit the settings for Astroberry HotSpot.   Click on the 'General' tab ensure that the 'Connect automatically with priority' has a tick next to it, and set the value to 0.  Close the editing window, and then close the 'Network Connections' window.  These steps will mean that when your RPi is switched on it will connect to your home network if it can, and if it cannot it will start up its own wifi HotSpot called 'astroberry'. At this point, you should be able to connect to your RPi from your computer.  Open a web browser, type or copy http://astroberry.local/desktop/ in the address line and press enter.  You should see a screen asking you to connect to Astroberry Server (which is running on your RPi).

       
      Click on the connect button; the password is astroberry (in fact, if in doubt try astroberry as the password for everything - it usually is!)  If this has all worked correctly, you should now be able to control you RPi remotely so you can disconnect the display, mouse and keyboard from your RPi.
      You'll see some other icons in the top left corner of the Astroberry desktop including one for PHD2 but don't go there yet!
      Before we do anything else we need to start the INDIserver service - this will load the drivers etc that you need to run your kit.  On the left of the screen is a blue-grey tab that will expand to show some buttons.  Click on the telescope icon which brings up the INDI Web Manager window.  You can go through and select the drivers for your equipment.  Click on the 'Start Server' button at the bottom of the INDI Web Manager window which starts INDIserver - this is like starting ASCOM.  Once you've done that, type a name in the 'New Profile' box and save it.  You can then select it from the 'Equipment Profile' box; delete the simulator profile if you like.  There are check boxes under the 'Equipment Profile' box that allow you to automatically start INDIsever select a particular profile and connect to your devices - so long as the devices are connected and powered on.  If you check these boxes you don't need to repeat the step of selecting your profile etc.
      This should have you more or less ready to go.  If you experience connection problems with kit that gets its electrical power from USB (e.g. the QHY5L-II guide camera) then use a powered USB hub as the RPi USB ports don't provide enough electrical power to properly power some equipment.
      There are icons for some astronomy programmes in the top left of the Astroberry desktop.  PHD2 is familiar to me and you can test that your kit is connecting in that.
      KStars (the telescope icon next to the left of the PHD2 icon) is planetarium software that also allows you to launch Ekos (Tools>Ekos or ctrl K) and this allows you to set up equipment profiles and run imaging sequences.
      Hopefully this guide will enable you to get things set up and your kit connected.  I haven't yet explored Kstars or Ekos much, nor much of the rest of the desktop but hopefully it will be fairly intuitive.
      I've written most of this guide from memory so if a step doesn't work then please let me know and I'll try to correct it.
      Hope this helps and huge thanks to the Astroberry developer, @RadekK, for making this software available to the community - I'm sure it took a huge amount of work.
      Clear skies, Ian
    • By PESKYWAABBIT
      Hey there,
      Curious about which CCD's you have been or are using successfully with auto guiding on a rpi2 or even a rpi3?
      lin_guider seems to support a bunch of manufacturers but a list of what models are proven to work with the Raspberry pi's will surely help my quest!
      Cheers for the help!
√ó
√ó
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. By using this site, you agree to our Terms of Use.