Setting up an MQTT system for Weather Sensing and Astro Control

[Gina]

So far so good but I shall only feel I have it working when I have a separate client SBC talking to a client on the RPi or better still on something else.  I have another BME280 which I can put on some stripboard with one of the ESP32s and program to read the indoor temperature, humidity and atmospheric pressure and send it to the broker.  Then I can subscribe to new topics and send data.  That way I can try out the MQTT system without venturing out to the obsy in a deluge and also leave the sensors working for real use.

[Gina]

This would seem to be what I want for the ESP32 and BME280 :- ESP32 MQTT – Publish and Subscribe with Arduino IDE

[Gina]

Ran the Arduino and went to Tools > Manage Libraries,  searched for PubSubClient and Installed it.  I already have the Adafruit_BME280  and Adafruit_Sensor libraries installed

[Gina]

This tutorial uses Node-RED installed on the RPi running the broker to provide a graphical interface on a browser.  Seems reasonable for the first setup.  Later I may use WeeWX but guess there's not really a limit and I could have both.

Any comments?

Edited August 14, 2020 by Gina

[wimvb]

 I believe you can use mqtt-explorer on another machine, if you point it to your broker. As far as I remember, mqtt-explorer lets you see all topics and subscribe to them. This allows you monitor the broker. Then clients can subscribe to the topics they need, and post messages in those topics. Mqtt-explorer is good to have while you experiment. As a further test, set up an esp32 to post internal hall sensor values to a topic. It only requires an esp, without additional hardware.

[Gina]

I'll look into that - thanks Wim 😀

[Gina]

OK - I've installed it but where is it?  It's not in the menu.

[wimvb]

6 minutes ago, Gina said:

OK - I've installed it but where is it?  It's not in the menu.

If you mean mqtt-broker, I only installed it on a windows laptop and put an icon on the desktop. Don't know how that works with Linux Mint.

[Gina]

No, I mean mqtt-explorer that you suggested above.

[rogmet]

On my Mint machine, MQTT Explorer shows up in the Programming section of the menu

[Gina]

Rebooted and it's there now.  Don't usually have to do that with Linux.

[Gina]

Next question is how to use it.  I have it running but it only seem to show what's on the net.

[rogmet]

Click on the Plus button beside "Connections" when you're not connected to anything. Give it whatever name you want. Put in the IP address or the hostname under Host. Leave the protocol and port number as they are, leave username and password blank.

It should show you all active publishing topics on the server, at the very least there'll be $SYS which is the MQTT server itself publishing info on what it's doing.    

[Gina]

I have it working.  No doubt I'll see what to do with it as I proceed.

[wimvb]

This is how mine looks now

Topic is 'hall'

Message is 'Magnetic field strength: xx.xxx'

Plus a code snippet (python):

while True:
  try:
    client.check_msg()
    if (time.time() - last_message) > message_interval:
      msg = b'Magnetic field strength: {0:.3f}'.format(esp32.hall_sensor())
      client.publish(topic_pub, msg)
#      print('message' + msg + 'published to topic' + topic_pub)
      last_message = time.time()
      counter += 1
  except OSError as e:
    restart_and_reconnect()

[wimvb]

The general idea is that if you have a client that posts messages to a topic, you will see more than just the broker IP address and $SYS

[wimvb]

If I strip the text from the published message, I get a graph

[jiberjaber]

MQTT has a structure, that is a TOPIC and a PAYLOAD, The payload is usually data and not used like say a print statement.  You also need to be careful of overloading the payload in Arduino as it is a buffer in the pubsub client so can only handle so much at a time (though this can be increased but need to keep an eye on the ration of memory used). 

My initial implementation was just to send a single value (eg: TOPIC = /home/hall/temp PAYLOAD = 27.01) but I've since been investigating using a JSON string I think it's called as I can then send more info in one message which helps with any other clients subscribing (just forward thinking, there's a bit of an issue with the pubsub library when subscribing to too many topics, and you don't want the software interrupt being triggered really often!) 

I can't remember the exact notation (see my git I guess for details) but using the JSON approach I could send TOPIC = /hom/hall PAYLOAD = {27.01,56.00,1011.11,falling} (or something like that) and that would allow me to deliver in one message temperature, humidity, pressure and trend, as an example. 

If you get the JSON format right the monitoring software will also be able to decode it too, but it's late and I have probably written it wrong in my example 😳

[Gina]

Now installing Node-RED a slow process...

Done.

Running Node-RED install for user pi at /home/pi on raspbian\n

This can take 20-30 minutes on the slower Pi versions - please wait.

  Stop Node-RED                       ✔
  Remove old version of Node-RED      ✔
  Remove old version of Node.js       ✔
  Install Node.js for Armv6           ✔   Node v10.22.0   Npm 6.14.7
  Clean npm cache                     ✔
  Install Node-RED core               ✔   1.1.3 
  Move global nodes to local          -
  Install extra Pi nodes              ✔
  Npm rebuild existing nodes          -
  Add shortcut commands               ✔
  Update systemd script               ✔
                                           

Any errors will be logged to   /var/log/nodered-install.log

All done.
  You can now start Node-RED with the command  node-red-start
  or using the icon under   Menu / Programming / Node-RED
  Then point your browser to localhost:1880 or http://{your_pi_ip-address}:1880

Started  Sat 15 Aug 09:06:28 BST 2020  -  Finished  Sat 15 Aug 09:21:48 BST 2020

rm: cannot remove '/usr/bin/update-nodejs-and-nodered': No such file or directory
pi@MQTTBrokerGina:~ $

 

Edited August 15, 2020 by Gina

[Gina]

Until I see the advantages of Node-RED I won't set the autostart on boot yet but start it when I want it with :-

node-red-start

or from the Programming menu.

If/when I find it useful I can make it start at boot with :-

sudo systemctl enable nodered.service
sudo reboot

Does anyone here use Node-RED and how useful do you find it?

[Gina]

The Node-RED Dashboard looks like one way of displaying my weather (and ROR) data and controlling the ROR.  It could do all I want, so I shall install it.

[Gina]

Hmm...  The Node-RED Dashboard is a fiddly thing to set up and will need some studying but designing any specialised GUI is going to be like that.  I reckon I can now see the way ahead but I think graphical displays are for the future and i need to get MQTT running first.  OK so Node-RED is supposed to be a short-cut to designing the MQTT network and the links between clients but it seems a lot to learn all at once!

OTOH the ESP32 MQTT – Publish and Subscribe with Arduino IDE tutorial has an example that almost exactly matches an ESP32 sensor unit that I want to use so I'll continue and set that up and gain experience.  No doubt I can then go on to modify things to suit my other sensor units.

[Gina]

Sketch from the tutorial above now uploaded to one of my ESP32s.  Now to connect the BME280 etc.

After a coffee break 😁

Edited August 15, 2020 by Gina

[wimvb]

I think I should start playing with Node-Red. But my Raspberry Pi is getting old, so I must upgrade that first.

[Gina]

Hardware soldered onto stripboard and all connected up.  Powered from bench PSU on 5v.  Devices are recognised but I can't get communications.

Node-RED links showing clients connected.

This is the Node-RED Dashboard.  At first I could move the Output switch from OFF to ON but it won't go back,

This seems to be saying that the ESP32 unit has received the on message but the LED isn't on.

Seems to be nearly working but not quite.