Weather Station Ideas

[Gina]

I shall have to try to arrange that a lost connection doesn't interfere with the calculations. 

I can see why the disconnections cause an increase in Gust speed and a reduction in Mean speed.  The Gust speed is obtained from the number of anemometer pulses and therefore the number of counts from the interrupt handler.  The interrupt handler is not interrupted by disconnection but the resulting count continues to rise if the 3s period is interrupted by disconnection.  OTOH the Mean is dependent on the summing process and if that is interrupted, data is lost resulting in a lower sum.

Edited September 3, 2020 by Gina

[globular]

I'd also suggest that you change meanArray and meanSpeed from int to long; there may be overflow breaches with int during the accumulations and this would cause the falling back to zero behaviour you are seeing. 
I know you are only (supposed to be) summing 200 readings before then calculating the average; so unlikely that it should be biting; but changing them to long, even if only temporarily, should allow you to definitively rule this in or out.

[Gina]

2 hours ago, skybadger said:

you can also drop this line as no longer required 

if (!client.connected()) {reconnect();}

since the Wifi object will auto reconnect if it detects a broken connection.

That reconnect is what is causing all the trouble it seems - leave the system frozen for at least 5s while it tries to reconnect.

I can easily remove that line but what about the following line - is that alright still.

void loop() {
  if (!client.connected()) {reconnect();}
  client.loop();

 

[Gina]

Now it isn't connecting to MQTT.

[Gina]

Maybe it needs to reconnect to MQTT each time it wants to send a message.  That's once a minute.

[Gina]

That's fixed it.

[Gina]

I need to sort it out but it looks like Timer Interrupts are what I want to guarantee that the calculations take precedence over MQTT reconnection.

[skybadger]

Sorry,  I was misleading you . I read that reconnect as a wifi reconnect while its not, its a mqtt reconnect. thats required for the client.loop calls as you mention. 

You only need to check for a connection when you have something to say and when you are expecting  subscription callbacks . Are you using subscription callbacks ? The client.loop function is about checking for and servicing the subscription callbacks and activates the callback handler. 

If not using callbacks then You don;t really care if the MQTT connection times out since it will be remade every time you connect out to it. 

 

 

 

 

[Gina]

Thanks.  I'm understanding MQTT better and better as I go along with the help of people like you.  😀

Edited September 4, 2020 by Gina

[Gina]

I'm looking at ESP32 Timer interrupts and found this from Espressif :- API Reference »Peripherals API »Timer

[Gina]

Found this and tried it :- Timer Interrupts Explained with Examples

volatile int interrupts;
int totalInterrupts;

hw_timer_t * timer = NULL;
portMUX_TYPE timerMux = portMUX_INITIALIZER_UNLOCKED;

void setup() {

  Serial.begin(115200);

  // Configure Prescaler to 80, as our timer runs @ 80Mhz
  // Giving an output of 80,000,000 / 80 = 1,000,000 ticks / second
  timer = timerBegin(0, 80, true);                
  timerAttachInterrupt(timer, &onTime, true);    
  // Fire Interrupt every 1m ticks, so 1s
  timerAlarmWrite(timer, 1000000, true);      
  timerAlarmEnable(timer);
}

void IRAM_ATTR onTime() {
  portENTER_CRITICAL_ISR(&timerMux);
  interrupts++;
  portEXIT_CRITICAL_ISR(&timerMux);
}

void loop() {
  if (interrupts > 0) {
    portENTER_CRITICAL(&timerMux);
    interrupts--;
    portEXIT_CRITICAL(&timerMux);
    totalInterrupts++;
    Serial.print("totalInterrupts");
    Serial.println(totalInterrupts);
  }
}

But it won't compile - with this error :-

I thought "This looks simple enough."  Thought I'd found something I could build on, after all it shouldn't be that difficult.  Just need the info.  Guess my search continues...  Unless anyone can help or explain this.

[Gina]

Tried moving the interrupt handler above the setup and it worked!!!

volatile int interrupts;
int totalInterrupts;

hw_timer_t * timer = NULL;
portMUX_TYPE timerMux = portMUX_INITIALIZER_UNLOCKED;

void IRAM_ATTR onTime() {
  portENTER_CRITICAL_ISR(&timerMux);
  interrupts++;
  portEXIT_CRITICAL_ISR(&timerMux);
}
 
void setup() {

  Serial.begin(115200);

  // Configure Prescaler to 80, as our timer runs @ 80Mhz
  // Giving an output of 80,000,000 / 80 = 1,000,000 ticks / second
  timer = timerBegin(0, 80, true);                
  timerAttachInterrupt(timer, &onTime, true);    
  // Fire Interrupt every 1m ticks, so 1s
  timerAlarmWrite(timer, 1000000, true);      
  timerAlarmEnable(timer);
}

void loop() {
  if (interrupts > 0) {
    portENTER_CRITICAL(&timerMux);
    interrupts--;
    portEXIT_CRITICAL(&timerMux);
    totalInterrupts++;
    Serial.print("totalInterrupts");
    Serial.println(totalInterrupts);
  }
}

 

[Gina]

Tried increasing number of timer ticks to 3,000,000 for 3s and that worked as expected.

Now I presume I just need to insert the do3sJobs(); in the interrupt handler instead of interrupts++;

void IRAM_ATTR onTime() {
  portENTER_CRITICAL_ISR(&timerMux);
  interrupts++;
  portEXIT_CRITICAL_ISR(&timerMux);
}

Like this

void IRAM_ATTR onTime() {
  portENTER_CRITICAL_ISR(&timerMux);
  do3sJobs();
  portEXIT_CRITICAL_ISR(&timerMux);
}

Then inside the do3sJobs(); count 20 times and do the do1mJobs(); after which it will return to what it was doing in the main loop including reconnecting.

Ah...  had a thought - the sending messages to MQTT would be in the time critical code.  Only the time critical calculations need doing within the interrupt coding.  I could set a flag and do the sending messages in the main loop.

[Gina]

Only the wind speed timings and calculations are time critical, the direction isn't and could be handled with periods in the main loop as before.  Same with all the MQTT messages with a flag to show when speed messages were ready to send.

[Gina]

I have yet to work out what I need to do to this to suit my system.

void loop() {
  if (interrupts > 0) {
    portENTER_CRITICAL(&timerMux);
    interrupts--;
    portEXIT_CRITICAL(&timerMux);
    totalInterrupts++;
    Serial.print("totalInterrupts");
    Serial.println(totalInterrupts);
  }
}

 

[Gina]

Guess I could put

interrupts++;

back into the ISR and remove the print statements from the loop.  OTOH I would really like to know what the code in the loop is actually doing.  I'm not happy just using code "blind".

[Padraic M]

Hi Gina,

We really don't like nested interrupts, so even if your do3sjobs() routine may work from within the ISR, it is bad practice to do much other than set a flag (or exit low-power mode if you were doing that kind of thing) in the ISR, and then get out of there before the next interrupt arrives. I'd recommend your 3sjobs, 1mjobs and 3mjobs should all be called from loop().

Bearing in mind that you'll get an ISR 'ping' once every second, I'd add three incrementing counters to count to 3, 60 and 180 isr pings respectively - or you could use one counter and some fancy modulo code if you prefer. Or even use your old code where you calculate the duration since last doxjob was done, which will remove any potential lag or jitter in the ISR timing. Then, you can loop within loop() waiting for the ISR flag to set, which frees you up to do your jobs. Something along these lines, using the 'interrupts' var as a flag rather than a counter:

void loop() {
  if (!client.connected()) {reconnect();}
  client.loop();

  while( interrupts == 0 ){} // will spin here until the ISR sets the flag

  // ISR has triggered - proceed and perform the jobs
  long now = millis();
  if(now - lastP > Ptest) {++PulseCount; lastP = now;}
  if(now - last3s > P3s) {do3sJobs(); last3s = now;}
  if(now - last1m > P1m) {do1mJobs(); last1m = now;}
  if(now - last3m > P3m) {do3mJobs(); last3m = now;}
  interrupts = 0; // Reset the flag

}

I'm mulling clearing the flag at the end or decrementing it. If there's any possibility that your jobs could take more than 1 second to complete, you will have multiple timer ISRs triggered in the interim. Do you want to process each interrupt without fail, or just process the latest one?

 

[Gina]

Since the wind measurements no longer mean anything (except the direction) I have unscrewed the mast from the obsy corner post and brought the whole contraption indoors.

[Gina]

I'm running the timer interrupt at 3s.  Only the number of hardware interrupts within that period is important.  The 1m speed integration can be done on counts of the timer ISR.  As long as there are 20 sets of counts summed in the 1m routine it doesn't really matter if the odd set of counts gets lost.

[Gina]

Decided to put the speed part of the 3s jobs directly in the ISR.

void IRAM_ATTR onTime() {
  portENTER_CRITICAL_ISR(&timerMux);
  interrupts++;
  windSpeed += PulseCount;  //  Accumulate mean speed
  if (PulseCount > windGust) {windGust = PulseCount;};  // Get max speed for gust
  PulseCount = 0;
  portEXIT_CRITICAL_ISR(&timerMux);
}

 

[Gina]

Think I may have it sussed.  Used the timer interrupt just to count the anemometer pulses plus a count of times it was enacted to use later for the divisor.

void IRAM_ATTR onTime() {
  portENTER_CRITICAL_ISR(&timerMux);
  interrupts++;
  windSpeed += PulseCount;  //  Accumulate mean speed
  if (PulseCount > windGust) {windGust = PulseCount;};  // Get max speed for gust
  PulseCount = 0; // clear count
  timerCount++;  // count number of timer interrupts for 1m speed sum                                      
  portEXIT_CRITICAL_ISR(&timerMux);
}

This sets the interrupts flag, counts the anemometer pulses, calculating sum and max for mean and gust speeds.  Then the PulseCount is cleared for the next count and the number of times this ISR is run is counted in timerCount.

[Gina]

I have renamed the routines better to reflect the operations and separated speed and direction processes.  The Timer and Hardware ISRs are now next to each other with the constant and variable declarations first.  I have changed the speed averaging period to 21x3s intervals to make the divisor an integer when dividing the sum and correcting the speed (speed integration time 3s rather than the 4.5s which would have given mph directly).

This is the sketch.

// Wind speed and direction 2020-09-04

/*********
  With thanks to Rui Santos
  Complete project details at https://randomnerdtutorials.com 
  and others
********
Modified and added to by Gina 2020-08-21 onward
********
*/
// Set up for timer interrupts
volatile int interrupts;
int totalInterrupts;

hw_timer_t * timer = NULL;
portMUX_TYPE timerMux = portMUX_INITIALIZER_UNLOCKED;
int timerCount = 0;

// periods or intervals in milliseconds
const int P3s = 3000;      // integer
const int P3m = 180000ul;  // unsigned long
long last3s = 0;
long last3m = 0;

// ADC pins
const int An1 = 34;
const int An2 = 35;
const int An3 = 32;
const int An4 = 33;
// ADC readings
int dir1 = 0;
int dir2 = 0;
int dir3 = 0;
int dir4 = 0;

// Gray to binary table
int codeArray[16] = {0,1,3,2,7,6,4,5,15,14,12,13,8,9,11,10};
int dirn = 0;

// Direction count bins
int bin[20] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};

// 10m speed averaging etc.
int meanArray[10] = {0,0,0,0,0,0,0,0,0,0};
int gustArray[10] = {0,0,0,0,0,0,0,0,0,0};
int ringIndex = 0;

// Output variables
int Direction = 0; // 0-359 degrees
int meanSpeed = 0; // 0-100mph
int gustSpeed = 0; // 0-100mph

// Set GPIO for Hall Sensor
const int HallSensorPin = 4;
int PulseCount = 0, windGust = 0;
float windSpeed = 0;

#include <WiFi.h>
#include <PubSubClient.h>

// Replace the next variables with your SSID/Password combination
const char* ssid = "Ubiquity";
const char* password = "********";

const char* mqtt_server = "192.168.1.140";

WiFiClient windClient;
PubSubClient client(windClient);

void IRAM_ATTR onTime() {
  portENTER_CRITICAL_ISR(&timerMux);
  interrupts++;
  windSpeed += PulseCount;  //  Accumulate mean speed
  if (PulseCount > windGust) {windGust = PulseCount;};  // Get max speed for gust
  PulseCount = 0; // clear count
  timerCount++;  // count number of timer interrupts for 1m speed sum                                      
  portEXIT_CRITICAL_ISR(&timerMux);
}

// Checks if Hall sensor was triggered - Interrupt Handler
void IRAM_ATTR HallTriggered() {
//  Serial.println("Hall Triggered");
  ++PulseCount; // Increment count
}

int Beaufort(int mph){
  if (mph < 1) return 0;
  else if (mph <= 3) return 1;
  else if (mph <= 7) return 2;
  else if (mph <= 12) return 3;
  else if (mph <= 18) return 4;
  else if (mph <= 24) return 5;
  else if (mph <= 31) return 6;
  else if (mph <= 38) return 7;
  else if (mph <= 46) return 8;
  else if (mph <= 54) return 9;
  else if (mph <= 63) return 10;
  else if (mph <= 72) return 11;
  else return 12;
}

void setup() {
  Serial.begin(115200);
  setup_wifi();
  client.setServer(mqtt_server, 1883);
  client.setCallback(callback);
  
  // Hall Sensor mode INPUT_PULLUP
  pinMode(HallSensorPin, INPUT_PULLUP);
  // Set HallSensor pin as interrupt, assign interrupt function and set FALLING mode
  attachInterrupt(digitalPinToInterrupt(HallSensorPin), HallTriggered, FALLING);
  if (!client.connected()) {reconnect();}
//
  // Configure Prescaler to 80, as our timer runs @ 80Mhz
  // Giving an output of 80,000,000 / 80 = 1,000,000 ticks / second
  timer = timerBegin(0, 80, true);                
  timerAttachInterrupt(timer, &onTime, true);    
  // Fire Interrupt every 3m ticks, so 3s
  timerAlarmWrite(timer, 3000000, true);      
  timerAlarmEnable(timer);
}

void setup_wifi() {
  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void callback(char* topic, byte* message, unsigned int length) {
  Serial.print("Message arrived on topic: ");
  Serial.print(topic);
  Serial.print(". Message: ");
  String messageTemp;
  
  for (int i = 0; i < length; i++) {
    Serial.print((char)message[i]);
    messageTemp += (char)message[i];
  }
  Serial.println();
}

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("windClient")) {
      Serial.println("connected");
      // Subscribe
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

//Get instantaneous wind direction - variable dirn
int readDirection(){    
    // Read wind vane optical sensor values
    dir1 = analogRead(An1);
    dir2 = analogRead(An2);
    dir3 = analogRead(An3);
    dir4 = analogRead(An4);
    // Convert Gray bits to integer
    int Gray = 0;
    if (dir1 > 2000) {Gray = 8;};
    if (dir2 > 2000) {Gray += 4;};
    if (dir3 > 2000) {Gray += 2;};
    if (dir4 > 2000) {Gray += 1;};
    //  Convert Gray to binary
    dirn = 15 - codeArray[Gray]; // correct rotation direction
    dirn = (dirn - 1) %16; // correct encoder for North
    return dirn;
}
//
void sendSpeedMessages(){
  if (!client.connected()) {reconnect();}
  // messages to send :-
  // wind/speed/mph -- meanSpeed
  // wind speed/force -- Beaufort(meanSpeed)
  // wind/gust/mph -- gustSpeed
  // wind/gust/force -- Beaufort(gustSpeed)
  // 
  // Convert the Mean Speed to a char array
  char msString[8];
  dtostrf(meanSpeed, 1, 1, msString);
  Serial.print("  Mean Speed: ");
  Serial.print(msString);
  client.publish("wind/speed/mph", msString);
  
  // Convert the Mean-Speed-Force to a char array
  char bsString[8];
  dtostrf(Beaufort(meanSpeed), 1, 0, bsString);
//  Serial.print("Force: ");
//  Serial.println(bsString);
  client.publish("wind/speed/force", bsString);
  
  // Convert the Gust-Speed to a char array
  char gsString[8];
  dtostrf(gustSpeed, 1, 0, gsString);
  Serial.print("  Gust: ");
  Serial.println(gsString);
  client.publish("wind/gust/mph", gsString);

  // Convert the Gust-Speed-Force to a char array
  char bgString[8];
  dtostrf(Beaufort(gustSpeed), 1, 0, bgString);
//  Serial.print("Gust Force: ");
//  Serial.println(bgString);
  client.publish("wind/gust/force", bgString);
}
void sendDirectionMessage(int Dir){
  // Convert the Direction to a char array
  char dirString[8];
  dtostrf(Dir, 1, 0, dirString);
//  Serial.print("  Direction: ");
//  Serial.println(dirString);
  client.publish("wind/direction", dirString);
}
void sendDirectionMessageInst(int Dir){
  // Convert the Direction to a char array
  char dirString[8];
  dtostrf(Dir, 1, 0, dirString);
//  Serial.print("Transient Direction: ");
//  Serial.println(dirString);
  client.publish("wind/direction/inst", dirString);
}
/*  Debugging only
void sendNumberMessage(int N){
  // Convert the number to a char array
  char numString[8];
  dtostrf(N, 1, 0, numString);
  Serial.print("Number: ");
  Serial.println(numString);
  client.publish("wind/number", numString);
}
void sendNumberMessage2(int N){
  // Convert the number to a char array
  char numString[8];
  dtostrf(N, 1, 0, numString);
  Serial.print("Number: ");
  Serial.println(numString);
  client.publish("wind/number2", numString);
} 
*/
void getDirection(){
  byte instDir = readDirection();
  ++bin[instDir]; // increment appropriate bin
  sendDirectionMessageInst(instDir); // send instantaneous direction 
}
void speed21sum(){
  int divisor = 140;
  Serial.print("ringIndex: ");
  Serial.print(ringIndex);
  // wind speed mean and gust ring arrays and report
  meanArray[ringIndex] = windSpeed; // put windSpeed into new array index
  gustArray[ringIndex] = windGust;  // put windGust into new array index
  meanSpeed = 0; gustSpeed = 0; windSpeed = 0; windGust = 0;
  for (int i = 0; i < 10; i++) { 
    meanSpeed += meanArray[i];  // sum the windSpeeds 
    if (gustArray[i] > gustSpeed){gustSpeed = gustArray[i];}} // find maximum gust speed
  divisor = timerCount * 10 / 1.5;  // with timerCount=21 this gives a whole number
  meanSpeed /=divisor;  // the first sum was over 21 values and then the second over 10 values
  gustSpeed *= 1.5;  // Speed count over 3s rather than 4.5s
  sendSpeedMessages();
  ringIndex = (ringIndex+1)%10; // move ringIndex on to next location in the ring arrays
}
void ConsensusAveraging(){
  // wind direction calculations and report
  int sum[16];
  int S=0,I=0,W=0;
  bin[16] = bin[0];
  bin[17] = bin[1];
  bin[18] = bin[2];
  bin[19] = bin[3];
  for (int i = 0; i < 16; i++) {sum[i] = bin[i] + bin[i+1] + bin[i+2] + bin[i+3] + bin[i+4];
    // find the index with the highest sum and save sum and index
    if (sum[i] > S){S = sum[i]; I = i;};}
  W = (bin[I+1] + 2 * bin[I+2] + 3 * bin[I+3] + 4 * bin[I+4]) * 45 / S;
  sendDirectionMessage((I * 45 + W)%720 /2);
  //Empty the bins ready for a new direction calculation
  for (int i = 0; i < 16; i++) {bin[i] = 0;};
}

void loop() {
//  if (!client.connected()) {reconnect();}
  client.loop();
  while( interrupts == 0 ){} // will spin here until the ISR sets the flag
  // ISR has triggered - proceed and perform the jobs
  if (timerCount >= 21) {speed21sum();};
  // time the various periods
  long now = millis();
  if(now - last3s > P3s) {getDirection(); last3s = now;}
  if(now - last3m > P3m) {ConsensusAveraging(); last3m = now;}
  interrupts = 0; // Reset the flag
}

 

Edited September 4, 2020 by Gina

[Gina]

Just ran it as a test and there's something wrong.  It repeatedly reconnects to the WiFi and does nothing else.

[Gina]

Something is causing it to reboot!!!

[Gina]

Hmmm...  Decided to try commenting out all my code from the timer ISR.  Now it connects and just sits there doing nothing more.