Setting up an MQTT system for Weather Sensing and Astro Control

[Gina]

Thanks for the warning.  I never really trust anything I haven't had experience of and found trustworthy.  I try to go for fail-safe and/or redundancy where it's important.  I also tend to over-engineer.

[Gina]

I have another problem.  Can't upload sketch to ESP32.  Tried cold rebooting Mint box but still the same.

Quote

Arduino: 1.8.10 (Linux), Board: "ESP32 Dev Module, Disabled, Default 4MB with spiffs (1.2MB APP/1.5MB SPIFFS), 240MHz (WiFi/BT), QIO, 80MHz, 4MB (32Mb), 115200, None"

Sketch uses 702242 bytes (53%) of program storage space. Maximum is 1310720 bytes.
Global variables use 40120 bytes (12%) of dynamic memory, leaving 287560 bytes for local variables. Maximum is 327680 bytes.
python /home/gina/.arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py --chip esp32 --port /dev/ttyS0 --baud 115200 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size detect 0xe000 /home/gina/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/partitions/boot_app0.bin 0x1000 /home/gina/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/bin/bootloader_qio_80m.bin 0x10000 /tmp/arduino_build_240309/MQTT_test_ESP32_and_BME280.ino.bin 0x8000 /tmp/arduino_build_240309/MQTT_test_ESP32_and_BME280.ino.partitions.bin
esptool.py v2.6
Serial port /dev/ttyS0
Connecting........_____....._____....._____....._____....._____....._____.....____An error occurred while uploading the sketch
_

A fatal error occurred: Failed to connect to ESP32: Timed out waiting for packet header

This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.

edit - Yes, I did press the BOOT button!

Tried it with 2 ESP32s.

HELP - please.

Edited August 17, 2020 by Gina

[jiberjaber]

3 hours ago, Gina said:

I have another problem.  Can't upload sketch to ESP32.  Tried cold rebooting Mint box but still the same.

edit - Yes, I did press the BOOT button!

Tried it with 2 ESP32s.

HELP - please.

I get this sometimes when there is a conflict with the serial port on the programming PC, a reboot usually solves it but it is one of the reasons I move to OTA as soon as I can.  Have you tried a reboot, or even another USB port? 

On Windows, Cura will grab the USB ports when it loads up resulting in the same issue, but you are on Linux, so I expect that to not be the problem

 

[Gina]

Yes tried a reboot - no joy.  I'll try a different USB port.

"I move to OTA" - What is OTA in this case.  Guess it's not Optical Tube Assembly 🤣

Edited August 17, 2020 by Gina

[Gina]

Tried other USB ports including a USB2 rather than USB3.  Still no joy!

Any other ideas?  I have 2 more ESP32s on the way but I hope the problem is not that I killed two ESP32s.  I ordered 2 more because I expect to want 2 more not to replace duds!!!

Edited August 17, 2020 by Gina

[jiberjaber]

21 minutes ago, Gina said:

Tried other USB ports including a USB2 rather than USB3.  Still no joy!

Any other ideas?  I have 2 more ESP32s on the way but I hope the problem is not that I killed two ESP32s.  I ordered 2 more because I expect to want 2 more not to replace duds!!!

OTA = Over The Air via IP (I think you experimented with that earlier when I mentioned it?

Are the ESP's still running the code that was on them, if so, they are not dead I think - It's been a while since I used these devices (the ones I have don't have a separate programme button) - Are you sure you are holding the right button down for the right sequence, I think it is programme button whilst power up isn't it? 

[Gina]

Yes, definitely pressed the right button.  I've changed code so often I could reprogram an ESP32 in my sleep!

The ESPs were programmed to work with MQTT but I can't remember what as I've been swapping about too much.  I do know that each was not working as intended but was partly working.

[Gina]

Thought I had a couple of ways of determining if the ESP32s were alright or not.  SSH to the IP address and ping to the IP address.  Both failed.  Then I tried with the known working ESP32 and that failed both tests too.

Edited August 17, 2020 by Gina

[Padraic M]

In my experience these devices are very hard to kill, and I've tried lots of ways.

Are you definitely on /dev/ttyS0?

Are you using a separate usb-to-serial adapter for your serial console - or have you anything connected to one of the UART pins? It gets me sometimes when I have a separate UART connected to the tx/rx pins - can't remember the details off-hand but one of these pins needs to be toggled by the flasher, and the serial adapter prevents it. It fails with a timeout similar to yours - I have to disconnect the UART and reflash.

[Gina]

Yes, port is correct.

I don't have anything connected to the UART pins.  In fact this is with the ESP32 connected to nothing other than the USB.  I'm using the standard Arduino IDE upload,

[jiberjaber]

You won't get any response to SSH or ping unless you coded to provide those services. If you can still see the web page then the device is still working but sounds like it's some odd conflict somewhere, timing or some such.

[wimvb]

One more thing to try: erase the flash. Locate the esptool.py, then erase the esp flash. I believe the syntax is

esptool.py —port /dev/s0 erase_flash

After that check if the esp is visible in the usb devices list.

more info on the Randomnerdstutorial site, getting started with esp32

[Gina]

Thanks.  Found :- [SOLVED] Failed to connect to ESP32: Timed out waiting for packet header

I guess I've worn out the BOOT push button with so many uploads!  Seems strange that a capacitor on the EN pin does what pressing the BOOT button does but sobeit...  I'll try it and also a good little mod anyway.

[Gina]

Connected 10µF capacitor between EN and Gnd with jumper wires and tried uploading but still no joy!  I don't think the device is dud because the USB port is recognised and it got as far as wanting the BOOT button pressed.  OTOH something is wrong.  I'm expecting 2 more ESP32 modules in the post so I'll try those.

[stash_old]

OK - a couple of common problems

1.Your code, although it compiled/linked ok, is causing the device to abort and restart - you have to set the port monitor baud rate to the default for that device and you should see the boot process and if it is looping. Google the RST codes to see what they mean. https://esp32.com/viewtopic.php?t=11439&start=10

2. Early in your code you could have a "deadly embrace" loop which is not allowing the normal boot procedure.

If I get either of the above I compile a safe small program(blink) and wait until just before the upload (so after linking) and then press the Boot button - you might have to do this process a couplt of times to get the timing right.

 

[jiberjaber]

Good call on the blink upload, I too have used that in the past to get going but if you can't get it in to a state to upload the code it might be a struggle to get it on there.  Certainly look at the 'old fashioned way' of getting the code on by shorting the GPIO_00 to ground before power up and holding it there.

https://circuits4you.com/2016/12/14/reset-programming-esp8266/

 

[stash_old]

Never tried this but might help (part is what I said above) https://esp32.com/viewtopic.php?f=2&t=5682&start=10

What module/make of ESP32 are you actually using?

[Gina]

JZK ESP-32S ESP32 Development Board 2.4GHz WiFi + Bluetooth Dual Mode Antenna Module with Ultra Low Power

I have 3 of them.  One is currently working fine and talking to MQTT and Node-RED Dashboard.  The two I'm trying to upload to have been working fine and I have uploaded to them dozens of times as I've been experimenting.  Something has happened to stop that working but I don't know what.  I don't really want to try the one that's working in case that fails too and I lose the working system (Living Room in Dashboard display).  

[Gina]

Made a mistake about when the new ESP32 modules were due for delivery - it's next Tuesday not today 😡

[wimvb]

If two devices show the same error at the same time, I would expect the error to be somewhere else, ie it’s not the devices that are at fault. I use uPython, but I also regularly flash new firmware to the devices I use for testing. And while experimenting, I regularly swap from uPython to the Arduino IDE and back. Usually erasing flash with the esptool solves any connection problems. And don’t ignore possible faulty usb cables.

[stash_old]

Have you a std Windows OS IDE set up - if so just test the same happens when the device is plugged into that - again use  blink to start with.

If case you haven't :-

1. Have you upgraded the IDE since the last working upload ?

2. Have you updated the Mint Linux system

3. Try another cable but POWER off the Linux box first

4. If you have one try an older device not ESP32 that it doesn't matter if you corrupt it - does that work.

5. Try stopping the MQTT broker before doing an IDE upload

All long shots 😞

[Gina]

I was going to say the USB cable is alright as it's currently running the ESP32 Living Room sensor but that only needs power the data is over WiFi so it could be that the cable has developed a fault in one of the data lines.  I can't find another micro USB cable so have ordered some more.

[Gina]

6 minutes ago, stash_old said:

Have you a std Windows OS IDE set up - if so just test the same happens when the device is plugged into that - again use  blink to start with.

If case you haven't :-

1. Have you upgraded the IDE since the last working upload ?

2. Have you updated the Mint Linux system

3. Try another cable but POWER off the Linux box first

4. If you have one try an older device not ESP32 that it doesn't matter if you corrupt it - does that work.

5. Try stopping the MQTT broker before doing an IDE upload

All long shots 😞

I don't have Windows at all other than in a Virtual Box on the Mint machine and I've never got USB to work from that.

1. No

2. No

3. See post above.

4. I do have an ESP8266 that I can try - worth risking a few quid.

5. Never stopped the MQTT broker when doing uploads before.

[Gina]

Just realised that I also have several old Arduinos that I don't care about but that's a different type of board altogether.  Would that be a realistic test?

[rogmet]

I think I've had the same problems as you before and I think I solved them by changing the "Tools... Programmer" option in the Arduino IDE to something older. I'm at work on the phone so can't check right now, sorry