2 hours ago, teoria_del_big_bang said:

If I use Ubutnu (KSTARS, EKPOS) only issue I really have is I cannot shut the CPU down remotely

Yes you can - at worst just use SSH (puttyy from Windows) and do a "sudo poweroff" or just start a command line via VNC Viewer and do the same. You will be asked for the password - although there are unsafe around that too!

Radek has issued this warning and he IS an expert on computer security https://indilib.org/forum/astroberry/7627-raspberry-pi-trojan-going-wild.html

12 hours ago, teoria_del_big_bang said:

If somebody could now make a stand alone application that encompasses all the good aspects of say APT, Kstars, Stellarium, EKOS etc that is robust I would pay for that no problem and would be happy

CCDCIEL - runs on anything(Windows/Linux or Mac) connects to both Indi or Ascom (or both with certain limitaions - Ascom being the limitation). Plus written by the same author as CDC which intergrates with CDCCIEL. A bit clinky but has more features than APT but read the documentation FIRST to see if it's for you. Doesn't need Ekos/Kstars ,intergrates fine with Astroberry and Indiserver stand alone type set up's.

Remember you are using a RPI (£50) - if you want bells and whistles(2 simultaneous Screens)  you will have to pay for it and run a more powerful (and normally lots more £'s)  system and use Ubuntu version of Indi or Windows/Ascom. You dont "get owt for nowt".

Plus this may help some https://help.realvnc.com/hc/en-us/articles/360016572531-How-do-I-use-VNC-Viewer-in-full-screen-mode-on-all-of-my-screens-

13 minutes ago, vlaiv said:

What I need is simple - shoot X subs, Y seconds each and store them on USB attached SSD or similar with RPI.

I think you may find Indigosky Web interface gives your all that just the command line running on RPI (No Ekos/Kstars etc) -worth a look if you dont want to create your own 🙂

18 hours ago, vlaiv said:

Get myself a RPI4, put some bare distro on it without GUI and INDI server

Then how do you expect to control you devices - not using the GUI(desktop) is no problem but you still need Indiserver (command line only) or something similar to control your camera/mount. Yes you can control your mount directly from your phone without Indiserver(or similar) but control of a non DSLR camera maybe a little harder.

Indigosky ( paid version works on Apple Iphones /Ipad but doesn't display Blob's) allows you to control devices via a web browser ,including basic image taking (but not DSLR long exp), but how feasible on small screen I cannot comment on.

You might also want to check that you are NOT sending random "Nan" values from the ESP - I have found that some devices/sensors provide rubbish non numeric values (hence NAN - not a number) which if not filtered out somewhere(best done in the ESP) before displaying the actual data create problems.

I agree that there seems very little reason to provide data from most devices every 5s - obviously any rain detection needs to send its started raining ASAP however the timings can and should be controlled ,IMHO, from within the ESP.

I doubt you "large" display is causing your issues as many Home Automation systems have far more complicated display layouts. If you still suspect that is the issue you can of course split the display into tabbed windows easily in Node-red Dashboard and see if that is really the issue.

But I go with Wimvb first try the 60s refresh/send loop.

Sorry Mick at your troubles but you are asking for something nearly impossible. Unless all AP setups are the same (camera,mount,guide camera etc AND each image is exactly the same from every person) - sorry its wont happen. Granted the Help files on some softwhere are a bit naff. APT has its own forum ,many here also subscribe to it, that forum is specific for APT and Yoda is a very helpful guy.   People could say the same about basic Astronomy for newcomers.

ALL Platesolving software depends on correct Focal length being entered - some require pixel size etc. APT only requires your correct focal length,as someone has already pointed out,so long as it recognises your camera to get the other paramters. Index's are vital and MUST match the f/l of your OTA.

All nig help to me was to do platesolving using the mount ,camera simulators and CDC(or other). Then load an image you have taken,make sure the f/l and camera details are correct. Then position the mount on the object you think the image was taken at. Then try Platesolving using Platesolve2 (this is what you say you have). Still having trouble download an image here and I am sure someone will test out the image. Just a note - PS2 is good at "near" platesolving but not 'Blind" solving. PS2 will do it but it takes ages - 5-20mins. Both ASPS and PS2 can be run "stand alone" to see if there is a problem - like wrong index's.  Also APT now allows ASTAP platesolver which is very fast but a bit "fussy" about round stars etc IMHO - Again ASTAP can be run stand alone without APT. Also ASTAP only uses 1 index file (min) but can use a secnd much larger catalogue so no problem with downloading wrong index's.

Its frustrating for you but getting Platesolving working works wonders (DD advert LOL)- so keep trying you will get there.

So I suggest if you think you have the correct info set up - load your image here (or better the other thread) and lets see if it solves:-)

A bit of "an eye in the storm" there - HPA reading.

2 hours ago, Gina said:

I think I may be beginning to see the light!

This is an extract of the Living Room client sketch.

WiFiClient espClient;
PubSubClient client(espClient);
long lastMsg = 0;
char msg[50];
int value = 0;

And this from the MQTT_Obsy_ESP32_and_BME280_and_DHT22 sketch that runs the outside and obsy topics.

WiFiClient espClient;
PubSubClient client(espClient);
long lastMsg = 0;
char msg[50];
int value = 0;

So I guess this is calling both the same ID of "espClient".  I hadn't taken in this part of the concept.

I guess I need to change one to something else.

Forgive me but I think you are changing the wrong bit it is the MQTT connect statement that has the ID - although I am having a bad day so probably wrong. 🙂

https://pubsubclient.knolleary.net/api.html

API Documentation

This site is good but not always running for some read.  https://www.hivemq.com/blog/mqtt-essentials-part-3-client-broker-connection-establishment/#:~:text=The client identifier (ClientId) identifies each MQTT client,ID should be unique per client and broker.

"

ClientId

The client identifier (ClientId) identifies each MQTT client that connects to an MQTT broker. The broker uses the ClientID to identify the client and the current state of the client.Therefore, this ID should be unique per client and broker. In MQTT 3.1.1 (the current standard), you can send an empty ClientId, if you don’t need a state to be held by the broker. The empty ClientID results in a connection without any state. In this case, the clean session flag must be set to true or the broker will reject the connection."

client.connect("ESP8266Client")

The only thing I THINK I can suggest if to check for WiFi status in your loop - what happens if the wifi has disconnected the MQTT should fail. Just needs a Wifi status check and if not connected connect Wifi.  Unless I have missed something. Just move 3 m3 of firewood so brain and old bones complaining.

Just checked my old code I don't check Wifi status either LOL - rest very much like yours except I left myself this note

  //
    //     change client id else problems ******************************************
    //

And send data every 60secs

The MQTT client name/ID isn't dupilcated is it - has to be unique

20 minutes ago, Gina said:

Do you think I should try an ESP8266 for some of my clients?

The only reason I did was that 3.5yrs ago the ESP32 was new new to me and I would have expect,after initial teething,the esp32 would be better supported depending on the Manu.

Gut feelings are normally not far off :-).

Have you any debug messages (that you can switch on or add) in your Living Room ESP to show WiFi disconnects, MQTT broker connection failures etc - always a thin line having Debug messages as they can impact on the flow IMO.

A suggestion - you could switch off all your devices(clients) but 1 see how it behaves and add a new device in a orderly manner(say 1hr apart) checking for problems!

Never had a bad problem with MQTT on a RPI but then I use a RPI3 not a zero. Also ,as Skybadger says, you have to allow ,in your code,reconnects for Wifi and dont use too tight loops which block the ESP Stack but I haven't changed any of the default ESP stack or MQTT parameters. This was all using ESP8266(Wemos) not a ESP32. Zero's had a bad reputation for Wifi drop outs at one time (google Zero Wifi problems) but I would have thought they had sorted them by now (or maybe not) - assuming you are using Std Raspian not Mint etc on RPI zero.

Plus I think you can access the Broker LOG via MQTT-explorer or equiv (else Linux file in \var etc) to see any problems that might be shown.

15 minutes ago, Gina said:

That says it's Windows!

My underline.

So it does oh well i will butt out then

1 hour ago, Gina said:

Just realised that I also have several old Arduinos that I don't care about but that's a different type of board altogether.  Would that be a realistic test?

just to see if its something to do with the board - if you are using the same cable / software IDE etc then at least it points to the board. Doesn't matter which INO sketch as you are testing the upload -so Blink still ok. Just have to remember to select the correct board type before you compile/link

As for flashing perhaps this linux s/w (ps Never used it myself) will help http://iot-bits.com/esp32/esp32-flash-download-tool-tutorial/

Have you a std Windows OS IDE set up - if so just test the same happens when the device is plugged into that - again use  blink to start with.

If case you haven't :-

1. Have you upgraded the IDE since the last working upload ?

2. Have you updated the Mint Linux system

3. Try another cable but POWER off the Linux box first

4. If you have one try an older device not ESP32 that it doesn't matter if you corrupt it - does that work.

5. Try stopping the MQTT broker before doing an IDE upload

All long shots 😞

Never tried this but might help (part is what I said above) https://esp32.com/viewtopic.php?f=2&t=5682&start=10

What module/make of ESP32 are you actually using?

OK - a couple of common problems

1.Your code, although it compiled/linked ok, is causing the device to abort and restart - you have to set the port monitor baud rate to the default for that device and you should see the boot process and if it is looping. Google the RST codes to see what they mean. https://esp32.com/viewtopic.php?t=11439&start=10

2. Early in your code you could have a "deadly embrace" loop which is not allowing the normal boot procedure.

If I get either of the above I compile a safe small program(blink) and wait until just before the upload (so after linking) and then press the Boot button - you might have to do this process a couplt of times to get the timing right.

Dont forget the current problem with RPI Kernel,Indi and FTDI adapters - https://github.com/raspberrypi/linux/issues/3779 not cured last time I checked

13 hours ago, discardedastro said:

I can't find anything on the NASA/IBM story but IBM are superb at press releases and have been strong promotors of MQTT for a long while.

Edit: I did actually find it - IBM were promoting a "Cognitive Telescope" project which appears to have turned into a few CS undergrad projects and went no further. MQTT was part of the picture largely to align with IBM's cloud telemetry portfolio. It's all vanished off the internet now as far as I can tell - just a lot of dead links that don't show up in the internet archive etc.

But MQTT is a perfectly good message broker which is suitable for control applications if and only if you put an application layer on top of it to handle the fact that it employs UDP-style semantics by default, if you want to talk network protocols. If a TCP packet does not arrive at the final destination, the sender is aware and can take action. If a MQTT frame is sent to a destination but not acknowledged, it is considered delivered. QOS 1 in MQTT only guarantees delivery semantics between broker and client, not between two clients, and does not guarantee delivery if your client suffers from byzantine failure modes (e.g. received it, parsed the frame and then blew up - this is considered received - compare to HTTP RESTful APIs where a bidirectional exchange between the two final endpoints occurs to validate proper receipt and processing; even QOS 2 in MQTT doesn't guarantee application delivery).

If NASA want to use it for control applications with an application that handles all this atop the basic protocol then that's all jolly good but it is not something you get for free with MQTT - you have to think about it as a developer - and therefore where I'd generally advise against using it as a building block for applications where you need that - because many other equally-accessible things can do that "for free" 😊 MQTT is superb for fire-and-forget telemetry, which is what it was designed for.

It's worth noting MQTT isn't alone in having some of these problems and error management as part of your control protocol is essential regardless of transport - MQTT (as well as similar messaging/fanout brokers in common modes of operation) just makes you handle more of this yourself than you get out of the box with other approaches.

ASCOM over MQTT is a protocol dumpster fire waiting to happen. With my network/systems engineering hat on, I've seen "oh just bung protocol X into protocol Y as a transport/encapsulation to modernise it" go horribly, horribly wrong plenty of times before, even with active protocol conversion going on. Lots of assumptions made by applications and even protocol semantics can break in incredibly hard to debug/fix ways at the application layer. ASCOM Alpaca is a better way to do ASCOM-over-IP since they can do protocol-aware translation at the network boundary, though ASCOM, being a basically "closed" ecosystem, isn't the way to go for this really imho.

The whole ecosystem for remote instrument control is a mess, but stuff like INDI (which is network-native and so has an understanding of these sorts of client/server-at-a-distance protocol considerations baked in, as well as being properly open-source and open-community) is the way to go. I'd stick to HTTP RESTful interfaces where required for control or ensure you're doing application level confirmation of commands in an RPC-style manner to ensure that the remote application both received your message and is able to act on it.

Doing local fallback for safety is a sensible approach as Gina is already doing.

Thanks for you verbose opinion 🙂

4 hours ago, discardedastro said:

It doesn't have very good mechanics for handling failures; well, any, really.

That must be why Nasa/IBM are suggesting it to be used across the world wide telescope control system 🙂 - they even suggest using ASCOM across MQTT . Its TCP that carries the error correction at a physical level and its up to your error procotol that is carried by MQTT to control error control. I used it for Moonlite Focuser control via Nano or ESP and it fine.  MQTT is a fast one to many and many to many message distribution protocol.

Dont know if its the same Wind Vane Widget but https://flows.nodered.org/flow/b9a57175ac5a5e240c9916bcce136dca

3 hours ago, Gina said:

Node-RED Dashboard should work for control but has a crude display and too few "widgets" to display data

True statement but make sure you look around for the Widgets to display data - plenty on Github and there are other "dashboards" or write your own!

For me and my Astro stuff I want functionality/reliabilitty/setup speed/ease/no Internet dependence -  not looks so the std Dashboard is fine for me - for a super all singing "dashboard"/system I would use something like OpenHab.

For std sensors/Motor control I dont even bother with Json formatting - no point IMO especially when doing something like Moonlite focuser control protocol - if you are clever you can use the MQTT Tipic to get your message routed to the correct piece of Arduino Code (or any other code) without having to decode - e.g. topics of \moonlite\movein or \moonlite\moveout can be used to "perform" a piece of code specifically to do that function - if you see what I mean. Anyways thats my way, everyone has there own opinion.

So by tomorrow I expect to see a fully working system - LOL

The tutorial for the ESP8266 on the site you are using shows it 0 dont know why the esp32 doesn't https://randomnerdtutorials.com/esp8266-and-node-red-with-mqtt/