Feature Req: Multiple concurrent MQTT client connections

[markterrill]

Summary
Multiple concurrent MQTT client connections that you can selectively subscribe and publish to.

Use case
Customers are asking for integration of their IoT devices with the likes of Home Assistant. We use MQTT bound to GCP for our command and control which precludes being able to configure the device's MQTT server as their local (or cloud) MQTT instance.
You can't chain their local (or cloud) instance with GCP without significant middleware for certs and customer support. You could forward requests from a pub/sub topic if their MQTT was cloud based, but that incurs extra cost. The cheapest and most efficient way to support these customers is simply having another MQTT client connection handle on the end MOS device.

Personally I see this as very useful more broadly for separating command/control from how you may wish to handle status notifications. IE MOS cloud for fleet management, AWS/GCP/local mqtt etc for status notifications and client app driven configuration changes. Forwarding is cool, but costs $, this costs nothing aside from a few cpu cycles on the iot device.

Lastly, while I'm at it, there are some security/transparency advantages. Some sensitive customer data may not be allowed to transmit to a cloud or third party service. A code audit would show the separation of command and status notifications and you could easily redact sensitive portions from any status requests from the command channel. This would open it up to lower security clearance requirements that mandate local status operation but enable periodic remote management/updates. It'd also satisfy what we call PH1 / Public Health Level 1 requirements that no command/control systems see patient information. I don't currently have that requirement, though one product currently using Particle P1 hardware may require operating on naval craft later in the year.

Implementation ideas
I'd rate my C as moderate, so bear with me re my ideas on mqtt.c.

• Maybe, possibly, a new mqtt library is introduced that enables a third independent mqtt client to be created. Code is nearly a copy/paste, just different variable and function names. ie mqtt_customer or mqtt_status or mqtt_alt
• Alternatively, building out mqtt.c will require an array like the pwm library to hold the multiple connections and be able to specify a connection via a human friendly facing known element such as the server URL ("mqtt.myserver.com") or mqtt server config handle (ie mqtt3). The various front facing functions like pub/sub etc would ideally have an optional argument for this friendly handle for backwards compatibility. The primary (or backup) server logic would remain the same, and would be the default connection used if no argument was specified.
• A nice to have to tidy up userland code would be a 'mirror' argument to publish that you could use as the friendly handle. Simple purpose being that if the handle is 'mirror' then it sends it to the primary/backup first, then sends to the mqtt_alt server as well. Removes duplication in userland code, though still provides the flexibility to determine which userland messages go where.

[pimvanpelt]

@markterrill totally hijacking your thread here, but seeing as you are referencing Home Assistant (I assume hass.io et al), I am just about to finish a library for Mongoose OS that exposes lots of things (sensors, binary_sensors, switches, etc) directly to HA in MQTT. You may find that useful? Mail me [email protected] if you'd like to discuss, or join in on gitter.im cesanta/mongoose-os and ping me there.

(sorry for the drive-by comment)

[rojer]

a8f2b0e
you can now create new MQTT connections

[markterrill]

@rojer thanks, looks awesome!

I'll happily contribute some docs. After a number of scrolls through the code, am I correct in saying the approach to use the new code is directly access mqtt_conn? IE, in rough pseudo code:

/* 
Chosen conn ID 1 to be after primary conn at 0
Note, used mgos_sys_config_get_mqtt3 !
*/
marksnew_conn = mgos_mqtt_conn_create(1, mgos_sys_config_get_mqtt3());  

mgos_mqtt_conn_pub(marksnew_conn, myTopic, mg_mk_str_n(marksMessage, len), qos,
                            retain);

[markterrill]

I'm sure I'll find out when I directly use it, but asking in case that's not the way it's meant to be used..

[markterrill]

silly me. stdout / stderr use the active cfg.

[rojer]

yes, you can use the mqtt_conn interface now to create additional connections.