fixed error in macro handling; speedup MQTT msg receive

README.md

@@ -8,38 +8,37 @@ Additionally, personal settings like WIFI SSID and Passphrase will be talken fro
 ## Hardware Requirements
 To be able to use DEEP_SLEEP functionality, you will most probably need a small hardware modification for you ESP board: connect pin D0 to RST pin. This will allow the ESP to wake up after the defined sleep time as defined in the `include/generic-config.h` file.  
 
+## An important notice on MQTT usage
+As the main intention of this program is to run on a MCU that is powered off (sleeping) most of the time, we will **only work with retained messages** here! This ensures that a client subscribing to a topic will receive the last value published "instantly" and does not need to wait for someone to publish "latest news".  
+**ATTENTION**: The current version **requires** retained messages for **all topics you subscribe to!** Not having retained message for a subscribed topic will lead to an endless loop until a message is being received. This behavior has been set up to speed up receiving messages for all subscribed topics after initiating the connection to the MQTT broker.
+
 ## Configuration
 In addition to the `platformio.ini` file, see header files in the `include/` folder for additional settings.  
 Configureable settings should be documented well enough there - hopefully ;-)
 
-### A short notice on MQTT usage
-As the main intention of this program is to run on a MCU that is powered off (sleeping) most of the time, we will **only work with retained messages** here! This ensures that a client subscribing to a topic will receive the last value published "instantly" and does not need to wait for someone to publish "latest news".
-
-### An even shorter notice on "instantly" receiving MQTT topics
-I've made the experience that it can take quite a while after subscribing to a MQTT topic on the broker until the actual value is being received by the ESP. "Quite a while" means an ESP uptime (millis) of about 27 seconds (see `WAIT_MILLIS_FOR_TOPICS_AFTER_BOOT` in file `mqtt-ota-config.h`). This is especially important for OTA updating, as we rely on some topics for the update process.
-
-## MQTT Topics used
-In order to run OTA updates, you will need at least the following MQTT topics on your broker (case sensitive) to be pre-created so ESP can subscribe to them:
+### MQTT Topics used
+In order to run OTA updates, you will need at least the following MQTT topics on your broker (case sensitive) to be pre-created with the default retained message so ESP can subscribe to them:
 
-* `topic/tree/OTAupdate` - default retained Value: **off**  
+* `topic/tree/OTAupdate` - default retained Message: **off**  
 This will be translated to a bool variable in the sketch. You will need to set the topic value either to "on" or "off". During normal operation, this Topic needs to be set to "off". If you want to run an OTA-update on your ESP, set it to "on" (retained).  
 After a successful update, the ESP will reset this flag to "off".
 
-* `topic/tree/OTAinProgress` - default retained Value: **off**  
+* `topic/tree/OTAinProgress` - default retained Message: **off**  
 This is a helper flag topic required by the ESP.
 
-* `topic/tree/OTAstatus` - default Value: none  
-The ESP will publish OTA status strings here. No need to pre-create this topic.
+* `topic/tree/OTAstatus` - default Message: none  
+The ESP will publish OTA status strings here. No need to pre-create this topic, sketch does not subscribe to it (only publish).
 
-* `topic/tree/Vcc` - default Value: none  
+* `topic/tree/Vcc` - default Message: none  
 The sketch will publish the voltage measured on the 3.3V supply here. Note that the accuracy is quite low, but it is good enough to detect if the battery is running low when you supply the ESP in example by a LiFePo4 accumulator directly on the 3.3V pin.  
-If you want to improve accuracy, measure the actual voltage with a multimeter and adopt the `VCCCORRDIV` in the `hardware-setup.h` file.
+If you want to improve accuracy, measure the actual voltage with a multimeter and adopt the `VCCCORRDIV` in the `hardware-setup.h` file.  
+Note that we do not subscribe to this topic, we only publish to it.
 
-## Importance of `ClientName` Setting
+### Importance of `ClientName` Setting
 Note that the `ClientName` configured in the `platformio.ini` file will also be used as the hostname reported to your DHCP server when the ESP fetches an IP-address. This is especially important, as OTA-flashing will also require your networking environment to be able to resolve this hostname to the ESP's IP-address!  
 See `upload_port`setting in the `platformio.ini` file. If you're having troubles with OTA-flashing, you might want to check that first by pinging the configured `ClientName`.
 
-## Compiling and flashing walkthrough
+### Compiling and flashing walkthrough
 I will give a rough walkthrough on the first steps, assuming you have a working PlatformIO environment:
 
 * Prepare system environment variables with WIFI and MQTT Data:
@@ -67,8 +66,33 @@ upload_flags = ${common_env_data.upload_flags}
 ```
 * Set topic `topic/tree/OTAupdate` retained "on"
 * wait for the ESP to start up (or reset your ESP)
-* When the ESP boots, it will slowly blink the onboard LED until all MQTT topics are received (`WAIT_MILLIS_FOR_TOPICS_AFTER_BOOT`)
+* When the ESP boots, it will slowly blink the onboard LED until all MQTT topics are received
 * After that, the onboard LED will start flashing rapidly. The ESP is now waiting for the new binary to be uploaded
 * Click "Upload" in PIO to compile and upload the new sketch
 * When the upload has finished, the ESP will boot the new sketch and finish the OTA update process
-* You can verify the status by reading the `topic/tree/OTAstatus` topic, which should throw the string "update_success"
+* You can verify the status by reading the `topic/tree/OTAstatus` topic, which should throw the string "update_success"
+
+### Adding your own MQTT topics to the sketch
+If you want to add your own MQTT topic subscription, you will need to adopt the following files:  
+
+* `include/mqtt-ota-config.h`  
+Define/declare your topic along with required vars here.  
+
+* `src/mqtt-ota-setup.cpp`  
+Define initial values of your vars here.  
+
+* `src/common-functions.cpp`  
+Include message handling of your topic(s) in the `MqttCallback` function by adding new "else if"´s:  
+```
+else if (String(topic) == your_defined_topic)
+```
+
+
+# Version History
+
+## Initial Release v1.0.0
+ATTN: OTA flashing did not work due to an error in macro handling!
+
+## Release 1.0.1
+- Fixed error in macro handling
+- Speedup receiving messages of subscribed topics

include/generic-config.h

@@ -10,7 +10,7 @@
 
 // Firmware Information
 #define FIRMWARE_NAME "PIO ESP8266 Template"
-#define FIRMWARE_VERSION "1.0.0"
+#define FIRMWARE_VERSION "1.0.1"
 
 // Serial Output configuration
 //

include/macro-handling.h

@@ -5,6 +5,7 @@
 #ifndef MACRO_HANDLING_H
 #define MACRO_HANDLING_H
 
-#define TEXTIFY(A) #A
+#define DOUBLEESCAPE(a) #a
+#define TEXTIFY(a) DOUBLEESCAPE(a)
 
 #endif //MACRO_HANDLING_H

include/mqtt-ota-config.h

@@ -19,9 +19,9 @@
 // Message buffer for incoming Data from MQTT subscriptions
 extern char message_buff[20];
 
-// Required time to fetch topics after a boot
-// Important during OTA-flash
-#define WAIT_MILLIS_FOR_TOPICS_AFTER_BOOT 27000
+// Keep an eye on subscribed / received topics
+extern unsigned int SubscribedTopics;
+extern unsigned int ReceivedTopics;
 
 // MQTT Topic Tree prepended to all topics
 #define TOPTREE "HB7/Test/"

src/common-functions.cpp

@@ -40,10 +40,12 @@ void MqttCallback(char *topic, byte *payload, unsigned int length)
         if (msgString == "on")
         {
             OTAupdate = true;
+            ReceivedTopics++;
         }
         else if (msgString == "off")
         {
             OTAupdate = false;
+            ReceivedTopics++;
         }
         else
         {
@@ -56,10 +58,12 @@ void MqttCallback(char *topic, byte *payload, unsigned int length)
         if (msgString == "on")
         {
             OtaInProgress = true;
+            ReceivedTopics++;
         }
         else if (msgString == "off")
         {
             OtaInProgress = false;
+            ReceivedTopics++;
         }
         else
         {

src/main.cpp

@@ -48,6 +48,7 @@ bool MqttSubscribe(const char *Topic)
   if (mqttClt.subscribe(Topic))
   {
     DEBUG_PRINTLN("Subscribed to " + String(Topic));
+    SubscribedTopics++;
     mqttClt.loop();
     return true;
   }
@@ -62,6 +63,9 @@ bool MqttSubscribe(const char *Topic)
 // Function to connect to MQTT Broker and subscribe to Topics
 bool ConnectToBroker()
 {
+  // Reset subscribed/received Topics counters
+  SubscribedTopics = 0;
+  ReceivedTopics = 0;
   bool RetVal = false;
   int ConnAttempt = 0;
   // Try to connect x times, then return error
@@ -117,14 +121,22 @@ void setup()
   // Setup MQTT Connection to broker and subscribe to topic
   if (ConnectToBroker())
   {
-    DEBUG_PRINTLN("Connected to MQTT broker, fetching topics..");
-    mqttClt.loop();
+    // New connection to broker, fetch topics
+    // ATTN: will run endlessly if subscribed topics
+    // does not have retained messages and no one posts a message
+    DEBUG_PRINT("Waiting for topics..");
+    while (ReceivedTopics < SubscribedTopics)
+    {
+      DEBUG_PRINT(".");
+      mqttClt.loop();
 #ifdef ONBOARD_LED
-    // broker connected - blink twice
-    ToggleLed(LED, 200, 4);
+      ToggleLed(LED, 100, 2);
 #else
-    delay(500);
+      delay(100);
 #endif
+    }
+    DEBUG_PRINTLN("");
+    DEBUG_PRINTLN("All topics received.");
   }
   else
   {
@@ -148,7 +160,6 @@ void setup()
 
 #ifdef ONBOARD_LED
   // Signal setup finished
-  delay(300);
   ToggleLed(LED, 200, 6);
 #endif
 }
@@ -165,7 +176,22 @@ void loop()
   {
     if (ConnectToBroker())
     {
-      mqttClt.loop();
+      // New connection to broker, fetch topics
+      // ATTN: will run endlessly if subscribed topics
+      // does not have retained messages and no one posts a message
+      DEBUG_PRINT("Waiting for topics..");
+      while (ReceivedTopics < SubscribedTopics)
+      {
+        DEBUG_PRINT(".");
+        mqttClt.loop();
+#ifdef ONBOARD_LED
+        ToggleLed(LED, 100, 2);
+#else
+        delay(100);
+#endif
+      }
+      DEBUG_PRINTLN("");
+      DEBUG_PRINTLN("All topics received.");
     }
     else
     {
@@ -191,18 +217,6 @@ void loop()
   // only loop through OTA function until finished (or reset by MQTT)
   if (OTAupdate)
   {
-    if (millis() < WAIT_MILLIS_FOR_TOPICS_AFTER_BOOT)
-    {
-      // this delay is required to make sure that we know our correct status before doing anything..
-      // shorter delay will not work reliably (fetching all MQTT topics takes a long time)
-      DEBUG_PRINTLN("Sketch just booted, delaying OTA operation until all MQTT topics arrived..");
-#ifdef ONBOARD_LED
-      ToggleLed(LED, 1000, 2);
-#else
-      delay(2000);
-#endif
-      return;
-    }
     if (OtaInProgress && !OtaIPsetBySketch)
     {
       DEBUG_PRINTLN("OTA firmware update successful, resuming normal operation..");

src/mqtt-ota-setup.cpp

@@ -14,4 +14,6 @@ bool OtaInProgress = false;
 bool OtaIPsetBySketch = false;
 bool SentOtaIPtrue = false;
 #endif
-float VCC = 3.333;
+float VCC = 3.333;
+unsigned int SubscribedTopics = 0;
+unsigned int ReceivedTopics = 0;

src/setup-functions.cpp

@@ -43,6 +43,8 @@ void wifi_setup()
     DEBUG_PRINTLN("WiFi connected");
     DEBUG_PRINT("Device IP Address: ");
     DEBUG_PRINTLN(WiFi.localIP());
+    DEBUG_PRINT("DHCP Hostname: ");
+    DEBUG_PRINTLN(WIFI_DHCPNAME);
 #ifdef ONBOARD_LED
     // WiFi connected - blink once
     ToggleLed(LED, 200, 2);