Latency, ESP32-S3

[ashojon]

I would like to build a MIDI keyboard based on an ESP32-S3 and use the MIDI-USB and MIDI-BLE functions of Control-Surfaces. Before designing a PCB for it, I would like to ask whether there might be any latency issues. Have you ever conducted latency measurements ? Is there a way to influence or reduce latency?

[tttapa]

I haven't done any precision testing, but in my experience, you can expect a two-way latency between 10 and 30 ms for MIDI over BLE.
This can be improved by changing the transmission timeout:

Control-Surface/src/MIDI_Interfaces/BLEMIDI/ThreadedBLEMIDISender.hpp

Lines 49 to 51 in b8a30b4

	/// Set the timeout, the number of milliseconds to buffer the outgoing MIDI
	/// messages.
	void setTimeout(std::chrono::milliseconds timeout);

By default, messages are buffered for 10 ms and then sent all at once. This improves throughput and limits the number of BLE packets need to be transmitted.

Another parameter is the BLE connection interval, which determines how often data can be exchanged between the Central and the Peripheral. By default, Control Surface sets the preferred connection interval to 15 ms (0x000C), but this is not always honored by the Central. You can set it to 7.5 ms (0x0006), which is the minimum possible value, but again, the device you're communicating with may not support this. For example, my Samsung phone always switches back to 15 ms, even if I request a lower value. By sending a new connection parameter update request whenever the Central changes the interval, Android does finally accept 7.5 ms. You can find a patch that does this below (for the NimBLE backend).

For MIDI over USB, Control Surface uses the USB library that's included with the espressif/arduino-esp32 Core, I haven't investigated the details of their implementation.

// BLE MIDI loopback example for latency measurements
#define CS_USE_NIMBLE

#include <Control_Surface.h>
#include <MIDI_Interfaces/BluetoothMIDI_Interface.hpp>

BluetoothMIDI_Interface midi;
MIDI_Pipe pipes;

void setup() {
  midi.setName("MIDI Adapter");
  midi.ble_settings.connection_interval.minimum = 0x0006;
  midi.ble_settings.connection_interval.maximum = 0x0006;
  using namespace std::chrono_literals;
  midi.setTimeout(1ms);
  midi >> pipes >> midi;
  MIDI_Interface::beginAll();
  pinMode(LED_BUILTIN, OUTPUT);
}

void loop() {
  MIDI_Interface::updateAll();
  digitalWrite(LED_BUILTIN, midi.isConnected() ? HIGH : LOW);
}

diff --git a/src/MIDI_Interfaces/BLEMIDI/ESP32-NimBLE/events.hpp b/src/MIDI_Interfaces/BLEMIDI/ESP32-NimBLE/events.hpp
index 2ab2a7e90..9baaa66df 100644
--- a/src/MIDI_Interfaces/BLEMIDI/ESP32-NimBLE/events.hpp
+++ b/src/MIDI_Interfaces/BLEMIDI/ESP32-NimBLE/events.hpp
@@ -22,6 +22,7 @@ struct MIDIBLEState {
     uint16_t midi_characteristic_handle = invalid_handle;
     uint8_t address_type = 0;
     BLESettings settings;
+    uint32_t requested_conn_itvl = 0;
 };
 
 extern MIDIBLEState *state;
diff --git a/src/MIDI_Interfaces/BLEMIDI/ESP32-NimBLE/events.ipp b/src/MIDI_Interfaces/BLEMIDI/ESP32-NimBLE/events.ipp
index 17cffcdc9..2aba7bfcf 100644
--- a/src/MIDI_Interfaces/BLEMIDI/ESP32-NimBLE/events.ipp
+++ b/src/MIDI_Interfaces/BLEMIDI/ESP32-NimBLE/events.ipp
@@ -227,6 +227,7 @@ inline int cs_midi_ble_gap_callback(struct ble_gap_event *event, void *) {
                 }
                 // Update the connection parameters
                 update_connection_params(conn_handle, settings);
+                cs::midi_ble_nimble::state->requested_conn_itvl = 0;
                 if (auto *inst = cs::midi_ble_nimble::state->instance)
                     inst->handleConnect(cs::BLEConnectionHandle {conn_handle});
             } else {
@@ -254,10 +255,16 @@ inline int cs_midi_ble_gap_callback(struct ble_gap_event *event, void *) {
         case BLE_GAP_EVENT_CONN_UPDATE: {
             ESP_LOGI("CS-BLEMIDI", "connection updated; status=%d ",
                      event->conn_update.status);
+            const auto conn_handle = event->conn_update.conn_handle;
             struct ble_gap_conn_desc desc;
-            auto rc = ble_gap_conn_find(event->conn_update.conn_handle, &desc);
+            auto rc = ble_gap_conn_find(conn_handle, &desc);
             assert(rc == 0);
             cs::midi_ble_nimble::print_conn_desc(&desc);
+            const auto &settings = cs::midi_ble_nimble::state->settings;
+            if ((desc.conn_itvl < settings.connection_interval.minimum ||
+                 desc.conn_itvl > settings.connection_interval.maximum) &&
+                (cs::midi_ble_nimble::state->requested_conn_itvl++ < 2))
+                update_connection_params(conn_handle, settings);
         } break;
 
         // Advertising done (e.g. after reaching the specified timeout)