Add encrypted TLS connection support using certificates or PSK

[avillacis]

This pull request adds support for establishing an encrypted TLS connection, if the underlying AsyncTCP library supports it and if the ASYNC_TCP_SSL_ENABLED is enabled, as used by several AsyncTCP forks implementing TLS. The TLS support simply forwards setting certificates and keys to the corresponding API calls in the underlying AsyncTCP client connection. Tested with my own reimplementation of the AsyncTCP API, AsyncTCPSock, but as the API is fully compatible with AsyncTCP TLS forks, it should work with them as well.

[proddy]

nice. I'll try it out too...

[martibc16]

Hi @avillacis,
I tested your library because I'm interested in using PSK with an ESP32.

What I saw is that the SSL options are disabled by default via the #if ASYNC_TCP_SSL_ENABLED
However, on the AsyncTCP library, which is the one being used for the ESP32:

#ifdef ESP32
#include <AsyncTCP.h>
#include <freertos/semphr.h>

This parameter is not mentioned anywhere in the AsyncTCP library (it is mentioned but not activated for the ESP8266). So the question is where should you define ASYNC_TCP_SSL_ENABLED to be able to use PSK in an ESP32?

[avillacis]

Hi @avillacis, I tested your library because I'm interested in using PSK with an ESP32.

What I saw is that the SSL options are disabled by default via the #if ASYNC_TCP_SSL_ENABLED However, on the AsyncTCP library, which is the one being used for the ESP32:

#ifdef ESP32
#include <AsyncTCP.h>
#include <freertos/semphr.h>

This parameter is not mentioned anywhere in the AsyncTCP library (it is mentioned but not activated for the ESP8266). So the question is where should you define ASYNC_TCP_SSL_ENABLED to be able to use PSK in an ESP32?

Sorry, my library was missing the official method to enable compile macros in Arduino. This is fixed now at yubox-node-org/AsyncTCPSock@9f82a7e .

In Arduino IDE, the official method to enable compile macros is to create a new file in the sketch project, with the name build_opt.h. The contents of this file should be all the #defines that should be used when compiling the project, one per line. For example, to enable the ASYNC_TCP_SSL_ENABLED compile macro, the file should contain (at least) one line like this:

-DASYNC_TCP_SSL_ENABLED=1

Multiple macros may be defined in this way, one per line.

[martibc16]

Hi @avillacis, I tested your library because I'm interested in using PSK with an ESP32.
What I saw is that the SSL options are disabled by default via the #if ASYNC_TCP_SSL_ENABLED However, on the AsyncTCP library, which is the one being used for the ESP32:

#ifdef ESP32
#include <AsyncTCP.h>
#include <freertos/semphr.h>

This parameter is not mentioned anywhere in the AsyncTCP library (it is mentioned but not activated for the ESP8266). So the question is where should you define ASYNC_TCP_SSL_ENABLED to be able to use PSK in an ESP32?

Sorry, my library was missing the official method to enable compile macros in Arduino. This is fixed now at yubox-node-org/AsyncTCPSock@9f82a7e .

In Arduino IDE, the official method to enable compile macros is to create a new file in the sketch project, with the name build_opt.h. The contents of this file should be all the #defines that should be used when compiling the project, one per line. For example, to enable the ASYNC_TCP_SSL_ENABLED compile macro, the file should contain (at least) one line like this:

-DASYNC_TCP_SSL_ENABLED=1

Multiple macros may be defined in this way, one per line.

The quick fix is really apreciated :)

For anyone on the same situation as me: I'm using platformIO, to enable the macro that @avillacis is mentioning you just have to add on the platform.ini file the following configuration:

build_flags=
	-DASYNC_TCP_SSL_ENABLED=1

Now that everything compiles with the setPSK() method I've created a sketch to test functionality:

/*
This example uses FreeRTOS softwaretimers as there is no built-in Ticker library
*/


#ifndef ASYNC_TCP_SSL_ENABLED
#error The macro ASYNC_TCP_SSL_ENABLED has not been correctly enabled in your environment!
#endif

#include <WiFi.h>
extern "C" {
	#include "freertos/FreeRTOS.h"
	#include "freertos/timers.h"
}
#include <AsyncMqttClient.h>

#define WIFI_SSID "some wifi SSID"
#define WIFI_PASSWORD "a password"

#define MQTT_HOST IPAddress(192, 168, 1, 223)
#define MQTT_PORT 1884
#define MQTT_KEY "a key"
#define PSKIDENT "an identity"

AsyncMqttClient mqttClient;
TimerHandle_t mqttReconnectTimer;
TimerHandle_t wifiReconnectTimer;

void connectToWifi() {
  Serial.println("Connecting to Wi-Fi...");
  WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
}

void connectToMqtt() {
  Serial.println("Connecting to MQTT...");
  mqttClient.connect();
}

void WiFiEvent(WiFiEvent_t event) {
    Serial.printf("[WiFi-event] event: %d\n", event);
    switch(event) {
    case SYSTEM_EVENT_STA_GOT_IP:
        Serial.println("WiFi connected");
        Serial.println("IP address: ");
        Serial.println(WiFi.localIP());
        connectToMqtt();
        break;
    case SYSTEM_EVENT_STA_DISCONNECTED:
        Serial.println("WiFi lost connection");
        xTimerStop(mqttReconnectTimer, 0); // ensure we don't reconnect to MQTT while reconnecting to Wi-Fi
        xTimerStart(wifiReconnectTimer, 0);
        break;
    }
}

void onMqttConnect(bool sessionPresent) {
  Serial.println("Connected to MQTT.");
  Serial.print("Session present: ");
  Serial.println(sessionPresent);
  uint16_t packetIdSub = mqttClient.subscribe("halp", 2);
  Serial.print("Subscribing at QoS 2, packetId: ");
  Serial.println(packetIdSub);
  mqttClient.publish("halp", 0, true, "test 1");
  Serial.println("Publishing at QoS 0");
  uint16_t packetIdPub1 = mqttClient.publish("halp", 1, true, "test 2");
  Serial.print("Publishing at QoS 1, packetId: ");
  Serial.println(packetIdPub1);
  uint16_t packetIdPub2 = mqttClient.publish("halp", 2, true, "test 3");
  Serial.print("Publishing at QoS 2, packetId: ");
  Serial.println(packetIdPub2);
}

void onMqttDisconnect(AsyncMqttClientDisconnectReason reason) {
  Serial.println("Disconnected from MQTT.");

  if (WiFi.isConnected()) {
    xTimerStart(mqttReconnectTimer, 0);
  }
}

void onMqttSubscribe(uint16_t packetId, uint8_t qos) {
  Serial.println("Subscribe acknowledged.");
  Serial.print("  packetId: ");
  Serial.println(packetId);
  Serial.print("  qos: ");
  Serial.println(qos);
}

void onMqttUnsubscribe(uint16_t packetId) {
  Serial.println("Unsubscribe acknowledged.");
  Serial.print("  packetId: ");
  Serial.println(packetId);
}

void onMqttMessage(char* topic, char* payload, AsyncMqttClientMessageProperties properties, size_t len, size_t index, size_t total) {
  Serial.println("Publish received.");
  Serial.print("  topic: ");
  Serial.println(topic);
  Serial.print("  qos: ");
  Serial.println(properties.qos);
  Serial.print("  dup: ");
  Serial.println(properties.dup);
  Serial.print("  retain: ");
  Serial.println(properties.retain);
  Serial.print("  len: ");
  Serial.println(len);
  Serial.print("  index: ");
  Serial.println(index);
  Serial.print("  total: ");
  Serial.println(total);
}

void onMqttPublish(uint16_t packetId) {
  Serial.println("Publish acknowledged.");
  Serial.print("  packetId: ");
  Serial.println(packetId);
}

void setup() {
  Serial.begin(115200);
  Serial.println();
  Serial.println();

  mqttReconnectTimer = xTimerCreate("mqttTimer", pdMS_TO_TICKS(2000), pdFALSE, (void*)0, reinterpret_cast<TimerCallbackFunction_t>(connectToMqtt));
  wifiReconnectTimer = xTimerCreate("wifiTimer", pdMS_TO_TICKS(2000), pdFALSE, (void*)0, reinterpret_cast<TimerCallbackFunction_t>(connectToWifi));

  WiFi.onEvent(WiFiEvent);

  mqttClient.onConnect(onMqttConnect);
  mqttClient.onDisconnect(onMqttDisconnect);
  mqttClient.onSubscribe(onMqttSubscribe);
  mqttClient.onUnsubscribe(onMqttUnsubscribe);
  mqttClient.onMessage(onMqttMessage);
  mqttClient.onPublish(onMqttPublish);
  mqttClient.setServer(MQTT_HOST, MQTT_PORT);
  mqttClient.setPsk(PSKIDENT, MQTT_KEY);
  connectToWifi();
}

void loop() {
}

However if I upload the code on the ESP32 I get:

Connecting to Wi-Fi...
[D][WiFiGeneric.cpp:374] _eventCallback(): Event: 0 - WIFI_READY
[D][WiFiGeneric.cpp:374] _eventCallback(): Event: 2 - STA_START
[WiFi-event] event: 0
[WiFi-event] event: 2
[D][WiFiGeneric.cpp:374] _eventCallback(): Event: 4 - STA_CONNECTED
[WiFi-event] event: 4
[D][WiFiGeneric.cpp:374] _eventCallback(): Event: 7 - STA_GOT_IP
[D][WiFiGeneric.cpp:419] _eventCallback(): STA IP: 192.168.1.159, MASK: 255.255.255.0, GW: 192.168.1.1
192.168.1.159
Connecting to MQTT...
[I][AsyncMqttClient.cpp:707] connect(): CONNECTING
[I][AsyncTCP.cpp:62] _start_asyncsock_task(): Creating asyncTcpSock task running in core -1 (-1 for any available core)...
[I][AsyncMqttClient.cpp:207] _onConnect(): TCP conn, MQTT CONNECT
[I][AsyncMqttClient.cpp:390] _addFront(): new front #1
[I][AsyncMqttClient.cpp:433] _handleQueue(): snd #1: (tls: 32) 32/32
[I][AsyncMqttClient.cpp:445] _handleQueue(): p #1 rel
[I][AsyncMqttClient.cpp:265] _onAck(): ack 32
[I][AsyncMqttClient.cpp:245] _onDisconnect(): TCP disconn
Disconnected from MQTT.

The ESP32 tries to connect to MQTT continously but for some reason it doesn't succed.

I'm pretty sure there's no problem on the broker side because if I use the pubsubclient I get succesful results:

Attempting to connect to SSID: some SSID
...Connected to some SSID
Attempting MQTT connection...connected
Message arrived [halp] hi

[pedros89]

Hello, I am trying to have my async-mqtt-client library to support TLS.
Thank you for your version that seems promising, I have substituted your modified 4 files in my library plus the build_opt.h in the sketch file.
If I define the macro forESP32and ASYNC_TCP_SSL_ENABLED=1 I get still the error that the classes has no members, I am missing something here.

C:\Users\Pietro\Documents\Arduino\libraries\async-mqtt-client-develop\src\AsyncMqttClient.cpp:153:11: error: 'class AsyncClient' has no member named 'setClientCert'
   _client.setClientCert(cli_cert, cli_cert_len);
           ^~~~~~~~~~~~~
C:\Users\Pietro\Documents\Arduino\libraries\async-mqtt-client-develop\src\AsyncMqttClient.cpp:154:11: error: 'class AsyncClient' has no member named 'setClientKey'
   _client.setClientKey(cli_key, cli_key_len);

I have also tried to modify the library enabling everything by hand but the same error follows.

Sorry but I still don't get where I should define those, at the beginning of which file?
Thank you

[luebbe]

Maybe take a look at https://github.com/bertmelis/espMqttClient, which is currently under development.

[pedros89]

Thank you @luebbe , I see that there is also an example of MQTT TLS for ESP32. If I manage to make that work I will write a guide and repost it because I have seen many posts where people cannot make the TLS work.
Especially all those like me that started their MQTT project from this blog article of Random Nerd Tutorial: Run Your Cloud MQTT Mosquitto Broker and ESP32
I have written so much code with the asyn-mqtt-client library that switching library now would be painful so I need to just activate TLS for asyn-mqtt-client.

[luebbe]

according to @bertmelis, his library aims to be a plug-in replacemant for async-mqtt-client. The next time I change some of my esp code I'll use Bert's library instead.

[pedros89]

Thank you @luebbe! I really recommend to anyone that is trying to implement TLS for MQTT to start from here (especially for ESP32):
https://github.com/bertmelis/espMqttClient/tree/main/examples
I have managed to include MQTT TLS in my project, while doing so I came across some problems. Basically while TLS MQTT was publishing or subscribing data I had other functionalities in my code that stopped working, such as OTA, HTTPS POST requests, webpage. Here you find a guide on how to sort them. Basically you have to disconnect form MQTT while doing certain other secure operations and introduce delays and make sure you wait after the set up() and launch of MQTT service to do OTA or HTTPS POSTs