ESP32-S3 requires clean session=false to receive subscribed messages after deep sleep

[ajay10000]

Hi to all and many thanks to @bertmelis for the terrific MQTT library. I am really enjoying the flexibility it enables, particularly logging while not connected and speedy data transfers between deep sleeps. I have some questions I'm hoping can be answered to help my understanding.

Firstly, as per the title, I have found that a custom ESP32-S3 board requires clean session=false set, to receive QoS1 subscribed messages from a local Mosquitto broker after deep sleep. If clean session is true, the messages are never received after a restart. Does that seem to be correct?

It is normally not a problem to have clean session = false, but for example if I send a reboot message via MQTT, the message is read on every boot with this setting (i.e. never acknowledged), creating a boot loop.

If this issue doesn't seem normal, I can share code, logs, etc.

Cheers - Andrew

[bertmelis]

The logic you describe seems the intended way MQTT works. But the issue you experience might have to do with the "persistent" flag. Could it be that it is set to "true"?

You send "reboot" with persistent flag to true --> broker sends out to the subscribers and stores the message. --> the ESP reboots and subscribes again --> broker sends the reboot instruction message again to all clients it hasn't send the message to. This includes the same physical clients with a new session.

If the message is not persistent, the brokers deletes the message once all subscribed clients have acknowledged to receipt.

Also: be sure to disconnect the client gracefully and not just reboot. When you call disconnect, the client first sends out all acks before disconnecting. You can use the onDisconnect handler.

[ajay10000]

Thank you for the quick response. I was a bit confused about the 'persistent' vs 'cleanSession' terminology. The article (https://www.hivemq.com/blog/mqtt-essentials-part-7-persistent-session-queuing-messages/) suggests that the cleanSession=false flag which I mentioned above creates a persistent session. Is that what you mean, or is there a different flag/process I should be aware of?

I will try checking onDisconnect before sleeping or rebooting as you suggest, thank you.

Is there a way of checking the client MQTT inbox/queue to see if there are any messages partially received and/or acknowledgements pending?

[bertmelis]

the cleanSession=false flag which I mentioned above creates a persistent session.
Correct

But also be aware of retained messages. If you publish a message with the "retained" flag to true, the broker stores the message. Every time a client subscribes to a matching topic, the broker sends that message to that client. Here the session also comes into play. If you reconnect with a clean session, you might resubscribe again and receive the retained message. If you keep the existing session, you reuse the existing subscription and the retained message is not sent.

[ajay10000]

That is true! I don't use the retain flag at all because I found that QoS1 suited my needs the best. The only hitch was the message getting 'stuck' if the client didn't acknowledge before restarting from sleep or cold start.

[bertmelis]

Then, make sure to disconnect gracefully before rebooting.

What happens:

1. the client receives a reboot instruction with QoS1
2. Upon receipt, a puback is queued
3. the onMessage callback is fired.

Now, if you at this moment reboot in the onMessage handler, the outgoing queue isn't processed anymore. Better is to set a flag (let's name it "doReboot") and disconnect the client.

4. In the "disconnecting" state, the client empties the outgoing queue (with full QoS handling) and stops processing incoming publish messages. It is also not possible anymore to publish messages.
5. As soon as the outgoing queue is empty, a disconnect message is sent and the client closes the TCP connection.
6. When the TCP connection is closed, the onDisconnect handler is fired.

You can check for the "doReboot" flag in the onDisconnect handler to check if the disconnection is intended or not. If intended, you can safely reboot.