Hang/stall of (new) message receipts ("Updating..." forever, caused by FCM Push?)

[gerryfrancis]

• Android version:
  Android 15.

• Device:
  Google Pixel 8.

• Delta Chat version:
  1.50.5.

• Expected behavior:
  (New) messages are received as they are looked up/fetched from the server.

• Actual behavior:
  (New) messages are not received, the app hangs/stalls at "Updating...". When this happens, the app must be terminated and relaunched in order to receive messages again.

• Steps to reproduce the problem:
  Unknown.

• Screenshots:
  N/A.

• Logs:
  deltachat-log-20250114-160450_2.txt

• Remarks:
  When this happens, a line like this has been recorded in the log:
  01-14 16:04:22.995 29692 29835 🟠 chat.delta: AIBinder_linkToDeath is being called with a non-null cookie and no onUnlink callback set. This might not be intended. AIBinder_DeathRecipient_setOnUnlinked should be called first.

[link2xt]

Could be a duplicate of #6477

Regarding background fetch in response to FCM push, in the log there was only one background fetch for 4 accounts and it finished:

238:01-14 16:04:23.019 29692 29757 🔵 DeltaChat: [accId=1] src/context.rs:552: background_fetch started fetching [email protected].
239:01-14 16:04:23.019 29692 29757 🔵 DeltaChat: [accId=2] src/context.rs:552: background_fetch started fetching [email protected].
240:01-14 16:04:23.020 29692 29757 🔵 DeltaChat: [accId=3] src/context.rs:552: background_fetch started fetching [email protected].
241:01-14 16:04:23.020 29692 29757 🔵 DeltaChat: [accId=4] src/context.rs:552: background_fetch started fetching [email protected].
333:01-14 16:04:23.951 29692 29757 🔵 DeltaChat: [accId=3] src/context.rs:590: background_fetch done for [email protected] took 932.135458ms.
337:01-14 16:04:24.053 29692 29757 🔵 DeltaChat: [accId=2] src/context.rs:590: background_fetch done for [email protected] took 1.034360393s.
366:01-14 16:04:25.476 29692 29757 🔵 DeltaChat: [accId=1] src/context.rs:590: background_fetch done for [email protected] took 2.458331992s.
482:01-14 16:04:29.846 29692 29757 🔵 DeltaChat: [accId=4] src/context.rs:590: background_fetch done for [email protected] took 6.828036543s.

[adbenitez]

I am having this problem recently as well, the "updating.." thing is there for a long time, this is right now:

[as400l]

I notice similar things happen on my wife phone while mine is almost always just fine.
The only difference between two of us is that she is using home wifi network and I almost never use it. My phone is always on mobile network.

I suspect it is a problem with how imap code works. Example:

1. Start Deltachat with wifi on and it connects to the imap server over wifi.
2. Put the phone away for 10-15 minutes (with wifi still on) and no messages arrive. Probably wifi card goes to sleep.
3. Both app and imap server think that the connection is still alive while in reality it isn't.
4. Wake up the phone - still nothing, no notification, no message until you start Deltachat. It somehow refreshes network connections and message arrives. Alternatively (don't start Deltachat), wake up the phone, switch off wifi and message arrives. Probably due to Deltachat refreshing connections because of change in network configuration.

Interestingly this is not happening with FairEmail which also relies of IMAP IDLE. So maybe you guys could take a peek at what Marcel is doing in his imap code ?

This is just a theory but in my case I can reproduce it every time I try.

EDIT:

This is a quote from FairEmail FAQ - maybe it will be useful for you:

Some providers send every two minutes something like 'Still here' resulting in network traffic and your device to wake up and causing unnecessary extra battery usage. You can inspect the Log via the main navigation menu to check if your provider is doing this. If your provider is using Dovecot as IMAP server, you could ask your provider to change the [imap_idle_notify_interval] setting to a higher value or better yet, to disable this. If your provider is not able or willing to change/disable this, you should consider switching to periodically instead of continuous synchronization. You can change this in the receive settings tab page.

In case of my imap server imap_idle_notify_interval is set to 29 minutes.

[gerryfrancis]

@as400l Thank you for your detailed report. Years ago I discovered exactly the same issue in regard to WLAN connections and Delta Chat on my former phone (Fairphone FP2 and Android 11). My current configuration uses mobile data only, so the problem no longer occurs to me anymore. But I noticed an important thing: In Android developer functions (tap build number seven times to enable them), there is a function named "Mobile data always active" enabled by default. After I disabled this function, the issue you described had been gone. So I believe that Delta Chat does not "listen" to such OS network updates, but holds on to the (inactive) mobile data gateway idling in the background when this function is enabled. So, if you were able to test whether or not this function makes a difference, we would be glad to read of your results. Many thanks in advance! :)

[as400l]

@gerryfrancis - thanks for confirmation. I will try to find out tomorrow. Good thing is that I'm not crazy :)

In the meantime I contacted Marcel of FairEmail. This is his reply to my questions about IMAP IDLE:

The app will automatically refresh the connection itself by sending a NOOP every 15 minutes to the email server. This time is configurable in the advanced account settings under "keep alive". In theory, the interval be longer, but please be aware that Android alarms are not always exact on Android.

[as400l]

@gerryfrancis I tested today.
Interestingly in case of Samsung this option was off by default. So I switched it on. Then disabled wifi and restarted Deltachat. At the end enabled wifi again. No change. When phone goes to sleep messages do not arrive. Even though there is notification "Background connection enabled".

BTW - on GrapheneOS the option is enabled by default.

[gerryfrancis]

At the end enabled wifi again. No change. When phone goes to sleep messages do not arrive.

@as400l I experienced the issue while "Mobile data always active" had been on, but instead messages were received in time while connected to WLAN and having that function off. Maybe you need to switch off the setting, then reboot the phone, then launch Delta Chat and see whether it makes a difference or not.

[as400l]

@gerryfrancis I tested this on that mentioned earlier Samsung device. It was actually off by default.
I am wondering now whether changing "imap_idle_notify_interval" would change something. I have it set to 29 minutes so mobile devices do not wake up too often. Maybe lowering that to say 20 minutes would change something ?

[link2xt]

Delta Chat does not use NOOP, but tries to refresh IDLE every 5 minutes by terminating it and starting again. This does not work as expected on Android though because of the bug rust-lang/rust#71860 and the timeout possibly taking much longer than 5 minutes of wall clock time if the app is in background.

I don't think the issue discussed in the last few posts (NAT forgetting about the connection because of no keepalives or too long keepalive interval) is related to the original issue. In case of NAT forgetting the connection Delta Chat recovers after 1 minute when you get it out of the background.

[as400l]

Maybe implementing unifiedpush would be a viable resolution to these notification problems ?
I mean if fcm push is there then why not also unifiedpush ?

[gerryfrancis]

@as400l AFAIK, the server must support push notifications, too, and there are just a few email providers who do that. I think it would not solve the core issue for users who cannot use FCM Push/UnifiedPush.

[as400l]

@gerryfrancis I think I found solution. I lowered "imap_idle_notify_interval" to 2 minutes and it works on both phones which were sleeping entire night.
If I'm not mistaken it's Dovecots default.

And I changed multiple power saving settings on Samsung. What a crap btw.

[gerryfrancis]

@as400l Thank you for your feedback. Please describe which power saving settings you changed, so i can check them on my Google Pixel 8 and do some tests here. Thank you in advance.

[as400l]

@gerryfrancis Just so we understand each other correctly. I have Google Pixel 7. My wife phone is Samsung. And I changed her phone settings according to ----> https://dontkillmyapp.com/samsung (for Android 13). This howto is useless for Pixel phones as Samsung adds multiple battery saving features not present in AOSP Android (my Pixel is running GrapheneOS). But anyways until I changed Dovecot setting (imap_idle_notify_interval) it wasn't working properly on both phones. Even though power saving settings were changed.

What I have changed on my Pixel:

1. Settings --> Apps --> DeltaChat --> Permissions --> Disabled "Manage app if unused".
2. Settings --> Apps --> DeltaChat --> Mobile Data Usage --> Enable "Background data" and "Unrestricted mobile data usage".
3. Settings --> Apps --> DeltaChat --> App battery usage --> tick "Allow background usage" and "Unrestricted".

[gerryfrancis]

@as400l Thank you for clarifying.

Settings --> Apps --> DeltaChat --> Permissions --> Disabled "Manage app if unused".

This is no power saving setting and has no immediate effect at the time of enabling. It just archives apps that have not been used for a certain period of time (days/weeks/months). Eventually granted rights are withdrawn, too, as soon as an app has been archived.

Settings --> Apps --> DeltaChat --> Mobile Data Usage --> Enable "Background data" and "Unrestricted mobile data usage".

Aren't we talking about WLAN connections?

Settings --> Apps --> DeltaChat --> App battery usage --> tick "Allow background usage" and "Unrestricted".

This is mandatory to keep background connections alive in Delta Chat.

[as400l]

@gerryfrancis I basically included everything I personally changed.

[WofWca]

I also just experienced something similar on 1.52.1 F-Droid on my testrun.org account (not Chatmail). Android 15, with no Google Play services. I have enabled "Force Background Connection".

No messages get received, but sending works. It constantly shows "Updating...". "Connectivity" says "Connected". It stayed like this for a few hours (from 2 to 16).
Other accounts work fine.
I did not find anything interesting in the logs, though I'm not sure what to look for.

[gerryfrancis]

1.52.1 F-Droid

@WofWca Releases from the F-Droid store do not support FCM Push, so your issue might be a different one, but I am not absolutely sure.

[gerryfrancis]

Moved to the core repository by appointment of @adbenitez . :)

[link2xt]

Let's close as duplicate of #6477 until someone produces more logs with a new version so we can figure out where IMAP loop got stuck. Most likely it's the same issue.

IDLE not resetting every 5 minutes because of rust-lang/rust#71860 and tokio-rs/tokio#3185 is another issue and offtopic for this one. Interesting that it can be worked around by reducing imap_idle_notify_interval.

[link2xt]

I also created #6533 for keepalive frequency issue.