Downloaded packages unexpectedly removed by a parallel transaction

[m-blaha]

If two transactions run in parallel - such as when DNF Automatic is configured - the RPM packages downloaded by one transaction may be removed by the other.

1. The first transaction is in the download phase:

❯ sudo dnf upgrade
Package                                            Arch   Version                   Reposit      Size
Upgrading:
 SDL2_image                                        x86_64 2.8.6-1.fc40              updates 217.9 KiB
   replacing SDL2_image                            x86_64 2.8.2-4.fc40              <unknow 213.3 KiB
 cmake                                             x86_64 3.30.8-1.fc40             updates  32.4 MiB
   replacing cmake                                 x86_64 3.30.7-1.fc40             updates  32.4 MiB
...
...
Transaction Summary:
 Upgrading:         54 packages
 Replacing:         54 packages

Total size of inbound packages is 186 MiB. Need to download 186 MiB.
After this operation, 283 KiB extra will be used (install 619 MiB, remove 619 MiB).
Is this ok [y/N]: y
[ 1/54] SDL2_image-0:2.8.6-1.fc40.x86_64         100% | 266.3 KiB/s | 109.4 KiB |  00m00s
[ 2/54] cmake-filesystem-0:3.30.8-1.fc40.x86_64  100% | 305.4 KiB/s |  16.8 KiB |  00m00s
[ 3/54] cmake-rpm-macros-0:3.30.8-1.fc40.noarch  100% | 159.4 KiB/s |  16.3 KiB |  00m00s
...

2. A second transaction is started in another terminal window and successfully completes before the first transaction finishes downloading.

❯ sudo dnf install acpi
Package                     Arch     Version                     Repository          Size
Installing:
 acpi                       x86_64   1.7-23.fc40                 fedora          47.5 KiB

Transaction Summary:
 Installing:         1 package

Total size of inbound packages is 27 KiB. Need to download 0 B.
After this operation, 47 KiB extra will be used (install 47 KiB, remove 0 B).
Is this ok [y/N]: y
[1/1] acpi-0:1.7-23.fc40.x86_64                  100% |   0.0   B/s |   0.0   B |  00m00s
-----------------------------------------------------------------------------------------
[1/1] Total                                      100% |   0.0   B/s |   0.0   B |  00m00s
Running transaction
[1/3] Verify package files                       100% | 500.0   B/s |   1.0   B |  00m00s
[2/3] Prepare transaction                        100% |   2.0   B/s |   1.0   B |  00m00s
[3/3] Installing acpi-0:1.7-23.fc40.x86_64       100% | 115.5 KiB/s |  49.0 KiB |  00m00s
Complete!

3. The first transaction completes its download and starts the RPM transaction, but it immediately fails because the packages downloaded before the second transaction finished were removed by the second transaction.

...
[54/54] vim-common-2:9.1.1169-1.fc40.x86_64      100% | 684.4 KiB/s |   8.2 MiB |  00m12s
-----------------------------------------------------------------------------------------
[54/54] Total                                    100% |   5.1 MiB/s | 185.8 MiB |  00m36s
Running transaction
Transaction failed: Signature verification failed.
OpenPGP check for package "SDL2_image-2.8.6-1.fc40.x86_64" (/var/cache/libdnf5/updates-02a32a5ce99e20ab/packages/SDL2_image-2.8.6-1.fc40.x86_64.rpm) from repo "updates" has failed: Problem occurred when opening the package.

The reason is that the libdnf5::repo::TempFilesMemory class stores the list of temporary files (i.e. downloaded RPM packages) in a toml file, which is shared between all running dnf5 instances.

Edit: this has another side-effect. The rest of the packages downloaded by the first transaction are never cleaned by following successful transaction. They will stay on the system until dnf clean is executed.

It's a regression compared to dnf4, which locks the transaction before the download of RPM packages begins.