deblobbify, replacing the system folder with several other components

[robertkirkman]

Solution to:

• termux-docker images should be properly labeled with their native arch #35
• The whole system directory may be removed from the repo... #60
• Hostnames fail to resolve #63
• Missing TERM environment variable #65
• Failed to start dnsmasq, host name resolution may fail [ macos arm64 ] #68

Summary of changes:

• Deletes the entire system folder and replaces it with a script generate.sh that can regenerate the entire system folder.
• The root filesystem, including system folder, is regenerated using purely a Termux bootstrap .zip, and .deb files that can be built using the termux-packages build and CD system, then retrieved from the termux.dev package host.
• Adds an environment variable $TERM that is set to a value of "xterm" for the clear command to have convenient compatibility with most terminal emulators, and this has also been tested with Kitty and TTY.
• Simplifies the instances of the string "com.termux" into a single instance for slightly more convenient use with custom bootstraps.
• Fixes the existing docker.io/termux/termux-docker:aarch64 image not working on aarch64 devices that happen to have an esoteric "Cortex-A34"-type or similar processor due to the elimination of this 32-bit ARM binary blob, which Cortex-A34-like processors (including those in Apple Silicon devices) do not have hardware support for executing, unlike "Cortex-A72"-like processors.
• Replaces Busybox with a real build of Toybox based on the source code of AOSP 9.0.0 that has been patched to meet the requirements of termux-docker, including providing su to log in as the system user within entrypoint.sh.
• Uses mksh and bash alone to implement sh, instead of Busybox's sh.
• Uses a build of iputils based on the source code of AOSP 9.0.0 to implement /system/bin/ping and /system/bin/ping6.
• Uses the preexisting termux-packages root/dnsmasq to implement dnsmasq, which works after some adjustments to its invocations.
• Adds libandroid-stub, which is described in multiple places as intended specifically for use with termux-docker.
  • I have confirmed that it indeed makes ffmpeg work inside termux-docker, fixing this error: linker: CANNOT LINK EXECUTABLE "ffmpeg": library "libandroid.so" not found
  • in its build.sh
  • xtkoba believed that libandroid-stub should be preinstalled in termux-docker
  • I think the negative reason why libandroid-stub should not be installed for users of normal devices is because it can somehow conflict with their existing /system/lib64/libandroid.so file in some situations. termux-docker does not have /system/lib64/libandroid.so, so it makes sense that libandroid-stub probably has only beneficial effects on termux-docker and no negative effects.
• Deletes /system/etc/ld.config.28.txt without replacing it with anything, since it does not appear to be necessary for anything in any test so far, but it could be easily re-added if it turns out that there is something that needs it.
  • I think that ld.config.28.txt is probably only necessary on ROMs that have a /vendor folder and stuff inside ("VNDK"), termux-docker does not have a /vendor folder or support anything related to that so, for now it seems unnecessary
• Adds a third method of working around the personality() system call permissions issue, a custom build of Docker, which is not required on all ARM devices, but which might be required sometimes, and is required on my armv7l device that the --privileged and --security-opt arguments both failed to work on.
• Replaces the use of QEMU in GitHub Actions with the new official public GitHub ARM runner.
• Enables updating the termux packages within the container before publishing the ARM images.
• Changes latest tag to x86_64 instead of i686.
  • requested by Ted and Biswa96
• Refactors entrypoint_root.sh and entrypoint.sh to have more similar guard clause structure to each other, for readability.

Tested both locally and in GitHub Actions:

• aarch64: working
• x86_64: working
• arm: working, but in some (not all) cases, the custom build of Docker with modified personality() system call permissions is required. It is unknown how to predict whether a device will require that or not, but it is suspected it might have something to do with the host kernel, which is a factor whose properties it is understood tend to vary dramatically between different ARM devices.
• i686: working

The "temporary" things like the temporary-package-storage folder and the blobs inside are only necessary for testing while this is not merged done:

• bionic-host: rename to aosp-libs, bump to 9.0.0-r76, add aosp-utils subpackage, revbump revdeps termux-packages#22906

The large amount of changes to the internal components of termux-docker could introduce unpredictable bugs or unexpected behavior that is not present with the current official termux-docker. For that reason, I hope to learn all of the current intended use cases that termux-docker is officially supposed to be used for, so that I can test my implementation with them and add additional software or fixes if needed.

Known Unique Limitations:

• For some reason, the -c argument of toybox-su is currently non-functional. For the current featureset of termux-docker, both before and after this PR, that does not seem to be a major priority because termux-docker currently only supports the root user via the entrypoint_root.sh and not via su or tsu from within the default entrypoint.sh, i.e. most termux-packages root-packages cannot currently be conveniently tested using termux-docker anyway. If "proper simulation of a rooted device", so to speak, is desirable in the future, to facilitate more convenient testing of root-packages, if ever necessary, that could be investigated and handled by a different issue+PR at a later date.

[twaik]

modifying the termux-packages bionic-host package is not acceptable

Nobody said it is not acceptable. I think nobody tried it.
I created the package as a solution for running android executables during regular building with toolchain to avoid using hostbuild step, but for some reason qemu does not want to run fine with aarch64 executables on amd64 host, at least in some cases.
Also it fits fine for using in termux-docker but it was not modified to use bionic-host package.

[robertkirkman]

for some reason qemu does not want to run fine with aarch64 executables on amd64 host, at least in some cases

I am not sure exactly what exact scenario you are talking about here, or what error it is you were seeing at the time, if you mean a scenario with a build.sh involving only the regular cross-compiling toolchain, then what I am about to say is probably not relevant,

But just in case knowing this helps you, there is a way to emulate an ARM processor on an x86 device that is very high in emulation accuracy and behaves almost identically to how real ARM devices behave.

When carefully configured, I have never seen qemu-system-aarch64 fail to execute ARM software on an x86 processor, as long as that same ARM software also works on real ARM devices. If I ever see frustrating errors while trying to use user-mode emulation of any kind, qemu-system-aarch64 is my go-to solution for accuracy to the exact behavior of real ARM devices, when I am in a place with only x86 devices and accessing real ARM devices is inconvenient.

Since there are very few good guides on the subject, and almost all other guides skip mentioning lots of necessary steps, leaving everything unexplained and causing most people to feel like qemu-system-aarch64 is much too tedious to consider using on a regular basis, I created this guide that covers every detail of the setup process in a very straightforward, completely explained, and future-proofed way based on the latest UEFI technology, which is also distro-agnostic to both the host and the guest, making it very clear what changes when the guest and host are mixed and matched.

[twaik]

I will just link this.
termux/termux-packages#18517 (comment)
I know it is not a simple scenario, but GIR-generated binary used for generating some XMLs for some reason hangs and CI is terminated with timeout after 6 hours so it is a no go for us. It was the very first and the last attempt (my attempt) to use cross-compiled binaries for building (to avoid using host_build step).

[robertkirkman]

GIR-generated binary used for generating some XMLs for some reason hangs and CI is terminated with timeout after 6 hours so it is a no go for us

I experimented with some local tests based on qemu-system technology (as opposed to qemu-user technology),

and while it can inherently avoid several of the issues described in that thread, such as the GIR hanging problem, the problems with any other binaries hanging, and the pid_max problem, due to the higher emulation accuracy and environment micromanagement of qemu-system compared to qemu-user,

very unfortunately, it cannot overcome the slowness problem that licy183 explained. Additionally, the higher emulation accuracy of qemu-system comes with a cost that the slowness is even more extreme, causing the build time for multiple things I need to build to go far over the 6 hours limit (my builds were successful locally, but took 16 hours). Even though that technically works for me locally, that makes it not viable for anything in GitHub Actions without also using the 6 hour limit bypass incremental build workaround technique every time.

I think the only efficient possible future plan on this topic, is to make use of termux-docker, or licy183's tur-avd, or another minimal android container, combined both with the new GitHub ARM runners and with the termux-packages clang package, to build some packages with cross compilation disabled and emulation also disabled.

I am already testing the GitHub ARM runner in this PR and it works for example here,
but since TUR has already been building packages in this same codepath using self-hosted ARM runners instead of cloud ARM runners, it is possible that maybe licy183 is already about to or in the process of adding the cloud ARM runners to TUR, which could serve as an example of how to proceed.
They had tested a PR using it a few months ago, but I did not see any other tests of it yet.
If they do not end up adding it soon, I could attempt my own way of adding support for it to the termux-packages repo, but first I need to finish the changes to termux-docker.

I cannot complete the changes to termux-docker without first removing blockers, like fixing the builds of pypy and pypy3 so that the dependency can be merged. Because of that, I think I am currently forced to first use the only workaround possible to force them to build in a single PR with the updated aosp-libs package, the pid_max change, then I could attempt to migrate the ARM builds of pypy and pypy3 to an actual ARM runner workflow some time afterward, after creating it. The pid_max change would remain required for running any 32-bit termux-docker-based workflows anyway, so it could be moved to the termux-docker-based workflow instead at the time that is finished. done

I have tested locally that pypy would build and work using termux-packages clang inside a complete termux-docker ARM system, it is just that there is no workflow set up for that yet. I would prefer waiting to try introducing termux-docker to such a workflow until after it has had the blobs removed.

[robertkirkman]

I have managed to encounter a test case where:

• normal devices are negatively affected by having libandroid-stub installed
• under equivalent conditions within termux-docker, libandroid-stub instead only helps, and the error from the normal device cannot be reproduced

That seems to support the assumption that the long-preexisting libandroid-stub package is probably entirely beneficial inside termux-docker with no anticipated downsides, so it makes sense to preinstall it.

How to reproduce the negative effect of libandroid-stub on a normal device

1. Use pkg install libandroid-stub
2. Use pkg install mpv
3. mpv

• Result

CANNOT LINK EXECUTABLE "mpv": cannot locate symbol "AImageReader_newWithUsage" referenced by "/system/lib64/libhwui.so"...

How to reproduce the above conditions inside termux-docker and observe no error

1. Use pkg install libandroid-stub (if not already installed)
2. Use pkg install mpv
3. Work around [possibly not a bug] "mpv": library "libOpenSLES.so" not found #70 by installing unofficial package libopensles-standalone as explained in that issue
4. mpv

• Result

mpv 0.39.0 Copyright © 2000-2024 mpv/MPlayer/mplayer2 projects
 built on Feb 26 2025 00:33:35
...

[twaik]

About libandroid-stub. Currently we package libandroid.so from NDK, but can we make our own version of libandroid which will dlopen+dlsym regular libandroid.so in the case if it exists and call it's methods and return some error code in the case if symbols are not present or libandroid is not available?

[twaik]

I mean in this case we will be able to ship libandroid-stub regardless of having android or docker and it should work fine on devices. But I am not sure about this.

[robertkirkman]

I am not sure, because I think I do not have one of the devices that (i assume) has a /system/lib64/libandroid.so, but where the libandroid-stub package is still needed, so I cannot test on that device.

If I can find a device or contact someone who owns such a device that contains a /system/lib64/libandroid.so but which does not work with the ffmpeg package unless libandroid-stub is installed, then maybe I could try to make code until I make it work on termux-docker, and also that device, and also normal devices all at the same time from one package.

• Here was one of them [Bug]: OpenCV is not working termux-packages#20334 (comment)

Samsung Galaxy M52