Build Rust packages for darwin with Nix

[link2xt]

This is broken due to NixOS/nixpkgs#180771 and waiting for NixOS/nixpkgs#256590

nix build .#deltachat-rpc-server works on main branch since 2c5a0ca, when running on arm64 macOS it builds a working binary.

Cross-compilation does not work, nix build .#deltachat-rpc-server-x86_64-darwin fails and even nix build .#deltachat-rpc-server-aarch64-darwin ("cross-compiling" to self) fails due to infinite recursion.

[link2xt]

Testing nix build .#deltachat-rpc-server-x86_64-darwin .#deltachat-rpc-server-aarch64-darwin with NixOS/nixpkgs#256590 now. This will take a while because it builds LLVM, Apple SDK etc. from source. At least it does not fail immediately due to infinite recursion.

EDIT: After building for a long time it failed with:

% nix build .#deltachat-rpc-server-aarch64-darwin
error: builder for '/nix/store/6fgqaq7bj0vy6m6rqgdhpj7n1qwxn935-compiler-rt-libc-aarch64-darwin-16.0.6.drv' failed with exit code 1;
       last 25 log lines:
       >
       >   It fails with the following output:
       >
       >     Change Dir: '/tmp/nix-build-compiler-rt-libc-aarch64-darwin-16.0.6.drv-0/compiler-rt-src-16.0.6/compiler-rt/build/CMakeFiles/CMakeScratch/TryCompile-J1ep0L'
       >
       >     Run Build Command(s): /nix/store/fxvgs4y0cxxqbyq8gfm9x7vwabn1y3qn-ninja-1.11.1/bin/ninja -v cmTC_6f217
       >     [1/2] /nix/store/xvajlzv6gqg4kba8yfw4ykg87k77qkjk-aarch64-darwin-clang-wrapper-16.0.6/bin/aarch64-darwin-clang++   -arch arm64 -MD -MT CMakeFiles/cmTC_6f217.dir/testCXXCompiler.cxx.o -MF CMakeFiles/cmTC_6f217.dir/testCXXCompiler.cxx.o.d -o CMakeFiles/cmTC_6f217.dir/testCXXCompiler.cxx.o -c /tmp/nix-build-compiler-rt-libc-aarch64-darwin-16.0.6.drv-0/compiler-rt-src-16.0.6/compiler-rt/build/CMakeFiles/CMakeScratch/TryCompile-J1ep0L/testCXXCompiler.cxx
       >     [2/2] : && /nix/store/xvajlzv6gqg4kba8yfw4ykg87k77qkjk-aarch64-darwin-clang-wrapper-16.0.6/bin/aarch64-darwin-clang++ -arch arm64 -Wl,-headerpad_max_install_names  CMakeFiles/cmTC_6f217.dir/testCXXCompiler.cxx.o -o cmTC_6f217   && :
       >     FAILED: cmTC_6f217
       >     : && /nix/store/xvajlzv6gqg4kba8yfw4ykg87k77qkjk-aarch64-darwin-clang-wrapper-16.0.6/bin/aarch64-darwin-clang++ -arch arm64 -Wl,-headerpad_max_install_names  CMakeFiles/cmTC_6f217.dir/testCXXCompiler.cxx.o -o cmTC_6f217   && :
       >     ld: warning: directory not found for option '-L/nix/store/g3v9ngw25b5k8zja1b868dym5j35fayd-clang-16.0.6-lib/aarch64-darwin/lib'
       >     ld: library not found for -lc++
       >     clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
       >     ninja: build stopped: subcommand failed.
       >
       >
       >
       >
       >
       >   CMake will not be able to correctly generate this project.
       > Call Stack (most recent call first):
       >   CMakeLists.txt:10 (project)
       >
       > 
       > -- Configuring incomplete, errors occurred!
       For full logs, run 'nix log /nix/store/6fgqaq7bj0vy6m6rqgdhpj7n1qwxn935-compiler-rt-libc-aarch64-darwin-16.0.6.drv'.
error: 1 dependencies of derivation '/nix/store/8gaif1a2nbva0gxyi0b7pry9wd2fp75i-aarch64-darwin-clang-wrapper-16.0.6.drv' failed to build
error: 1 dependencies of derivation '/nix/store/mbvmpykgyyw4hnh7b2wa4gh0wh0qgqnb-deltachat-1.136.2.drv' failed to build

Similar error for x86_64-darwin:

% nix build .#deltachat-rpc-server-x86_64-darwin 
error: builder for '/nix/store/ph5zbyd4dlff534mgsxrgnyn2nmvczc9-compiler-rt-libc-x86_64-darwin-16.0.6.drv' failed with exit code 1;
       last 25 log lines:
       >
       >   It fails with the following output:
       >
       >     Change Dir: '/tmp/nix-build-compiler-rt-libc-x86_64-darwin-16.0.6.drv-0/compiler-rt-src-16.0.6/compiler-rt/build/CMakeFiles/CMakeScratch/TryCompile-g7jLQ6'
       >
       >     Run Build Command(s): /nix/store/fxvgs4y0cxxqbyq8gfm9x7vwabn1y3qn-ninja-1.11.1/bin/ninja -v cmTC_864b5
       >     [1/2] /nix/store/6s6bw9lk9zjirsgnggl00jgi8s1qy8y9-x86_64-darwin-clang-wrapper-16.0.6/bin/x86_64-darwin-clang++   -arch x86_64 -MD -MT CMakeFiles/cmTC_864b5.dir/testCXXCompiler.cxx.o -MF CMakeFiles/cmTC_864b5.dir/testCXXCompiler.cxx.o.d -o CMakeFiles/cmTC_864b5.dir/testCXXCompiler.cxx.o -c /tmp/nix-build-compiler-rt-libc-x86_64-darwin-16.0.6.drv-0/compiler-rt-src-16.0.6/compiler-rt/build/CMakeFiles/CMakeScratch/TryCompile-g7jLQ6/testCXXCompiler.cxx
       >     [2/2] : && /nix/store/6s6bw9lk9zjirsgnggl00jgi8s1qy8y9-x86_64-darwin-clang-wrapper-16.0.6/bin/x86_64-darwin-clang++ -arch x86_64 -Wl,-headerpad_max_install_names  CMakeFiles/cmTC_864b5.dir/testCXXCompiler.cxx.o -o cmTC_864b5   && :
       >     FAILED: cmTC_864b5
       >     : && /nix/store/6s6bw9lk9zjirsgnggl00jgi8s1qy8y9-x86_64-darwin-clang-wrapper-16.0.6/bin/x86_64-darwin-clang++ -arch x86_64 -Wl,-headerpad_max_install_names  CMakeFiles/cmTC_864b5.dir/testCXXCompiler.cxx.o -o cmTC_864b5   && :
       >     ld: warning: directory not found for option '-L/nix/store/g3v9ngw25b5k8zja1b868dym5j35fayd-clang-16.0.6-lib/x86_64-darwin/lib'
       >     ld: library not found for -lc++
       >     clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
       >     ninja: build stopped: subcommand failed.
       >
       >
       >
       >
       >
       >   CMake will not be able to correctly generate this project.
       > Call Stack (most recent call first):
       >   CMakeLists.txt:10 (project)
       >
       > 
       > -- Configuring incomplete, errors occurred!
       For full logs, run 'nix log /nix/store/ph5zbyd4dlff534mgsxrgnyn2nmvczc9-compiler-rt-libc-x86_64-darwin-16.0.6.drv'.
error: 1 dependencies of derivation '/nix/store/0m7k4hq20j8f25xmpb8lbqbn1633cnfz-x86_64-darwin-clang-wrapper-16.0.6.drv' failed to build
error: 1 dependencies of derivation '/nix/store/dnn5ldj3y63hlsz18snxi2xg81zm2ph2-deltachat-1.136.2.drv' failed to build

Inside of /nix/store/g3v9ngw25b5k8zja1b868dym5j35fayd-clang-16.0.6-lib/ there are lib and libexec directories, but no x86_64-darwin (and x86_64-darwin/lib).
Same for /nix/store/g3v9ngw25b5k8zja1b868dym5j35fayd-clang-16.0.6-lib/.

[reckenrode]

FYI I updated my Darwin cross branch and will be rebasing it regularly against staging. It contains quite a few commits because it’s now based on top of my cctools/ld64 update work. The priority is to get the cctools/ld64 update merged after the release of 24.05 with clang 18 and Darwin cross following.

[link2xt]

@reckenrode I updated the flake, now it builds but fails to link in the end:

% nix build .#deltachat-rpc-server-aarch64-darwin
error: builder for '/nix/store/igxf91j1qn3vkbmxakr68gjna2pfwbi0-deltachat-1.137.4.drv' failed with exit code 101;
       last 25 log lines:
       > Running phase: buildPhase
       > cargo build $cargo_release -j "$NIX_BUILD_CORES" --message-format=$cargo_message_format --package deltachat-rpc-server
       > warning: Both `/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/.cargo-home/config` and `/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/.cargo-home/config.toml` exist. Using `/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/.cargo-home/config`
       >    Compiling deltachat-contact-tools v0.1.0 (/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/deltachat-contact-tools)
       >    Compiling deltachat_derive v2.0.0 (/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/deltachat_derive)
       >    Compiling ratelimit v1.0.0 (/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/deltachat-ratelimit)
       >    Compiling format-flowed v1.0.0 (/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/format-flowed)
       >    Compiling deltachat-time v1.0.0 (/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/deltachat-time)
       >    Compiling deltachat v1.137.4 (/private/tmp/nix-build-deltachat-1.137.4.drv-0/source)
       >    Compiling deltachat-jsonrpc v1.137.4 (/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/deltachat-jsonrpc)
       >    Compiling deltachat-rpc-server v1.137.4 (/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/deltachat-rpc-server)
       > error: could not compile `deltachat-rpc-server` (bin "deltachat-rpc-server") due to 2 previous errors
       > error: linking with `/nix/store/vhwmjv83gva7vzcj6hyi6g2bb2pc4zqp-aarch64-darwin-clang-wrapper-16.0.6/bin/aarch64-darwin-cc` failed: exit status: 1
       >   |
       >   = note: env -u IPHONEOS_DEPLOYMENT_TARGET -u TVOS_DEPLOYMENT_TARGET LC_ALL="C" PATH="/nix/store/97xj69wad38jxmsynil8fc4fh13rrvvr-rust-mixed/lib/rustlib/aarch64-apple-darwin/bin:/nix/store/97xj69wad38jxmsynil8fc4fh13rrvvr-rust-mixed/bin:/nix/store/83gc9s65qmarz1wi448l8hp6wl54rmj1-jq-1.7.1-bin/bin:/nix/store/6is7l57hmhzlia0lg3dgpv3pq6pm9l8s-rsync-3.3.0/bin:/nix/store/q793v9cz622y5mgrccvxpzzmzli6hisb-perl-5.38.2/bin:/nix/store/9vvbpgbfnknxcj3ngmh8gwqdabkz31z8-clang-wrapper-16.0.6/bin:/nix/store/9giqrplp283v1629w856y9b0ikip2k0x-clang-16.0.6/bin:/nix/store/iv441p6h7j97nhmazkq04h0w8jldgc2q-coreutils-9.5/bin:/nix/store/wdvmwq14ph78hd61bhmmbm2lksk6ggal-cctools-binutils-darwin-wrapper-1010.6/bin:/nix/store/s0iki2bk01rs15sn2z6hnvf3jx18y1hv-cctools-binutils-darwin-1010.6/bin:/nix/store/iv441p6h7j97nhmazkq04h0w8jldgc2q-coreutils-9.5/bin:/nix/store/n0qnp1g35z879m0kfrcpqkzwsjffiqy9-findutils-4.9.0/bin:/nix/store/01yrhx9mqrh01fnmpa5s5xk092b763sy-diffutils-3.10/bin:/nix/store/r5g6vlvdpabbd6jhwv9999jrl8pkjpi8-gnused-4.9/bin:/nix/store/h7fdzf4i1b1blv14dr11gw71n0j842j9-gnugrep-3.11/bin:/nix/store/n12x9q6kb2c0rgh3ic4pzqgjbd3j6g7w-gawk-5.2.2/bin:/nix/store/p12h1wg5mlx23p4i6byi3k11j80vh00m-gnutar-1.35/bin:/nix/store/yxd5b2f9r07akn6v5fn2jxlqqq8rc05a-gzip-1.13/bin:/nix/store/9z2hw1ygfgvbwkprhh59j4p8yrnym9db-bzip2-1.0.8-bin/bin:/nix/store/ygh1xbg7v9iih1yr2zzqscly956lvnvc-gnumake-4.4.1/bin:/nix/store/qbabcj3dnaf4nxx2xpcw7khgv1bjcihr-bash-5.2p26/bin:/nix/store/2p8bqhm008kk0farc0wkp340xszfpwdr-patch-2.7.6/bin:/nix/store/lzp9i1v75c9q5sxmrlj3yb4ka2ha7nxl-xz-5.4.6-bin/bin:/nix/store/8a4mrkll478cjl43c8xqxcm509wrjv7s-file-5.45/bin" VSLANG="1033" ZERO_AR_DATE="1" "/nix/store/vhwmjv83gva7vzcj6hyi6g2bb2pc4zqp-aarch64-darwin-clang-wrapper-16.0.6/bin/aarch64-darwin-cc" "-arch" "arm64" "/private/tmp/nix-build-deltachat-1.137.4.drv-0/rustcPV1G2W/symbols.o" "/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/target/aarch64-apple-darwin/release/deps/deltachat_rpc_server-470e434e51f51378.deltachat_rpc_server.8a09ee6852c0364b-cgu.0.rcgu.o" "-L" "/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/target/aarch64-apple-darwin/release/deps" "-L" "/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/target/release/deps" "-L" "/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/target/aarch64-apple-darwin/release/build/libsqlite3-sys-d362104065645d20/out" "-L" "/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/target/aarch64-apple-darwin/release/build/openssl-sys-ea576ce4f34320a8/out/openssl-build/install/lib" "-L" "/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/target/aarch64-apple-darwin/release/build/blake3-3c31d957ff792c16/out" "-L" "/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/target/aarch64-apple-darwin/release/build/ring-bd217a76495b916b/out" "-L" "/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/target/aarch64-apple-darwin/release/build/ring-f7526934905cfedb/out" "-L" "/nix/store/97xj69wad38jxmsynil8fc4fh13rrvvr-rust-mixed/lib/rustlib/aarch64-apple-darwin/lib" "/private/tmp/nix-build-deltachat-1.137.4.drv-0/rustcPV1G2W/libring-db902b431099c2ec.rlib" "/private/tmp/nix-build-deltachat-1.137.4.drv-0/rustcPV1G2W/libring-ccdcfb07ab166ef1.rlib" "/private/tmp/nix-build-deltachat-1.137.4.drv-0/rustcPV1G2W/libblake3-a570354281984162.rlib" "/private/tmp/nix-build-deltachat-1.137.4.drv-0/rustcPV1G2W/liblibsqlite3_sys-832dd90e40e3af63.rlib" "/private/tmp/nix-build-deltachat-1.137.4.drv-0/rustcPV1G2W/libopenssl_sys-18ed37dfbe760d21.rlib" "/nix/store/6554p77gyxrm3knv67p9ivx6l5yvjxyf-rust-std-stable-2024-04-09/lib/rustlib/aarch64-apple-darwin/lib/libcompiler_builtins-6d556b0e8bf6f23e.rlib" "-framework" "Security" "-framework" "Security" "-framework" "SystemConfiguration" "-framework" "CoreFoundation" "-liconv" "-lSystem" "-lc" "-lm" "-L" "/nix/store/97xj69wad38jxmsynil8fc4fh13rrvvr-rust-mixed/lib/rustlib/aarch64-apple-darwin/lib" "-o" "/private/tmp/nix-build-deltachat-1.137.4.drv-0/source/target/aarch64-apple-darwin/release/deps/deltachat_rpc_server-470e434e51f51378" "-Wl,-dead_strip" "-nodefaultlibs"
       >   = note: ld: warning: directory not found for option '-L/nix/store/ivmygw1a5ajs7vbgkjiydcira1gvckx3-clang-16.0.6-lib/aarch64-darwin/lib'
       >           ld: framework not found Security
       >           clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
       >
       >
       >
       > error: aborting due to 1 previous error
       >
       >
       > [naersk] cargo returned with exit code 101, exiting
       For full logs, run 'nix log /nix/store/igxf91j1qn3vkbmxakr68gjna2pfwbi0-deltachat-1.137.4.drv'.

In /nix/store/ivmygw1a5ajs7vbgkjiydcira1gvckx3-clang-16.0.6-lib/aarch64-darwin there is no aarch64-darwin:

% ls /nix/store/ivmygw1a5ajs7vbgkjiydcira1gvckx3-clang-16.0.6-lib
lib     libexec

This -L is not generated by me, so might be a bug in clang package?