Static binary with nix (and docker)

[denibertovic]

I don't really know how this compares to #1 but perhaps @nh2 and @dysinger can weigh in? All I know is that the existing static-base/Dockerfile didn't build for me anymore. Anyway, I've only tested this binary on debian and ubuntu so far. I haven't tested with alpine but I imagine it should work with every distro since it is a true static binary.

I had to bump the LTS to get a newer compiler version because otherwise stack2nix complained. I used a version I have on some of my other projects just because it was convenient but other than that it was arbitrary.

To build it run make static. It will output the path to the binary at the end. Of course this will take a very long time unless you already have the build cache locally from some other projects (like I have)...or a remote builder etc. Then again this should not necessarily be ran locally but in CI (ideally using cachix and other types of caching). I'll see if I can find the time to make a github actions pipeline but I figured that can be a separate PR and doesn't have to wait for this. In any case this is what I used to make the latest release here: https://github.com/fpco/pid1/releases/tag/pid1-0.1.3.0

[snoyberg]

If you want to merge and maintain that, it’s your call. I’m not going to take responsibility for anything nix related.

[psibi]

All I know is that the existing static-base/Dockerfile didn't build for me anymore.

Around 6 months ago, I built a static pid1 using the infrastructure here: https://github.com/fpco/alpine-haskell-stack for one of my hobby projects. Unfortunately I don't seem to have that code lying around, but it shouldn't be hard to create if required.

And I pushed the images here: https://hub.docker.com/r/psibi/pid1/tags

And I just ran that docker image to confirm that it was indeed static:

/bin # /usr/bin/ldd /sbin/pid1
/lib/ld-musl-x86_64.so.1: /sbin/pid1: Not a valid dynamic program

[denibertovic]

If you want to merge and maintain that, it’s your call. I’m not going to take responsibility for anything nix related.

@snoyberg haha yeah I can totally understand that! :D My thinking was that this "app" is mostly done and I don't see it needing to be updated all that often...like say a library released to hackage might. The dependencies are very minimal and unless there's a catastrophic bug in GHC that produces programs that eat your computer I doubt we have to update it at all. And this way everything is "cemented" forever.
I have no idea how easy it is to update stuff and I'm not sure I'd recommended this unless the above wasn't true. I was hoping Niklas might chime in with his thoughts.

@psibi I don't know anything about https://github.com/fpco/alpine-haskell-stack .. this is the first time I'm hearing about it. It's not mentioned anywhere in this repo.
I was talking about using the static image provided in the repo at ./static-base/Dockerfile which does not build for me anymore! This is what I used for the previous 0.1.2.0 release (at least I think I did!). The dockerfile was added ~5 years ago it seems. Looking at this further it seems that ./build-docker.sh uses an image called fpco/docker-static-haskell which was pushed ~5 years ago. I imagine it was built manually from the static-base/Dockerfile in this repo and pushed to dockerhub ... but I can't be sure of that. In any case the dockerfile doesn't build anymore. When I run build-docker.sh (after I commented out the push parts) it seems that it can indeed build the static pid1 binary and it copies it in the build-docker/ folder on the host machine. There's a lack of caching that could speed things up but it does appear to work. Perhaps that's how I used it last time but have forgotten!? It's still a huge problem that the image on docker hub can no longer be rebuilt though. :/

I tried changing build-docker.sh by using fpco/alpine-haskell-stack (that you mention) instead of the old fpco/docker-static-haskell image. Sadly this image is also not built automatically it seems so some tags that the git repo mentions are not available on the docker hub (the tag 8.10.4 for instance). I tried all of them but it seems that 8.6.5 works if I bump the resolver in stack.yaml to "lts-14.27". You can see the changes here: denibertovic@c87a48a
(I have no idea what upx does and how not having it affects the produced binary...I think the resulting binary is just larger without it).

In summary...if https://github.com/fpco/alpine-haskell-stack is maintained we can switch to that by merging the changes linked above instead of the nix stuff in this PR. It's not perfect but if you're more comfortable in debugging that setup instead of nix that's fine by me of course. It could probably also be automated in a github action just fine.

What do you guys think?

[psibi]

(I have no idea what upx does and how not having it affects the produced binary...I think the resulting binary is just larger without it).

I think I had to remove the upx step to make it work as the binary was segfaulting with it. Seems it was removing something extra. (Maybe I'm misremembering things)

In summary...if https://github.com/fpco/alpine-haskell-stack is maintained

I do think it is maintained. I think stack also uses it to build the static binary: https://github.com/commercialhaskell/stack/blob/69f55d401d8ef719d255b7876884899fbfcd546e/stack.yaml#L14

It's not perfect but if you're more comfortable in debugging that setup instead of nix that's fine by me of course.

I'm fine with either of those things. But since you are the going to be maintaining it, I would leave the decision to you. :-)

[nh2]

Hi Deni, it looks good to me.

If you like, you can upgrade the LTS to a newer one, and use a newer static-haskell-nix version that uses GHC 8.10. That newer version needs less stuff built, so compiles should be faster.

I made a PR into your branch here, in case you want to check it out: denibertovic#1

On maintenance:

For a simple program like this, I think any approach will do that produces a static binary that people can use. If Docker currently doens't work and adding a nix build provides an easy solution without having to figure things out, that's good, and in case that in the future a Docker build becomes easier/faster again, that could take over again.

[denibertovic]

@nh2 those were my thoughts as well. That said I thought I could bring back the docker build stuff and update to use the newer maintained images (fpco/alpine-haskell-stack).

I pushed the new changes that should streamline the process a bit. Basically:

• make docker-static (to build the static binary in out/bin using fpco/alpine-haskell-stack) [1]
• make docker-image (to use that binary and make the docker image)
• make docker push (to tag and push to docker hub)

[1] I also made sure to actually use the /home/build directory this time so that global .stack persist.

Now the interesting bits. This worked fine with lts-14.27 and fpco/alpine-haskell-stack:8.6.5.
But since we bumped to stack.yaml to lts-17.5 I had to use fpco/alpine-haskell-stack:8.10.4 and with that I get the following error:

Building library for pid1-0.1.3.0..
[1 of 1] Compiling System.Process.PID1
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/crtbeginT.o: relocation R_X86_64_32 against hidden symbol `__TMC_END__' can not be used when making a shared object
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/crtend.o: relocation R_X86_64_32 against `.ctors' can not be used when making a shared object; recompile with -fPIC
collect2: error: ld returned 1 exit status
`cc' failed in phase `Linker'. (Exit code: 1)

--  While building package pid1-0.1.3.0 (scroll up to its section to see the error) using:
      /home/build/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_3.2.1.0_ghc-8.10.4 --builddir=.stack-work/dist/x86_64-linux/Cabal-3.2.1.0 build lib:pid1 exe:pid1 --ghc-options ""
    Process exited with code: ExitFailure 1
make: *** [Makefile:35: docker-build-static] Error 1

I have no idea why it would work with lts-14.27 and fpco/alpine-haskell-stack:8.6.5 and not with the latest version. :/

@snoyberg @psibi Perhaps some of you know how to fix this?
Regardless, I still think it's valuable to have this method in the repo as well so there's a choice when/if we need it. Especially since it's broken down into steps that we can easily add to CI later (or run locally for that matter).

(I didn't want to touch the .travis.yml file in this PR even though it doesn't work anymore since the project was not moved to the new travis site....but then again it didn't actually use any of the scripts used to release things so....)

[psibi]

@psibi Perhaps some of you know how to fix this?

Sorry about the late reply. Finally at work, I had some requirements for working on pid1. I faced similar issue:

Building library for pid1-0.1.3.0..
[1 of 1] Compiling System.Process.PID1
Warning: -rtsopts and -with-rtsopts have no effect with -shared.
    Call hs_init_ghc() from your main() function to set these options.
/nix/store/gh0fk4qi1vbfbl6fyd1r6g6w3sh63syj-binutils-2.39/bin/ld: /nix/store/s5njr0mfqns5s54vcjvz7dw39d6k7wg9-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-musl/11.3.0/crtbeginT.o: relocation R_X86_64_32 against hidden symbol `__TMC_END__' can not be used when making a shared object
/nix/store/gh0fk4qi1vbfbl6fyd1r6g6w3sh63syj-binutils-2.39/bin/ld: failed to set dynamic section sizes: bad value
collect2: error: ld returned 1 exit status
`cc' failed in phase `Linker'. (Exit code: 1)

I fixed it like this: #12

I have also pushed newer docker image based on ubuntu 22.04 LTS.

[denibertovic]

@psibi @snoyberg Sorry for the late reply but this fell of the radar for me. :/
If there is now a working method for releasing static binaries I'm fine closing this.