ci: add nix CI

[rvolosatovs]

• Build, test and upload as artifacts:
  • Statically-linked (towards Musl) x86_64 and aarch64 Linux wash binaries
  • Darwin x86_64, aarch64 and "universal" binaries
  • Windows x86_64
  • OCI images containing the above
• Lint the nix flake
• Run clippy, nextest and fmt via nix

Note how fast the nix CI jobs have finished (most under 1 minute), that is due to the caching from previous runs

Related Issues

Blocked by #408
This includes #584 and will prevent issues like #564 from occurring in the future

Release Information

Consumer Impact

Testing

Built on platform(s)

• x86_64-linux
• aarch64-linux
• x86_64-darwin
• aarch64-darwin
• x86_64-windows

Tested on platform(s)

• x86_64-linux
• aarch64-linux
• x86_64-darwin
• aarch64-darwin
• x86_64-windows

Unit Test(s)

Acceptance or Integration

Manual Verification

Open a workflow run summary, e.g. https://github.com/wasmCloud/wash/actions/runs/4303159397

Pull appropriate artifact, e.g. https://github.com/wasmCloud/wash/suites/11277959533/artifacts/578333825 for x86_64 Linux OCI

Then:

$ unzip wash-x86_64-unknown-linux-musl-oci.zip 
Archive:  wash-x86_64-unknown-linux-musl-oci.zip
  inflating: wash-x86_64-unknown-linux-musl-oci  
$ podman load < wash-x86_64-unknown-linux-musl-oci 
Getting image source signatures
Copying blob 4246f70d4a5d done  
Copying config 64d39d429d done  
Writing manifest to image destination
Storing signatures
Loaded image: localhost/wash:0.16.0
$ podman run --rm localhost/wash:0.16.0           

_________________________________________________________________________________
                               _____ _                 _    _____ _          _ _
                              / ____| |               | |  / ____| |        | | |
 __      ____ _ ___ _ __ ___ | |    | | ___  _   _  __| | | (___ | |__   ___| | |
 \ \ /\ / / _` / __| '_ ` _ \| |    | |/ _ \| | | |/ _` |  \___ \| '_ \ / _ \ | |
  \ V  V / (_| \__ \ | | | | | |____| | (_) | |_| | (_| |  ____) | | | |  __/ | |
   \_/\_/ \__,_|___/_| |_| |_|\_____|_|\___/ \__,_|\__,_| |_____/|_| |_|\___|_|_|
_________________________________________________________________________________

A single CLI to handle all of your wasmCloud tooling needs


Usage: wash [OPTIONS] <COMMAND>

Commands:
  app       Manage declarative applications and deployments (wadm) (experimental)
  build     Build (and sign) a wasmCloud actor, provider, or interface
  call      Invoke a wasmCloud actor
  claims    Generate and manage JWTs for wasmCloud actors
  ctl       Interact with a wasmCloud control interface
  ctx       Manage wasmCloud host configuration contexts
  down      Tear down a wasmCloud environment launched with wash up
  drain     Manage contents of local wasmCloud caches
  gen       Generate code from smithy IDL files
  keys      Utilities for generating and managing keys
  lint      Perform lint checks on smithy models
  new       Create a new project from template
  par       Create, inspect, and modify capability provider archive files
  reg       Interact with OCI compliant registries
  up        Bootstrap a wasmCloud environment
  validate  Perform validation checks on smithy models
  help      Print this message or the help of the given subcommand(s)

Options:
  -o, --output <OUTPUT>  Specify output format (text or json) [default: text]
  -h, --help             Print help information
  -V, --version          Print version information

Alternatively, e.g. aarch64 Linux musl binary (https://github.com/wasmCloud/wash/suites/11277959533/artifacts/578333818), run via QEMU:

$ unzip wash-aarch64-unknown-linux-musl.zip         
Archive:  wash-aarch64-unknown-linux-musl.zip
  inflating: wash-aarch64-unknown-linux-musl
$ chmod +x wash-aarch64-unknown-linux-musl
$ file ./wash-aarch64-unknown-linux-musl
./wash-aarch64-unknown-linux-musl: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, not stripped
$ ./wash-aarch64-unknown-linux-musl help
_________________________________________________________________________________
                               _____ _                 _    _____ _          _ _
                              / ____| |               | |  / ____| |        | | |
 __      ____ _ ___ _ __ ___ | |    | | ___  _   _  __| | | (___ | |__   ___| | |
 \ \ /\ / / _` / __| '_ ` _ \| |    | |/ _ \| | | |/ _` |  \___ \| '_ \ / _ \ | |
  \ V  V / (_| \__ \ | | | | | |____| | (_) | |_| | (_| |  ____) | | | |  __/ | |
   \_/\_/ \__,_|___/_| |_| |_|\_____|_|\___/ \__,_|\__,_| |_____/|_| |_|\___|_|_|
_________________________________________________________________________________

A single CLI to handle all of your wasmCloud tooling needs


Usage: wash-aarch64-unknown-linux-musl [OPTIONS] <COMMAND>

Commands:
  app       Manage declarative applications and deployments (wadm) (experimental)
  build     Build (and sign) a wasmCloud actor, provider, or interface
  call      Invoke a wasmCloud actor
  claims    Generate and manage JWTs for wasmCloud actors
  ctl       Interact with a wasmCloud control interface
  ctx       Manage wasmCloud host configuration contexts
  down      Tear down a wasmCloud environment launched with wash up
  drain     Manage contents of local wasmCloud caches
  gen       Generate code from smithy IDL files
  keys      Utilities for generating and managing keys
  lint      Perform lint checks on smithy models
  new       Create a new project from template
  par       Create, inspect, and modify capability provider archive files
  reg       Interact with OCI compliant registries
  up        Bootstrap a wasmCloud environment
  validate  Perform validation checks on smithy models
  help      Print this message or the help of the given subcommand(s)

Options:
  -o, --output <OUTPUT>  Specify output format (text or json) [default: text]
  -h, --help             Print help information
  -V, --version          Print version information

[brooksmtownsend]

I've got a couple questions that I left comments for, but my biggest question is, do we need this AND our other CI?

It seems like this PR quickly builds binaries for a few different OS/arch pairs, are they only useful for users of nix? If this is a solution that we can use to build and distribute wash, I would prefer to supplant our current release pipelines with something like this, or keep this scoped to just building.

All in all I'm happy to bring this in now, I just want to understand our future CI

[rvolosatovs]

I've got a couple questions that I left comments for, but my biggest question is, do we need this AND our other CI?
It seems like this PR quickly builds binaries for a few different OS/arch pairs, are they only useful for users of nix? If this is a solution that we can use to build and distribute wash, I would prefer to supplant our current release pipelines with something like this, or keep this scoped to just building.

All in all I'm happy to bring this in now, I just want to understand our future CI

This PR builds OCI (docker) images for Linux and Mac, both x86_64 and aarch64, as well as the binaries themselves. All of this is reproducible and portable - it runs on any OS. Please see the PR description for steps on how to try it out.

If you're on a Mac you should be able to download the architecture-appropriate or universal binary archive, unzip and execute wash. You can also try to download an OCI (docker) image, it's also documented in the description.

The obvious benefit I see with this PR is that now anyone can press on nix CI run on any commit and pull whichever Mac or Linux wash binaries or OCI images they need

I'm happy get rid of usage of things like https://github.com/cross-rs/cross, but I am not a maintainer of this repo, so I suppose it's not up to me to decide - I do not know what's the procedure here.

Alternatively, we could enable https://garnix.io/, which is a GitHub app and does not require any config, it parses the flake.nix and builds everything it can build e.g. https://github.com/rvolosatovs/nixify/pull/81/checks?check_run_id=11654903212

There would not be any GitHub artifacts uploaded in this case

[ghost]

Seconding what @brooksmtownsend said. I don't think we should be running the same CI twice, once "vanilla" and once in nix. If we want to adopt nix I think we'd need to get rid of the existing CI in favor of it.

So then it comes down to the pros/cons of running CI in nix vs in the vanilla way, and how much of the CI process is it replacing. Would we remove the release jobs we currently have and replace everything with nix? From your comment it sounds like that is an advantage of the tool, but i'm not familiar enough to know what the benefits are

[brooksmtownsend]

@rvolosatovs After the quick thread in slack I like the idea of getting this out and watching it for a little while, and eventually trying to replace our CI with these runs. Only request is to restrict the pull_request and push actions to the main branch given our possible rate limiting that we've run into

[brooksmtownsend]

Hey @rvolosatovs this seems super close! Is there anything we can do to push it into main?

[connorsmith256]

Looking forward to comparing this to the existing action setup 👍