nerdctl: Docker-compatible CLI for containerd

nerdctl is a Docker-compatible CLI for containerd.

• Examples
  • Basic usage
  • Debugging Kubernetes
  • Rootless mode
• Install
• Motivation
• Features present in nerdctl but not present in Docker
• Similar tools
• Developer guide
  • Compiling nerdctl from source
  • Test suite
    • Running test suite against nerdctl
    • Running test suite against Docker
  • Contributing to nerdctl
• Command reference
  • Run & Exec
    • 🐳 nerdctl run
    • 🐳 nerdctl exec
  • Container management
    • 🐳 nerdctl ps
    • 🐳 nerdctl inspect
    • 🐳 nerdctl logs
    • 🐳 nerdctl port
    • 🐳 nerdctl rm
    • 🐳 nerdctl stop
    • 🐳 nerdctl kill
    • 🐳 nerdctl pause
    • 🐳 nerdctl unpause
  • Build
    • 🐳 nerdctl build
    • 🐳 nerdctl commit
  • Image management
    • 🐳 nerdctl images
    • 🐳 nerdctl pull
    • 🐳 nerdctl push
    • 🐳 nerdctl load
    • 🐳 nerdctl save
    • 🐳 nerdctl tag
    • 🐳 nerdctl rmi
    • 🤓 nerdctl image convert
  • Registry
    • 🐳 nerdctl login
    • 🐳 nerdctl logout
  • Network management
    • 🐳 nerdctl network create
    • 🐳 nerdctl network ls
    • 🐳 nerdctl network inspect
    • 🐳 nerdctl network rm
  • Volume management
    • 🐳 nerdctl volume create
    • 🐳 nerdctl volume ls
    • 🐳 nerdctl volume inspect
    • 🐳 nerdctl volume rm
  • System
    • 🐳 nerdctl events
    • 🐳 nerdctl info
    • 🐳 nerdctl version
  • Global flags
  • Unimplemented Docker commands
• Additional documents

Examples

Basic usage

To run a container with the default CNI network (10.4.0.0/24):

# nerdctl run -it --rm alpine

To build an image using BuildKit:

# nerdctl build -t foo .
# nerdctl run -it --rm foo

Debugging Kubernetes

To list Kubernetes containers:

# nerdctl --namespace k8s.io ps -a

Rootless mode

To launch rootless containerd:

$ containerd-rootless-setuptool.sh install

To run a container with rootless containerd:

$ nerdctl run -d -p 8080:80 --name nginx nginx:alpine

See ./docs/rootless.md.

Install

Binaries are available for amd64, arm64, and arm-v7: https://github.com/AkihiroSuda/nerdctl/releases

In addition to containerd, the following components should be installed (optional):

• CNI plugins: for using nerdctl run.
• CNI isolation plugin: for isolating bridge networks (nerdctl network create)
• BuildKit: for using nerdctl build. BuildKit daemon (buildkitd) needs to be running.
• RootlessKit and slirp4netns: for Rootless mode
  • RootlessKit needs to be v0.10.0 or later
  • slirp4netns needs toe be v0.4.0 or later

To run nerdctl inside Docker:

docker build -t nerdctl .
docker run -it --rm --privileged nerdctl

Motivation

The goal of nerdctl is to facilitate experimenting the cutting-edge features of containerd that are not present in Docker.

Such features includes, but not limited to, lazy-pulling and encryption of images.

Note that competing with Docker is not the goal of nerdctl. Those cutting-edge features are expected to be eventually available in Docker as well.

Also, nerdctl might be potentially useful for debugging Kubernetes clusters, but it is not the primary goal.

Features present in nerdctl but not present in Docker

• Namespacing: nerdctl --namespace=<NS> ps . (NOTE: All Kubernetes containers are in the k8s.io containerd namespace regardless to Kubernetes namespaces)
• Lazy-pulling using Stargz Snapshotter: nerdctl --snapshotter=stargz run .
• Exporting Docker/OCI dual-format archives: nerdctl save .
• Importing OCI archives as well as Docker archives: nerdctl load .
• Specifying a non-image rootfs: nerdctl run -it --rootfs <ROOTFS> /bin/sh . The CLI syntax conforms to Podman convention.
• Inspecting raw OCI config: nerdctl container inspect --mode=native .

Similar tools

• ctr: incompatible with Docker CLI, and not friendly to users. Notably, ctr lacks the equivalents of the following Docker CLI commands:

  • docker run -p <PORT>
  • docker run --restart=always --net=bridge
  • docker pull with ~/.docker/config.json and credential helper binaries such as docker-credential-ecr-login
  • docker logs

• crictl: incompatible with Docker CLI, not friendly to users, and does not support non-CRI features

• k3c v0.2 (abandoned): needs an extra daemon, and does not support non-CRI features

• Rancher Kim (nee k3c v0.3): needs Kubernetes, and only focuses on image management commands such as kim build and kim push

• PouchContainer (abandoned?): needs an extra daemon

Developer guide

Compiling nerdctl from source

Run make && sudo make install.

Using go get github.com/AkihiroSuda/nerdctl is possible, but unrecommended because it does not fill version strings printed in nerdctl version

Test suite

Running test suite against nerdctl

Run go test -exec sudo -v ./... after make && sudo make install.

For testing rootless mode, -exec sudo is not needed.

To run tests in a container:

docker build -t test --target test .
docker run -t --rm --privileged test

Running test suite against Docker

Run go test -exec sudo -test.target=docker . to ensure that the test suite is compatible with Docker.

Contributing to nerdctl

Lots of commands and flags are currently missing. Pull requests are highly welcome.

Please certify your Developer Certificate of Origin (DCO), by signing off your commit with git commit -s and with your real name.

---

Command reference

🐳 = Docker compatible

🤓 = nerdctl specific

Unlisted docker CLI flags are unimplemented yet in nerdctl CLI. It does not necessarily mean that the corresponding features are missing in containerd.

Run & Exec

🐳 nerdctl run

Run a command in a new container.

Basic flags:

• 🐳 -i, --interactive: Keep STDIN open even if not attached"
• 🐳 -t, --tty: Allocate a pseudo-TTY
  • ⚠️ WIP: currently -t requires -i, and conflicts with -d
• 🐳 -d, --detach: Run container in background and print container ID
• 🐳 --restart=(no|always): Restart policy to apply when a container exits
  • Default: "no"
  • ⚠️ No support for on-failure and unless-stopped
• 🐳 --rm: Automatically remove the container when it exits
• 🐳 --pull=(always|missing|never): Pull image before running
  • Default: "missing"

Network flags:

• 🐳 --network=(bridge|host|none): Connect a container to a network
  • Default: "bridge"
• 🐳 -p, --publish: Publish a container's port(s) to the host
• 🐳 --dns: Set custom DNS servers
• 🐳 -h, --hostname: Container host name

Cgroup flags:

• 🐳 --cpus: Number of CPUs
• 🐳 --memory: Memory limit
• 🐳 --pids-limit: Tune container pids limit
• 🐳 --cgroupns=(host|private): Cgroup namespace to use
  • Default: "private" on cgroup v2 hosts, "host" on cgroup v1 hosts

User flags:

• 🐳 -u, --user: Username or UID (format: <name|uid>[:<group|gid>])

Security flags:

• 🐳 --security-opt seccomp=<PROFILE_JSON_FILE>: specify custom seccomp profile
• 🐳 --security-opt apparmor=<PROFILE>: specify custom AppArmor profile
• 🐳 --security-opt no-new-privileges: disallow privilege escalation, e.g., setuid and file capabilities
• 🐳 --cap-add=<CAP>: Add Linux capabilities
• 🐳 --cap-drop=<CAP>: Drop Linux capabilities
• 🐳 --privileged: Give extended privileges to this container

Runtime flags:

• 🐳 --runtime: Runtime to use for this container, e.g. "crun", or "io.containerd.runsc.v1".

Volume flags:

• 🐳 -v, --volume: Bind mount a volume

Rootfs flags:

• 🐳 --read-only: Mount the container's root filesystem as read only
• 🤓 --rootfs: The first argument is not an image but the rootfs to the exploded container. Corresponds to Podman CLI.

Env flags:

• 🐳 -w, --workdir: Working directory inside the container
• 🐳 -e, --env: Set environment variables

Metadata flags:

• 🐳 --name: Assign a name to the container
• 🐳 -l, --label: Set meta data on a container
• 🐳 --label-file: Read in a line delimited file of labels

🐳 nerdctl exec

Run a command in a running container.

• 🐳 -i, --interactive: Keep STDIN open even if not attached
• 🐳 -t, --tty: Allocate a pseudo-TTY
  • ⚠️ WIP: currently -t requires -i, and conflicts with -d
• 🐳 -d, --detach: Detached mode: run command in the background
• 🐳 -w, --workdir: Working directory inside the container
• 🐳 -e, --env: Set environment variables
• 🐳 --privileged: Give extended privileges to the command

Container management

🐳 nerdctl ps

List containers.

Flags:

• 🐳 -a, --all: Show all containers (default shows just running)
• 🐳 --no-trunc: Don't truncate output
• 🐳 -q, --quiet: Only display container IDs

🐳 nerdctl inspect

Display detailed information on one or more containers.

Flags:

• 🤓 --mode=(dockercompat|native): Inspection mode. "native" produces more information.

🐳 nerdctl logs

Fetch the logs of a container.

⚠️ Currently, only containers created with nerdctl run -d are supported.

🐳 nerdctl port

List port mappings or a specific mapping for the container.

🐳 nerdctl rm

Remove one or more containers.

Flags:

• 🐳 -f: Force the removal of a running|paused|unknown container (uses SIGKILL)

🐳 nerdctl stop

Stop one or more running containers.

🐳 nerdctl kill

Kill one or more running containers.

🐳 nerdctl pause

Pause all processes within one or more containers.

🐳 nerdctl unpause

Unpause all processes within one or more containers.

Build

🐳 nerdctl build

Build an image from a Dockerfile.

ℹ️ Needs buildkitd to be running.

Flags:

• 🤓 --buildkit-host=<BUILDKIT_HOST>: BuildKit address
• 🐳 -t, --tag: Name and optionally a tag in the 'name:tag' format
• 🐳 -f, --file: Name of the Dockerfile
• 🐳 --target: Set the target build stage to build
• 🐳 --build-arg: Set build-time variables
• 🐳 --no-cache: Do not use cache when building the image
• 🐳 --progress=(auto|plain|tty): Set type of progress output (auto, plain, tty). Use plain to show container output
• 🐳 --secret: Secret file to expose to the build: id=mysecret,src=/local/secret
• 🐳 --ssh: SSH agent socket or keys to expose to the build (format: default|<id>[=<socket>|<key>[,<key>]])

🐳 nerdctl commit

Create a new image from a container's changes

Flags:

• 🐳 -a, --author: Author (e.g., "nerdctl contributor [email protected]")
• 🐳 -m, --message: Commit message

Image management

🐳 nerdctl images

List images

Flags:

• 🐳 -q, --quiet: Only show numeric IDs
• 🐳 --no-trunc: Don't truncate output

🐳 nerdctl pull

Pull an image from a registry.

🐳 nerdctl push

Pull an image from a registry.

🐳 nerdctl load

Load an image from a tar archive or STDIN.

🤓 Supports both Docker Image Spec v1.2 and OCI Image Spec v1.0.

Flags:

• 🐳 -i, --input: Read from tar archive file, instead of STDIN

🐳 nerdctl save

Save one or more images to a tar archive (streamed to STDOUT by default)

🤓 The archive implements both Docker Image Spec v1.2 and OCI Image Spec v1.0.

Flags:

• 🐳 -o, --output: Write to a file, instead of STDOUT

🐳 nerdctl tag

Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE.

🐳 nerdctl rmi

Remove one or more images

🤓 nerdctl image convert

Convert an image format.

e.g., nerdctl image convert --estargz --oci example.com/foo:orig example.com/foo:esgz

Flags:

• --estargz : convert legacy tar(.gz) layers to eStargz for lazy pulling. Should be used in conjunction with '--oci'
• --estargz-record-in=<FILE> : read ctr-remote optimize --record-out=<FILE> record file. ⚠️ This flag is experimental and subject to change.
• --estargz-compression-level=<LEVEL>: eStargz compression level (default: 9)
• --estargz-chunk-size=<SIZE> : eStargz chunk size
• --uncompress : convert tar.gz layers to uncompressed tar layers
• --oci : convert Docker media types to OCI media types
• --platform=<PLATFORM> : convert content for a specific platform
• --all-platforms : convert content for all platforms (default: false)

Registry

🐳 nerdctl login

Log in to a Docker registry.

Flags:

• 🐳 -u, --username: Username
• 🐳 -p, --password: Password
• 🐳 --password-stdin: Take the password from stdin

🐳 nerdctl logout

Log out from a Docker registry

Network management

🐳 nerdctl network create

Create a network

ℹ️ To isolate CNI bridge, CNI isolation plugin needs to be installed.

Flags:

• 🐳 --subnet: Subnet in CIDR format that represents a network segment, e.g. "10.5.0.0/16"

🐳 nerdctl network ls

List networks

🐳 nerdctl network inspect

Display detailed information on one or more networks

⚠️ The output format is not compatible with Docker.

🐳 nerdctl network rm

Remove one or more networks

Volume management

🐳 nerdctl volume create

Create a volume

🐳 nerdctl volume ls

List volumes

• 🐳 -q, --quiet: Only display volume names

🐳 nerdctl volume inspect

Display detailed information on one or more volumes

🐳 nerdctl volume rm

Remove one or more volumes

System

🐳 nerdctl events

Get real time events from the server.

⚠️ The output format is not compatible with Docker.

🐳 nerdctl info

Display system-wide information

🐳 nerdctl version

Show the nerdctl version information

Global flags

• 🤓 -a, --address: containerd address, optionally with "unix://" prefix
• 🐳 -H, --host: Docker-compatible alias for -a, --address
• 🤓 -n, --namespace: containerd namespace
• 🤓 --snapshotter: containerd snapshotter
• 🤓 --cni-path: CNI binary path (default: /opt/cni/bin) [$CNI_PATH]
• 🤓 --cni-netconfpath: CNI netconf path (default: /etc/cni/net.d) [$NETCONFPATH]
• 🤓 --data-root: nerdctl data root, e.g. "/var/lib/nerdctl"
• 🤓 --cgroup-manager=(cgroupfs|systemd): cgroup manager

Unimplemented Docker commands

Container management:

• docker attach

• docker cp

• docker diff

• docker rename

• docker start

• docker wait

• docker container prune

• docker checkpoint *

Stats:

• docker stats
• docker top

Image:

• docker export and docker import

• docker history

• docker trust

• docker image prune

• docker manifest *

Network management:

• docker network connect
• docker network disconnect
• docker network prune

Registry:

• docker search

Others:

• docker context
• Swarm commands are unimplemented and will not be implemented: docker swarm|node|service|config|secret|stack *
• Plugin commands are unimplemented and will not be implemented: docker plugin *

---

Additional documents

• ./docs/dir.md: Directory layout (/var/lib/nerdctl)
• ./docs/registry.md: Registry authentication (~/.docker/config.json)
• ./docs/rootless.md: Rootless mode
• ./docs/stargz.md: Lazy-pulling using Stargz Snapshotter