From 0d4fada7759c16d4b03ef17e82541da4926a4ca0 Mon Sep 17 00:00:00 2001 From: Sebastian Kunert Date: Tue, 8 Nov 2022 15:59:54 +0100 Subject: [PATCH 1/7] Add simple example on how to spin up network. --- README.md | 132 +++++++++++++++----------- zombienet_examples/small_network.toml | 25 +++++ 2 files changed, 100 insertions(+), 57 deletions(-) create mode 100644 zombienet_examples/small_network.toml diff --git a/README.md b/README.md index 3d6a47fbba6..b8eaffbcf91 100644 --- a/README.md +++ b/README.md @@ -37,10 +37,82 @@ and treat as best. A Polkadot [collator](https://wiki.polkadot.network/docs/en/learn-collator) for the parachain is implemented by the `polkadot-parachain` binary (previously called `polkadot-collator`). -## Installation - +## Installation and Setup Before building Cumulus SDK based nodes / runtimes prepare your environment by following Substrate [installation instructions](https://docs.substrate.io/main-docs/install/). +You can use [zombienet](https://github.com/paritytech/zombienet) for quick setup and experimentation or follow the [manual setup](#manual-setup). + +### Zombienet +We use zombienet to spin up networks for integration tests and local networks. +A simple network specification with two relay chain nodes and one collator is located at `zombienet_examples/simple_network.toml`. + +#### Which provider should I use? +Zombienet offers multiple providers to run networks. For development on the node, the `native` provider is a good pick but requires a `polkadot` and `polkadot-parachain` binary. +If you just want to spin up a network for experimentation, you can use the `podman` provider. + +#### How to run +To run the example network, use the following commands: + +```bash +# Podman provider +zombienet --provider podman spawn ./zombienet_examples/simple_network.toml + +# Native provider, assumes polkadot and polkadot-parachains binary in $PATH +zombienet --provider native spawn ./zombienet_examples/simple_network.toml +``` + +### Manual Setup +#### Launch the Relay Chain + +```bash +# Clone +git clone https://github.com/paritytech/polkadot +cd polkadot + +# Compile Polkadot with the real overseer feature +cargo build --release --bin polkadot + +# Generate a raw chain spec +./target/release/polkadot build-spec --chain rococo-local --disable-default-bootnode --raw > rococo-local-cfde.json + +# Alice +./target/release/polkadot --chain rococo-local-cfde.json --alice --tmp + +# Bob (In a separate terminal) +./target/release/polkadot --chain rococo-local-cfde.json --bob --tmp --port 30334 +``` + +#### Launch the Parachain + +```bash +# Clone +git clone https://github.com/paritytech/cumulus +cd cumulus + +# Compile +cargo build --release --bin polkadot-parachain + +# Export genesis state +./target/release/polkadot-parachain export-genesis-state > genesis-state + +# Export genesis wasm +./target/release/polkadot-parachain export-genesis-wasm > genesis-wasm + +# Collator1 +./target/release/polkadot-parachain --collator --alice --force-authoring --tmp --port 40335 --ws-port 9946 -- --execution wasm --chain ../polkadot/rococo-local-cfde.json --port 30335 + +# Collator2 +./target/release/polkadot-parachain --collator --bob --force-authoring --tmp --port 40336 --ws-port 9947 -- --execution wasm --chain ../polkadot/rococo-local-cfde.json --port 30336 + +# Parachain Full Node 1 +./target/release/polkadot-parachain --tmp --port 40337 --ws-port 9948 -- --execution wasm --chain ../polkadot/rococo-local-cfde.json --port 30337 +``` + +#### Register the parachain + +![image](https://user-images.githubusercontent.com/2915325/99548884-1be13580-2987-11eb-9a8b-20be658d34f9.png) + + ## Statemint 🪙 This repository also contains the Statemint runtime (as well as the canary runtime Statemine and the @@ -63,7 +135,7 @@ CHAIN=westmint # or statemine ./target/release/polkadot-parachain --chain $CHAIN ``` -Refer to the [setup instructions below](#local-setup) to run a local network for development. +Refer to the [setup instructions](#local-setup) to run a local network for development. ## Contracts 📝 @@ -117,60 +189,6 @@ The network uses horizontal message passing (HRMP) to enable communication betwe the relay chain and, in turn, between parachains. This means that every message is sent to the relay chain, and from the relay chain to its destination parachain. -### Local Setup - -Launch a local setup including a Relay Chain and a Parachain. - -#### Launch the Relay Chain - -```bash -# Clone -git clone https://github.com/paritytech/polkadot -cd polkadot - -# Compile Polkadot with the real overseer feature -cargo build --release - -# Generate a raw chain spec -./target/release/polkadot build-spec --chain rococo-local --disable-default-bootnode --raw > rococo-local-cfde.json - -# Alice -./target/release/polkadot --chain rococo-local-cfde.json --alice --tmp - -# Bob (In a separate terminal) -./target/release/polkadot --chain rococo-local-cfde.json --bob --tmp --port 30334 -``` - -#### Launch the Parachain - -```bash -# Clone -git clone https://github.com/paritytech/cumulus -cd cumulus - -# Compile -cargo build --release - -# Export genesis state -./target/release/polkadot-parachain export-genesis-state > genesis-state - -# Export genesis wasm -./target/release/polkadot-parachain export-genesis-wasm > genesis-wasm - -# Collator1 -./target/release/polkadot-parachain --collator --alice --force-authoring --tmp --port 40335 --ws-port 9946 -- --execution wasm --chain ../polkadot/rococo-local-cfde.json --port 30335 - -# Collator2 -./target/release/polkadot-parachain --collator --bob --force-authoring --tmp --port 40336 --ws-port 9947 -- --execution wasm --chain ../polkadot/rococo-local-cfde.json --port 30336 - -# Parachain Full Node 1 -./target/release/polkadot-parachain --tmp --port 40337 --ws-port 9948 -- --execution wasm --chain ../polkadot/rococo-local-cfde.json --port 30337 -``` - -#### Register the parachain - -![image](https://user-images.githubusercontent.com/2915325/99548884-1be13580-2987-11eb-9a8b-20be658d34f9.png) - ### Containerize After building `polkadot-parachain` with cargo or with Parity CI image as documented in [this chapter](#build--launch-rococo-collators), diff --git a/zombienet_examples/small_network.toml b/zombienet_examples/small_network.toml new file mode 100644 index 00000000000..5c9c9157e1d --- /dev/null +++ b/zombienet_examples/small_network.toml @@ -0,0 +1,25 @@ +[relaychain] +default_image = "parity/polkadot:latest" +default_command = "polkadot" +chain = "rococo-local" + + [[relaychain.nodes]] + name = "alice" + validator = true + + [[relaychain.nodes]] + name = "bob" + validator = true + +[[parachains]] +id = 2000 +cumulus_based = true +chain = "statemine-local" + + # run charlie as parachain collator + [[parachains.collators]] + name = "charlie" + validator = true + image = "docker.io/parity/polkadot-parachain:latest" + command = "polkadot-parachain" + args = ["--force-authoring"] From 11bdb920bfa1f027538ef265af1ee8faef0954a8 Mon Sep 17 00:00:00 2001 From: Sebastian Kunert Date: Tue, 8 Nov 2022 16:07:57 +0100 Subject: [PATCH 2/7] Update readme --- README.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index b8eaffbcf91..594f4e46ebc 100644 --- a/README.md +++ b/README.md @@ -40,15 +40,18 @@ implemented by the `polkadot-parachain` binary (previously called `polkadot-coll ## Installation and Setup Before building Cumulus SDK based nodes / runtimes prepare your environment by following Substrate [installation instructions](https://docs.substrate.io/main-docs/install/). -You can use [zombienet](https://github.com/paritytech/zombienet) for quick setup and experimentation or follow the [manual setup](#manual-setup). +To launch a local network, you can use [zombienet](https://github.com/paritytech/zombienet) for quick setup and experimentation or follow the [manual setup](#manual-setup). ### Zombienet -We use zombienet to spin up networks for integration tests and local networks. +We use zombienet to spin up networks for integration tests and local networks. Follow [these installation steps](https://github.com/paritytech/zombienet#requirements-by-provider) to set it up on your machine. A simple network specification with two relay chain nodes and one collator is located at `zombienet_examples/simple_network.toml`. + #### Which provider should I use? -Zombienet offers multiple providers to run networks. For development on the node, the `native` provider is a good pick but requires a `polkadot` and `polkadot-parachain` binary. -If you just want to spin up a network for experimentation, you can use the `podman` provider. +Zombienet offers multiple providers to run networks. Choose the one that best fits your needs: +- **Podman:** Choose this if you want to spin up a network quick and easy. +- **Native:** Choose this if you want to develop and deploy your changes. Requires compilation of the binaries. +- **Kubernetes:** Choose this for advanced use-cases or running on cloud-infrastructure. #### How to run To run the example network, use the following commands: From 5d2998fb9c6b67b4858ee026e084ba1df746f5ec Mon Sep 17 00:00:00 2001 From: Sebastian Kunert Date: Tue, 8 Nov 2022 16:17:11 +0100 Subject: [PATCH 3/7] Remove unnecessary prefix --- zombienet_examples/small_network.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/zombienet_examples/small_network.toml b/zombienet_examples/small_network.toml index 5c9c9157e1d..ac5ee11e5bb 100644 --- a/zombienet_examples/small_network.toml +++ b/zombienet_examples/small_network.toml @@ -20,6 +20,6 @@ chain = "statemine-local" [[parachains.collators]] name = "charlie" validator = true - image = "docker.io/parity/polkadot-parachain:latest" + image = "parity/polkadot-parachain:latest" command = "polkadot-parachain" args = ["--force-authoring"] From be9c4be9a2432f1344d9601867b71341aaa89ed8 Mon Sep 17 00:00:00 2001 From: Sebastian Kunert Date: Tue, 8 Nov 2022 17:47:46 +0100 Subject: [PATCH 4/7] Improve folder structure --- scripts/ci/gitlab/pipeline/zombienet.yml | 2 +- {zombienet_examples => zombienet/examples}/small_network.toml | 0 ...0001-sync_blocks_from_tip_without_connected_collator.feature | 0 .../0001-sync_blocks_from_tip_without_connected_collator.toml | 0 {zombienet_tests => zombienet/tests}/0002-pov_recovery.feature | 0 {zombienet_tests => zombienet/tests}/0002-pov_recovery.toml | 0 .../tests}/0003-full_node_catching_up.feature | 0 .../tests}/0003-full_node_catching_up.toml | 0 .../tests}/0004-runtime_upgrade.feature | 0 {zombienet_tests => zombienet/tests}/0004-runtime_upgrade.toml | 0 .../tests}/0005-migrate_solo_to_para.feature | 0 .../tests}/0005-migrate_solo_to_para.toml | 0 .../tests}/0006-rpc_collator_builds_blocks.feature | 0 .../tests}/0006-rpc_collator_builds_blocks.toml | 0 {zombienet_tests => zombienet/tests}/migrate_solo_to_para.js | 0 {zombienet_tests => zombienet/tests}/register-para.js | 0 {zombienet_tests => zombienet/tests}/runtime_upgrade.js | 0 17 files changed, 1 insertion(+), 1 deletion(-) rename {zombienet_examples => zombienet/examples}/small_network.toml (100%) rename {zombienet_tests => zombienet/tests}/0001-sync_blocks_from_tip_without_connected_collator.feature (100%) rename {zombienet_tests => zombienet/tests}/0001-sync_blocks_from_tip_without_connected_collator.toml (100%) rename {zombienet_tests => zombienet/tests}/0002-pov_recovery.feature (100%) rename {zombienet_tests => zombienet/tests}/0002-pov_recovery.toml (100%) rename {zombienet_tests => zombienet/tests}/0003-full_node_catching_up.feature (100%) rename {zombienet_tests => zombienet/tests}/0003-full_node_catching_up.toml (100%) rename {zombienet_tests => zombienet/tests}/0004-runtime_upgrade.feature (100%) rename {zombienet_tests => zombienet/tests}/0004-runtime_upgrade.toml (100%) rename {zombienet_tests => zombienet/tests}/0005-migrate_solo_to_para.feature (100%) rename {zombienet_tests => zombienet/tests}/0005-migrate_solo_to_para.toml (100%) rename {zombienet_tests => zombienet/tests}/0006-rpc_collator_builds_blocks.feature (100%) rename {zombienet_tests => zombienet/tests}/0006-rpc_collator_builds_blocks.toml (100%) rename {zombienet_tests => zombienet/tests}/migrate_solo_to_para.js (100%) rename {zombienet_tests => zombienet/tests}/register-para.js (100%) rename {zombienet_tests => zombienet/tests}/runtime_upgrade.js (100%) diff --git a/scripts/ci/gitlab/pipeline/zombienet.yml b/scripts/ci/gitlab/pipeline/zombienet.yml index 4682af16ee4..2238d6117fd 100644 --- a/scripts/ci/gitlab/pipeline/zombienet.yml +++ b/scripts/ci/gitlab/pipeline/zombienet.yml @@ -21,7 +21,7 @@ artifacts: true variables: POLKADOT_IMAGE: "docker.io/paritypr/polkadot-debug:master" - GH_DIR: "https://github.com/paritytech/cumulus/tree/${CI_COMMIT_SHORT_SHA}/zombienet_tests" + GH_DIR: "https://github.com/paritytech/cumulus/tree/${CI_COMMIT_SHORT_SHA}/zombienet/tests" COL_IMAGE: "docker.io/paritypr/test-parachain:${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHORT_SHA}" allow_failure: true retry: 2 diff --git a/zombienet_examples/small_network.toml b/zombienet/examples/small_network.toml similarity index 100% rename from zombienet_examples/small_network.toml rename to zombienet/examples/small_network.toml diff --git a/zombienet_tests/0001-sync_blocks_from_tip_without_connected_collator.feature b/zombienet/tests/0001-sync_blocks_from_tip_without_connected_collator.feature similarity index 100% rename from zombienet_tests/0001-sync_blocks_from_tip_without_connected_collator.feature rename to zombienet/tests/0001-sync_blocks_from_tip_without_connected_collator.feature diff --git a/zombienet_tests/0001-sync_blocks_from_tip_without_connected_collator.toml b/zombienet/tests/0001-sync_blocks_from_tip_without_connected_collator.toml similarity index 100% rename from zombienet_tests/0001-sync_blocks_from_tip_without_connected_collator.toml rename to zombienet/tests/0001-sync_blocks_from_tip_without_connected_collator.toml diff --git a/zombienet_tests/0002-pov_recovery.feature b/zombienet/tests/0002-pov_recovery.feature similarity index 100% rename from zombienet_tests/0002-pov_recovery.feature rename to zombienet/tests/0002-pov_recovery.feature diff --git a/zombienet_tests/0002-pov_recovery.toml b/zombienet/tests/0002-pov_recovery.toml similarity index 100% rename from zombienet_tests/0002-pov_recovery.toml rename to zombienet/tests/0002-pov_recovery.toml diff --git a/zombienet_tests/0003-full_node_catching_up.feature b/zombienet/tests/0003-full_node_catching_up.feature similarity index 100% rename from zombienet_tests/0003-full_node_catching_up.feature rename to zombienet/tests/0003-full_node_catching_up.feature diff --git a/zombienet_tests/0003-full_node_catching_up.toml b/zombienet/tests/0003-full_node_catching_up.toml similarity index 100% rename from zombienet_tests/0003-full_node_catching_up.toml rename to zombienet/tests/0003-full_node_catching_up.toml diff --git a/zombienet_tests/0004-runtime_upgrade.feature b/zombienet/tests/0004-runtime_upgrade.feature similarity index 100% rename from zombienet_tests/0004-runtime_upgrade.feature rename to zombienet/tests/0004-runtime_upgrade.feature diff --git a/zombienet_tests/0004-runtime_upgrade.toml b/zombienet/tests/0004-runtime_upgrade.toml similarity index 100% rename from zombienet_tests/0004-runtime_upgrade.toml rename to zombienet/tests/0004-runtime_upgrade.toml diff --git a/zombienet_tests/0005-migrate_solo_to_para.feature b/zombienet/tests/0005-migrate_solo_to_para.feature similarity index 100% rename from zombienet_tests/0005-migrate_solo_to_para.feature rename to zombienet/tests/0005-migrate_solo_to_para.feature diff --git a/zombienet_tests/0005-migrate_solo_to_para.toml b/zombienet/tests/0005-migrate_solo_to_para.toml similarity index 100% rename from zombienet_tests/0005-migrate_solo_to_para.toml rename to zombienet/tests/0005-migrate_solo_to_para.toml diff --git a/zombienet_tests/0006-rpc_collator_builds_blocks.feature b/zombienet/tests/0006-rpc_collator_builds_blocks.feature similarity index 100% rename from zombienet_tests/0006-rpc_collator_builds_blocks.feature rename to zombienet/tests/0006-rpc_collator_builds_blocks.feature diff --git a/zombienet_tests/0006-rpc_collator_builds_blocks.toml b/zombienet/tests/0006-rpc_collator_builds_blocks.toml similarity index 100% rename from zombienet_tests/0006-rpc_collator_builds_blocks.toml rename to zombienet/tests/0006-rpc_collator_builds_blocks.toml diff --git a/zombienet_tests/migrate_solo_to_para.js b/zombienet/tests/migrate_solo_to_para.js similarity index 100% rename from zombienet_tests/migrate_solo_to_para.js rename to zombienet/tests/migrate_solo_to_para.js diff --git a/zombienet_tests/register-para.js b/zombienet/tests/register-para.js similarity index 100% rename from zombienet_tests/register-para.js rename to zombienet/tests/register-para.js diff --git a/zombienet_tests/runtime_upgrade.js b/zombienet/tests/runtime_upgrade.js similarity index 100% rename from zombienet_tests/runtime_upgrade.js rename to zombienet/tests/runtime_upgrade.js From 80bd8258bd167f15e717f5af38d4b6bf311c35bb Mon Sep 17 00:00:00 2001 From: Sebastian Kunert Date: Tue, 8 Nov 2022 17:53:41 +0100 Subject: [PATCH 5/7] Add link to file --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 594f4e46ebc..245bcc21c3e 100644 --- a/README.md +++ b/README.md @@ -44,7 +44,7 @@ To launch a local network, you can use [zombienet](https://github.com/paritytech ### Zombienet We use zombienet to spin up networks for integration tests and local networks. Follow [these installation steps](https://github.com/paritytech/zombienet#requirements-by-provider) to set it up on your machine. -A simple network specification with two relay chain nodes and one collator is located at `zombienet_examples/simple_network.toml`. +A simple network specification with two relay chain nodes and one collator is located at [zombienet_examples/simple_network.toml](zombienet_examples/simple_network.toml). #### Which provider should I use? From 633ef7434eba75285b8e4378496ada93c86ad104 Mon Sep 17 00:00:00 2001 From: Sebastian Kunert Date: Tue, 8 Nov 2022 18:00:03 +0100 Subject: [PATCH 6/7] Fix paths in readme --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 245bcc21c3e..3d192fb551d 100644 --- a/README.md +++ b/README.md @@ -44,7 +44,7 @@ To launch a local network, you can use [zombienet](https://github.com/paritytech ### Zombienet We use zombienet to spin up networks for integration tests and local networks. Follow [these installation steps](https://github.com/paritytech/zombienet#requirements-by-provider) to set it up on your machine. -A simple network specification with two relay chain nodes and one collator is located at [zombienet_examples/simple_network.toml](zombienet_examples/simple_network.toml). +A simple network specification with two relay chain nodes and one collator is located at [zombienet/examples/small_network.toml](zombienet/examples/small_network.toml). #### Which provider should I use? @@ -58,10 +58,10 @@ To run the example network, use the following commands: ```bash # Podman provider -zombienet --provider podman spawn ./zombienet_examples/simple_network.toml +zombienet --provider podman spawn ./zombienet/examples/small_network.toml # Native provider, assumes polkadot and polkadot-parachains binary in $PATH -zombienet --provider native spawn ./zombienet_examples/simple_network.toml +zombienet --provider native spawn ./zombienet/examples/small_network.toml ``` ### Manual Setup From 7e35ee5ef7d02103da0f1ee2a00dc93debe43ba3 Mon Sep 17 00:00:00 2001 From: Sebastian Kunert Date: Tue, 8 Nov 2022 18:59:44 +0100 Subject: [PATCH 7/7] Update README.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Bastian Köcher --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 3d192fb551d..e4e1e86ec42 100644 --- a/README.md +++ b/README.md @@ -43,7 +43,7 @@ Before building Cumulus SDK based nodes / runtimes prepare your environment by f To launch a local network, you can use [zombienet](https://github.com/paritytech/zombienet) for quick setup and experimentation or follow the [manual setup](#manual-setup). ### Zombienet -We use zombienet to spin up networks for integration tests and local networks. Follow [these installation steps](https://github.com/paritytech/zombienet#requirements-by-provider) to set it up on your machine. +We use Zombienet to spin up networks for integration tests and local networks. Follow [these installation steps](https://github.com/paritytech/zombienet#requirements-by-provider) to set it up on your machine. A simple network specification with two relay chain nodes and one collator is located at [zombienet/examples/small_network.toml](zombienet/examples/small_network.toml).