Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): add e2e tests for cloud distros #1259

Open
wants to merge 81 commits into
base: main
Choose a base branch
from
Open

Conversation

noahpb
Copy link
Contributor

@noahpb noahpb commented Feb 5, 2025

Description

Adapts our nightly pipelines (RKE2, EKS and AKS) to run our e2e tests. This PR generally takes our e2e tests that we currently use for k3d and expands them to run against additional k8s distributions and architectures. Notable changes are:

  • Adds public load balancers for RKE2 to make Ingress Gateways accessible from github workflows
  • Adds a fix for metrics scraping of kube-apiserver components for STIG'd RKE2
  • Adds coredns setup tasks for each distro to allow connectivity to *.uds.dev from apps within the cluster
  • Adds a new task to ./tasks/test.yaml called uds-core-non-k3d for triggering e2e tests against various distros. This task includes an optional input called EXCLUDED_PACKAGES that allows callers to skip uneccessary tests, such as metrics-server for some distros.
  • Allows configuration of the architecture for the ./src/test/package
  • Adds a new task to ./tasks/utils.yaml called setup-hosts that configures the local /etc/hosts file to point to *.uds.dev and *.admin.uds.dev domains that are hosted on the cluster that e2e tests are running against.
  • Adds timeouts to the curl commands in ./test/jest/network.spec.ts to produce more meaningful error messages when timeouts are occurring during connectivity tests (before these would run indefinitely and the jest timeout would ultimately be reached).
  • In ./test/jest/network.spec.ts, changes explicit expect().toBe("503") responses to expect().not.toBe("200") to account for different behavior of cluster CNIs when rejecting traffic due to NetworkPolicy deny rules.

Related Issue

Fixes #729

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Other (security config, docs update, etc)

Steps to Validate

  • If this PR introduces new functionality to UDS Core or addresses a bug, please document the steps to test the changes.

Checklist before merging

@noahpb noahpb requested a review from a team as a code owner February 5, 2025 19:58
@noahpb noahpb marked this pull request as draft February 5, 2025 19:58
@noahpb noahpb marked this pull request as ready for review February 5, 2025 20:05
@noahpb noahpb marked this pull request as draft February 6, 2025 19:34
@noahpb noahpb changed the title chore: add e2e tests for cloud distros chore(ci): add e2e tests for cloud distros Mar 14, 2025
@noahpb noahpb marked this pull request as ready for review March 14, 2025 18:00
- name: Test UDS Core
run: uds run -f tasks/test.yaml uds-core-non-k3d --set EXCLUDED_PACKAGES="metrics-server"
continue-on-error: true

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noting the continue-on-error: true here. EKS VPC CNI permits outbound traffic to the internet for the network tests, despite not having a NetworkPolicy to explicitly allow this. Ignoring failures for now until we decide how to proceed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Run validation tasks against nightly deployments
2 participants