Add topology spread constraints test for RayCluster #3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
YoussefEssDS
force-pushed
the
add-topology-spread-constraints-test-v3
branch
from
799e3d1 to
9fbb17a
Compare
added 6 commits
October 24, 2024 16:22
Update the YAML file to not succeed resources available for the test Add script to validate the toplogy spread constraints Add script to validate the toplogy spread constraints Sets minReplicas to replicas to avoid pods killing themselves prematurely Fix formating issue Add more visibility about pending pods Adjust the the expected running pod count to include the head pod Check the hostnames for testing env Add 2 workers to the created k8s cluster Add visibility to pods Add visibility to pods Fix autoscaler sidecar not launching Add more visibility Add more visibility Add cleanup of previous test pods Cleanup the topology validation script Cleanup the topology validation script Move the topology test to avoid breaking the e2e test Cleanup the topology constraint test cluster Fix formatting issue Update helm chart values and template Fix helm chart lint issue Fix formatting issue
YoussefEssDS
force-pushed
the
add-topology-spread-constraints-test-v3
branch
from
ab24d23 to
cfadbaa
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a test to verify the functionality of topology spread constraints in a RayCluster setup. The test ensures that worker pods respect the topology spread constraints defined in the cluster's YAML file, particularly focusing on distributing pods across nodes while adhering to maxSkew and avoiding node overloading.
Thanks to for the help!
The modifications include:
A new YAML file with topology spread constraints applied to the worker groups.
An update to the e2e test setup to deploy the cluster with the new topology spread configuration.
Validation logic to ensure pods are scheduled according to the specified constraints.
Why are these changes needed?
Topology spread constraints are a Kubernetes feature that helps to distribute pods evenly across nodes to improve high availability and fault tolerance. This test is necessary to ensure that RayCluster configurations support these constraints properly, providing users with greater control over pod scheduling. It helps in scenarios where workload distribution across nodes is critical for resource balancing.
Related issue number
This PR addresses the need for testing topology spread constraints in the KubeRay setup, as requested in a related issue ray-project#2273
Checks
I've made sure the tests are passing.
Testing Strategy
Manual tests: Validated topology spread constraint behavior by observing pod distribution in the test cluster + description of pending worker pods.