Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add capability to run RawDeployment E2Es in OpenShift-ci #220

Merged
merged 1 commit into from
Mar 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: inferenceservice-config
namespace: kserve
data:
deploy: |-
{
"defaultDeploymentMode": "RawDeployment"
}
ingress: |-
{
"ingressGateway" : "knative-serving/knative-ingress-gateway",
"ingressService" : "istio-ingressgateway.istio-system.svc.cluster.local",
"localGateway" : "knative-serving/knative-local-gateway",
"localGatewayService" : "knative-local-gateway.istio-system.svc.cluster.local",
"ingressDomain" : "$OPENSHIFT_INGRESS_DOMAIN",
"ingressClassName" : "openshift-default",
"domainTemplate": "{{ .Name }}-{{ .Namespace }}.{{ .IngressDomain }}",
"urlScheme": "http",
"disableIstioVirtualHost": false,
"disableIngressCreation": false
}
2 changes: 0 additions & 2 deletions python/kserve/kserve/models/v1beta1_inference_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -150,8 +150,6 @@ def metadata(self, metadata):
:param metadata: The metadata of this V1beta1InferenceService. # noqa: E501
:type: V1ObjectMeta
"""
if metadata is not None:
metadata.annotations = {"serving.knative.openshift.io/enablePassthrough": "true"}

self._metadata = metadata

Expand Down
1 change: 1 addition & 0 deletions test/e2e/predictor/test_raw_deployment.py
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ def test_raw_deployment_runtime_kserve():

@pytest.mark.grpc
@pytest.mark.raw
@pytest.mark.skip("The custom-model-grpc image fails in OpenShift with a permission denied error")
def test_raw_isvc_with_multiple_container_port():
service_name = "raw-multiport-custom-model"
model_name = "custom-model"
Expand Down
1 change: 1 addition & 0 deletions test/e2e/transformer/test_collocation.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ def test_transformer_collocation():


@pytest.mark.raw
@pytest.mark.skip("The torchserve container fails in OpenShift with permission denied errors")
def test_raw_transformer_collocation():
service_name = "raw-custom-model-collocation"
model_name = "mnist"
Expand Down
1 change: 1 addition & 0 deletions test/e2e/transformer/test_raw_transformer.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@


@pytest.mark.raw
@pytest.mark.skip("The torchserve container fails in OpenShift with permission denied errors")
def test_transformer():
service_name = 'raw-transformer'
predictor = V1beta1PredictorSpec(
Expand Down
41 changes: 33 additions & 8 deletions test/scripts/openshift-ci/run-e2e-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -63,10 +63,12 @@ pushd $PROJECT_ROOT/python/kserve >/dev/null
popd

# Install KServe stack
echo "Installing OSSM"
$MY_PATH/deploy.ossm.sh
echo "Installing Serverless"
$MY_PATH/deploy.serverless.sh
if [ "$1" != "raw" ]; then
echo "Installing OSSM"
$MY_PATH/deploy.ossm.sh
echo "Installing Serverless"
$MY_PATH/deploy.serverless.sh
fi

echo "Installing KServe with Minio"
kustomize build $PROJECT_ROOT/config/overlays/test | \
Expand All @@ -75,10 +77,21 @@ kustomize build $PROJECT_ROOT/config/overlays/test | \
sed "s|kserve/router:latest|${KSERVE_ROUTER_IMAGE}|" | \
sed "s|kserve/kserve-controller:latest|${KSERVE_CONTROLLER_IMAGE}|" | \
oc apply -f -

# Patch the inferenceservice-config ConfigMap, when running RawDeployment tests
if [ "$1" == "raw" ]; then
export OPENSHIFT_INGRESS_DOMAIN=$(oc get ingresses.config cluster -o jsonpath='{.spec.domain}')
oc patch configmap inferenceservice-config -n kserve --patch-file <(cat config/overlays/test/configmap/inferenceservice-openshift-ci-raw.yaml | envsubst)
oc delete pod -n kserve -l control-plane=kserve-controller-manager
fi

# Wait until KServe starts
oc wait --for=condition=ready pod -l control-plane=kserve-controller-manager -n kserve --timeout=300s

echo "Installing odh-model-controller"
oc apply -k $PROJECT_ROOT/test/scripts/openshift-ci
if [ "$1" != "raw" ]; then
echo "Installing odh-model-controller"
oc apply -k $PROJECT_ROOT/test/scripts/openshift-ci
fi

echo "Add testing models to minio storage ..." # Reference: config/overlays/test/minio/minio-init-job.yaml
curl -L https://storage.googleapis.com/kfserving-examples/models/sklearn/1.0/model/model.joblib -o /tmp/sklearn-model.joblib
Expand All @@ -96,7 +109,10 @@ apiVersion: v1
kind: Namespace
metadata:
name: kserve-ci-e2e-test
---
EOF

if [ "$1" != "raw" ]; then
cat <<EOF | oc apply -f -
apiVersion: maistra.io/v1
kind: ServiceMeshMember
metadata:
Expand All @@ -107,6 +123,7 @@ spec:
namespace: istio-system
name: basic
EOF
fi

oc apply -f $PROJECT_ROOT/config/overlays/test/minio/minio-user-secret.yaml -n kserve-ci-e2e-test

Expand All @@ -115,9 +132,17 @@ kustomize build $PROJECT_ROOT/config/overlays/test/clusterresources | \
sed "s|kserve/sklearnserver:latest|${SKLEARN_IMAGE}|" | \
oc apply -n kserve-ci-e2e-test -f -

#
# Add the enablePassthrough annotation to the ServingRuntimes, to let Knative to
# generate passthrough routes. If RawDeployment test are being run, this annotation would have
# no effect, because of missing Knative
oc annotate servingruntimes -n kserve-ci-e2e-test --all serving.knative.openshift.io/enablePassthrough=true

echo "Run E2E tests: $1"
pushd $PROJECT_ROOT >/dev/null
# Note: The following images are set by openshift-ci. Uncomment if you are running on your own machine.
# export CUSTOM_MODEL_GRPC_IMG_TAG=kserve/custom-model-grpc:latest
# export IMAGE_TRANSFORMER_IMG_TAG=kserve/image-transformer:latest

export GITHUB_SHA=$(git rev-parse HEAD)
export CI_USE_ISVC_HOST="1"
./test/scripts/gh-actions/run-e2e-tests.sh "$1"
Expand Down
Loading