Skip to content
This repository was archived by the owner on Feb 13, 2025. It is now read-only.

Commit ebe0c03

Browse files
LaVLaSLaVLaS
committed
Add initial tier 2 components to ODH Contrib
Adds components to the ODH Contrib * AI-Library * Airflow Operator * Hue * Kafka * Argo Workflows * Pachyderm * Radanalyticsio spark-operator * Ray * Superset * Hive Thriftserver * Trino Signed-off-by: Landon LaSmith <[email protected]>
1 parent 9ce15e6 commit ebe0c03

File tree

208 files changed

+16123
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

208 files changed

+16123
-0
lines changed

‎OWNERS

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# Each list is sorted alphabetically, additions should maintain that order
2+
approvers:
3+
- anishasthana
4+
- lavlas
5+
- samuelvl
6+
- VaishnaviHire
7+
8+
reviewers:
9+
- anishasthana
10+
- lavlas
11+
- samuelvl
12+
- VaishnaviHire

‎README.md

+39
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Open Data Hub Manifests
2+
A repository for kustomize manifests contributed by [Open Data Hub](https://opendatahub.io) community members that provide integrations of value to the greater community
3+
4+
## Community
5+
* Open Data Hub Community: [https://github.com/opendatahub-io/opendatahub-community](https://github.com/opendatahub-io/opendatahub-community)
6+
* Community Discussions: [https://odh-io.slack.com](https://odh-io.slack.com)
7+
* Community meetings: [http://opendatahub.io/community.html](http://opendatahub.io/community.html)
8+
9+
10+
## ODH Contrib Components
11+
12+
Open Data Hub is an end-to-end AI/ML platform on top of OpenShift Container Platform that provides a core set of integrated components to support end end-to-end MLOps workflow for Data Scientists and Engineers. The components currently available as part of the ODH Core deployment are:
13+
14+
15+
| ODH Version | Component |
16+
| ----------- | ------------------------------------------------------- |
17+
| 1.4 | [Ray.io](ray/README.md) |
18+
| 1.4 | [Pachyderm](odhpachyderm/README.md) |
19+
| 1.4 | [Superset](superset/README.md) |
20+
| 1.3 | [ArgoWorkflows](odhargo/README.md) |
21+
| 1.3 | [Hue](hue/README.md) |
22+
| 1.3 | [Hive Thrift Server](thriftserver/README.md) |
23+
| 1.3 | [Trino](trino/README.md) |
24+
| 1.3 | [Radanalytics Spark Operator](radanalyticsio/README.md) |
25+
26+
27+
Any components that were removed with the update to ODH 1.4 have been relocated to the [ODH Contrib](https://github.com/opendatahub-io-contrib) organization under the [odh-contrib-manifests](https://github.com/opendatahub-io-contrib/odh-contrib-manifests) repo. You can reference the [odh-contrib kfdef](kfdef/odh-contrib.yaml) as a reference on how to deploy any of the odh-contrib-manifests components
28+
29+
## Deploy
30+
31+
We are relying on [Kustomize v3](https://github.com/kubernetes-sigs/kustomize), [kfctl](https://github.com/kubeflow/kfctl) and [Open Data Hub Operator](https://github.com/opendatahub-io/opendatahub-operator/blob/master/operator.md) for deployment.
32+
33+
The two ways to deploy are:
34+
35+
1. Following [Getting Started](http://opendatahub.io/docs/getting-started/quick-installation.html) guide using a KFDef from this repository as the custom resource.
36+
1. Using `kfctl` and follow the documentation at [Kubeflow.org](https://www.kubeflow.org/docs/openshift/). The only change is to use this repository instead of Kubeflow manifests.
37+
38+
## Issues
39+
To submit issues please file a GitHub issue in [odh-contrib-manifests](https://github.com/opendatahub-io/odh-contrib-manifests/issues)

‎ai-library/README.md

+43
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# AI Library
2+
3+
The AI Library component installs the AI Library operator as well as the CRD for AI Library.
4+
5+
### Folders
6+
There are 2 folders that contain installation files:
7+
8+
`cluster/base` This folder includes the yaml files that will install the AI Library CRD on your cluster.
9+
10+
`operator/base` This folder includes the yaml files to set up the deployment for the operator in your namespace along with the service account, role, and role binding that is necessary.
11+
12+
### Installation
13+
To install AI Library add the following to the `KfDef` in your yaml file.
14+
15+
Note, to use AI Library, you also need to have [Seldon](../odhseldon/README.md) installed.
16+
In addition, Seldon must be placed before ai-library in the KfDef. This is shown in the example below.
17+
18+
```yaml
19+
- kustomizeConfig:
20+
repoRef:
21+
name: manifests
22+
path: odhseldon/cluster
23+
name: odhseldon
24+
- kustomizeConfig:
25+
repoRef:
26+
name: manifests
27+
path: ai-library/cluster
28+
name: ai-library-cluster
29+
- kustomizeConfig:
30+
repoRef:
31+
name: manifests
32+
path: ai-library/operator
33+
name: ai-library-operator
34+
```
35+
36+
### Instantiate an AI Library instance
37+
You can install the sample AI Library custom resource found in the AI Library repository by running the following command.
38+
39+
```sh
40+
oc create -f https://gitlab.com/opendatahub/ai-library/-/raw/master/operator/deploy/crds/ailibrary_v1alpha1_ailibrary_cr.yaml
41+
```
42+
43+
For details on how to work with AI Library, see the official AI Library repository [https://gitlab.com/opendatahub/ai-library/]

‎ai-library/cluster/base/crd.yaml

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
apiVersion: apiextensions.k8s.io/v1beta1
2+
kind: CustomResourceDefinition
3+
metadata:
4+
name: ailibraries.ailibrary.opendatahub.io
5+
spec:
6+
group: ailibrary.opendatahub.io
7+
names:
8+
kind: AILibrary
9+
listKind: AILibraryList
10+
plural: ailibraries
11+
singular: ailibrary
12+
scope: Namespaced
13+
subresources:
14+
status: {}
15+
version: v1alpha1
16+
versions:
17+
- name: v1alpha1
18+
served: true
19+
storage: true

‎ai-library/cluster/base/kustomization.yaml

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
apiVersion: kustomize.config.k8s.io/v1beta1
2+
kind: Kustomization
3+
4+
resources:
5+
- crd.yaml
6+
7+
commonLabels:
8+
opendatahub.io/component: "true"
9+
component.opendatahub.io/name: ailibrary
10+
app.kubernetes.io/part-of: ailibrary
11+
12+
generatorOptions:
13+
disableNameSuffixHash: true

‎ai-library/operator/base/kustomization.yaml

+23
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
apiVersion: kustomize.config.k8s.io/v1beta1
2+
kind: Kustomization
3+
4+
resources:
5+
- service-account.yaml
6+
- role.yaml
7+
- role-binding.yaml
8+
- operator.yaml
9+
10+
namespace: opendatahub
11+
12+
commonLabels:
13+
opendatahub.io/component: "true"
14+
component.opendatahub.io/name: ailibrary
15+
app.kubernetes.io/part-of: ailibrary
16+
17+
images:
18+
- name: quay.io/opendatahub/ai-library-operator
19+
newName: quay.io/opendatahub/ai-library-operator
20+
newTag: v0.6
21+
22+
generatorOptions:
23+
disableNameSuffixHash: true

‎ai-library/operator/base/operator.yaml

+49
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: ailibrary-operator
5+
spec:
6+
replicas: 1
7+
selector:
8+
matchLabels:
9+
name: ailibrary-operator
10+
template:
11+
metadata:
12+
labels:
13+
name: ailibrary-operator
14+
spec:
15+
serviceAccountName: ailibrary-operator
16+
containers:
17+
- name: ansible
18+
command:
19+
- /usr/local/bin/ao-logs
20+
- /tmp/ansible-operator/runner
21+
- stdout
22+
image: "quay.io/opendatahub/ai-library-operator"
23+
imagePullPolicy: "Always"
24+
volumeMounts:
25+
- mountPath: /tmp/ansible-operator/runner
26+
name: runner
27+
readOnly: true
28+
- name: ailibrary-operator
29+
image: "quay.io/opendatahub/ai-library-operator"
30+
imagePullPolicy: "Always"
31+
volumeMounts:
32+
- mountPath: /tmp/ansible-operator/runner
33+
name: runner
34+
env:
35+
- name: WATCH_NAMESPACE
36+
valueFrom:
37+
fieldRef:
38+
fieldPath: metadata.namespace
39+
- name: POD_NAME
40+
valueFrom:
41+
fieldRef:
42+
fieldPath: metadata.name
43+
- name: OPERATOR_NAME
44+
value: "ailibrary-operator"
45+
- name: ANSIBLE_VERBOSITY
46+
value: "4"
47+
volumes:
48+
- name: runner
49+
emptyDir: {}

‎ai-library/operator/base/role-binding.yaml

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
kind: RoleBinding
2+
apiVersion: rbac.authorization.k8s.io/v1
3+
metadata:
4+
name: ailibrary-operator
5+
subjects:
6+
- kind: ServiceAccount
7+
name: ailibrary-operator
8+
roleRef:
9+
kind: Role
10+
name: ailibrary-operator
11+
apiGroup: rbac.authorization.k8s.io

‎ai-library/operator/base/role.yaml

+104
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
apiVersion: rbac.authorization.k8s.io/v1
2+
kind: Role
3+
metadata:
4+
creationTimestamp: null
5+
name: ailibrary-operator
6+
rules:
7+
- apiGroups:
8+
- ""
9+
resources:
10+
- pods
11+
- services
12+
- services/finalizers
13+
- endpoints
14+
- persistentvolumeclaims
15+
- events
16+
- configmaps
17+
- secrets
18+
verbs:
19+
- '*'
20+
- apiGroups:
21+
- apps
22+
resources:
23+
- deployments
24+
- daemonsets
25+
- replicasets
26+
- statefulsets
27+
verbs:
28+
- '*'
29+
- apiGroups:
30+
- monitoring.coreos.com
31+
resources:
32+
- servicemonitors
33+
verbs:
34+
- get
35+
- create
36+
- apiGroups:
37+
- apps
38+
resourceNames:
39+
- operator
40+
resources:
41+
- deployments/finalizers
42+
- services/finalizers
43+
verbs:
44+
- '*'
45+
- apiGroups:
46+
- ""
47+
resources:
48+
- pods
49+
verbs:
50+
- get
51+
- apiGroups:
52+
- apps
53+
resources:
54+
- replicasets
55+
verbs:
56+
- get
57+
- apiGroups:
58+
- ailibrary.opendatahub.io
59+
resources:
60+
- '*'
61+
verbs:
62+
- '*'
63+
- apiGroups:
64+
- machinelearning.seldon.io
65+
resources: ["*"]
66+
verbs: ["*"]
67+
- apiGroups:
68+
- ""
69+
- route.openshift.io
70+
resources:
71+
- routes/status
72+
- routes
73+
verbs:
74+
- '*'
75+
- apiGroups:
76+
- ""
77+
- apps.openshift.io
78+
resources:
79+
- deploymentconfigs
80+
- deploymentconfigs/status
81+
- deploymentconfigs/scale
82+
verbs:
83+
- create
84+
- delete
85+
- deletecollection
86+
- get
87+
- list
88+
- patch
89+
- update
90+
- watch
91+
- apiGroups:
92+
- ""
93+
- batch
94+
resources:
95+
- jobs
96+
- jobs/status
97+
verbs:
98+
- create
99+
- delete
100+
- get
101+
- list
102+
- patch
103+
- update
104+
- watch

‎ai-library/operator/base/service-account.yaml

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
apiVersion: v1
2+
kind: ServiceAccount
3+
metadata:
4+
name: ailibrary-operator

‎ai-library/tests/README.md

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Basic tests for ai-library can be added to the ODH CI tests runs by copying the contents of the `ai-library/tests` directory to the `tests` folder for CI runs.
2+
3+
```sh
4+
cp ${ODH_MANIFESTS_REPO_DIR}/ai-library/tests/* ${ODH_MANIFESTS_REPO_DIR}/tests
5+
```
6+

‎ai-library/tests/basictests/ailibrary.sh

+61
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
#!/bin/bash
2+
3+
source $TEST_DIR/common
4+
5+
MY_DIR=$(readlink -f `dirname "${BASH_SOURCE[0]}"`)
6+
7+
source ${MY_DIR}/../util
8+
9+
os::test::junit::declare_suite_start "$MY_SCRIPT"
10+
11+
OPERATOR_IMAGE="quay.io/opendatahub/ai-library-operator:v0.6"
12+
AI_LIBRARY_CR="${MY_DIR}/../resources/ai_library_cr.yaml"
13+
14+
function test_ai_library() {
15+
header "Testing AI Library installation"
16+
os::cmd::expect_success "oc project ${ODHPROJECT}"
17+
os::cmd::try_until_text "oc get deployment ailibrary-operator" "ailibrary-operator" $odhdefaulttimeout $odhdefaultinterval
18+
os::cmd::try_until_text "oc get deployment ailibrary-operator -o jsonpath='{$.spec.template.spec.containers[0].image}'" ${OPERATOR_IMAGE} $odhdefaulttimeout $odhdefaultinterval
19+
os::cmd::try_until_text "oc get pods -l name=ailibrary-operator --field-selector='status.phase=Running' -o jsonpath='{$.items[*].metadata.name}'" "ailibrary-operator" $odhdefaulttimeout $odhdefaultinterval
20+
runningpods=($(oc get pods -l name=ailibrary-operator --field-selector="status.phase=Running" -o jsonpath="{$.items[*].metadata.name}"))
21+
os::cmd::expect_success_and_text "echo ${#runningpods[@]}" "1"
22+
}
23+
24+
function create_ai_lib_cr() {
25+
os::cmd::expect_success "oc create -f ${AI_LIBRARY_CR}"
26+
header "The AI Library Opeator should create seldondeployments and a dc for the UI, wait for them to show up"
27+
os::cmd::try_until_not_text "oc get seldondeployment flakes-predict" "not found" $odhdefaulttimeout $odhdefaultinterval
28+
os::cmd::try_until_not_text "oc get seldondeployment regression-predict" "not found" $odhdefaulttimeout $odhdefaultinterval
29+
os::cmd::try_until_not_text "oc get deploymentconfig ai-library-ui" "not found" $odhdefaulttimeout $odhdefaultinterval
30+
}
31+
32+
function delete_ai_lib_cr() {
33+
os::cmd::expect_success "oc delete -f ${AI_LIBRARY_CR}"
34+
header "The AI Library Opeator should delete seldondeployments and a dc for the UI, wait for them to disappear"
35+
os::cmd::try_until_text "oc get seldondeployment flakes-predict" "not found" $odhdefaulttimeout $odhdefaultinterval
36+
os::cmd::try_until_text "oc get seldondeployment regression-predict" "not found" $odhdefaulttimeout $odhdefaultinterval
37+
os::cmd::try_until_text "oc get deploymentconfig ai-library-ui" "not found" $odhdefaulttimeout $odhdefaultinterval
38+
}
39+
40+
function verify_functionality() {
41+
header "Verifying that the SeldonDeployments are up and running"
42+
os::cmd::try_until_text 'oc get seldondeployment flakes-predict -o jsonpath="{$.status.state}"' "Available" $odhdefaulttimeout $odhdefaultinterval
43+
os::cmd::try_until_text 'oc get seldondeployment regression-predict -o jsonpath="{$.status.state}"' "Available" $odhdefaulttimeout $odhdefaultinterval
44+
45+
# Check that the UI is indeed up by curling the route
46+
uiroute=$(oc get route ui -o jsonpath="{$.status.ingress[0].host}")
47+
os::cmd::try_until_text "curl -k https://$uiroute" "AI Library"
48+
}
49+
50+
function test_ai_library_functionality() {
51+
header "Testing AI Library functionality"
52+
os::cmd::expect_success "oc project ${ODHPROJECT}"
53+
create_ai_lib_cr
54+
verify_functionality
55+
delete_ai_lib_cr
56+
}
57+
58+
test_ai_library
59+
test_ai_library_functionality
60+
61+
os::test::junit::declare_suite_end

0 commit comments

Comments
 (0)