RHOAIENG-19478, RHOAIENG-19715, RHOAIENG-19485 - Explore versions compatibility, upgrade Jupyter packages and upgrade Runtime images

[daniellutz]

This PR aims to solve three issues:

• RHOAIENG-19478 - Explore versions compatibility for the upcoming updates on the workbenches
• RHOAIENG-19715 - Update JupyterLab images
• RHOAIENG-19485 - Update runtime images

Description

Based on the research of packages that could and should be upgraded, these are the versions that could be upgraded without generating conflicts on Pipfile.lock files.

The Runtime images changes were planned to be present on another PR, but since there are many similarities with the Explore versions and Update JupyterLab changes, it has been added here as well to ease the readability.

There were some limitations, or some changes that are a little bit different from previous package versions:

1. JupyterLab could not be upgraded to latest because odh-elyra requires a specific version (except minimal, that do not contain Elyra, but kept them all with the same version to avoid the confusion of having different versions across the notebook for such important package as jupyterlab), as follows:

The conflict is caused by:
    The user requested jupyterlab==4.3.5
    odh-elyra 4.2.0 depends on jupyterlab~=4.2.5

2. Tensorflow requires different packages due to run on different accelerators:

Tensorflow on ROCm:

tensorflow_rocm = "~=2.14.0.600"
tensorboard = "~=2.14.0"

Tensorflow on CUDA:

tensorflow = {version = "~=2.18.0", extras = ["and-cuda"]}
tensorboard = "~=2.18.0"

3. Still talking about Tensorflow on ROCm, looks like we need to replace the tensorflow-rocm package (that is locked on version 2.14.0.600 since January 2024) with a ROCm wheel repository, as follows:

https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/3rd-party/tensorflow-install.html#using-a-wheels-package

4. Previously, Pytorch had releases only with the version - torch = {version = "~=2.4.0", index = "pytorch"}, but now they require the local version identifier - torch = {version = "==2.6.0+rocm6.2.4", index = "pytorch"}

---

Besides that, the vast majority of packages were successfully upgraded without major problems.

TrustyAI wasn't updated because it maintains its libraries with specific versions, not requiring an upgrade unless necessary.

How Has This Been Tested?

The test has been done by building the notebook images locally and running them with Podman, as follows:

For each one of the following images, the instructions below have been executed:

Notebook images:

• jupyter/datascience/ubi9-python-3.11
• jupyter/minimal/ubi9-python-3.11
• jupyter/pytorch/ubi9-python-3.11
• jupyter/rocm/pytorch/ubi9-python-3.11
• jupyter/rocm/tensorflow/ubi9-python-3.11
• jupyter/tensorflow/ubi9-python-3.11

Instructions

1. Build the notebook image with the make command:

$ make {image} -e IMAGE_REGISTRY=quay.io/{myuser}/workbench-images -e RELEASE=2025a

Example:

$ make jupyter-minimal-ubi9-python-3.11 -e IMAGE_REGISTRY=quay.io/{myuser}/workbench-images -e RELEASE=2025a

2. Deploy the image using Podman, after the build process is done:

$ podman  run -it -p  8888:8888 quay.io/{myuser}/workbench-images:{tag}-{hash}

Example:

$ podman  run -it -p  8888:8888 quay.io/johndoe/workbench-images:cuda-jupyter-minimal-ubi9-python-3.11-2025a_20250310

7. Open the JupyterLab provided URL in the terminal and ensure it runs properly.

Merge criteria:

• The commits are squashed in a cohesive manner and have meaningful messages.
• Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
• The developer has manually tested the changes and verified that the changes work

[atheo89]

For this point:

The conflict is caused by:
    The user requested jupyterlab==4.3.5
    odh-elyra 4.2.0 depends on jupyterlab~=4.2.5

we should coordinate with @caponetto And as Adriel mentioned on scrum we can bring the fix on a fast upcoming release

[jiridanek]

I don't see any updates being made to /manifests files, please also update versions of packages in manifests. Or is that a separate Jira task?

[atheo89]

I don't see any updates being made to /manifests files, please also update versions of packages in manifests. Or is that a separate Jira task?

@jiridanek There is a separated task: https://issues.redhat.com/browse/RHOAIENG-19486

[atheo89]

TrustyAI wasn't updated because it maintains its libraries with specific versions, not requiring an upgrade unless necessary.

Yes and no, some packages yes they related with trustyai itself but some others nolike for example: https://github.com/opendatahub-io/notebooks/blob/main/jupyter/trustyai/ubi9-python-3.11/Pipfile#L30C1-L30C6 did you try to updated and you faced conflict with trusty itself?

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from jiridanek. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[daniellutz]

hey, I did a couple of changes:

• tried to keep up with the requirements.txt files synced with the Pipfile.lock after changes;
• added the runtime images to this PR as well;

pending:

• still need to double check the ROCm Tensorflow packages to make sure we can go with TF 2.16 using the ROCm wheel repository (unless this strategy is wrong and I need to do something else);
• fix all tests for the notebooks;
• update the common packages for TrustyAI (the ones that are not limited by TrustyAI requirements);

again: any suggestion, concern, feedback, they are all really appreciated.

thanks!

[openshift-ci]

@daniellutz: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/rocm-runtimes-ubi9-e2e-tests	8df03b7	link	true	/test rocm-runtimes-ubi9-e2e-tests
ci/prow/rocm-notebooks-e2e-tests	8df03b7	link	true	/test rocm-notebooks-e2e-tests
ci/prow/runtimes-ubi9-e2e-tests	8df03b7	link	true	/test runtimes-ubi9-e2e-tests
ci/prow/notebooks-ubi9-e2e-tests	8df03b7	link	true	/test notebooks-ubi9-e2e-tests

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[atheo89]

Hey Daniel, apart of the pending items that you have already listed, i noticed that may we need to update jupyterlab-git dependency as well.