Update k8s dependencies to v0.26.2 #17211

aledbf · 2023-04-13T20:20:06Z

Description

Sync k8s dependencies to a more recent version.
The goal here is to update clusters to 1.26

How to test

Workspaces should open without issues

Release Notes

NONE

Documentation

Build Options:

/werft with-werft
Run the build with werft instead of GHA
leeway-no-cache
/werft no-test
Run Leeway with --dont-test

Publish Options

/werft publish-to-npm
/werft publish-to-jb-marketplace

Installer Options

with-dedicated-emulation
with-ws-manager-mk2
workspace-feature-flags
Add desired feature flags to the end of the line above, space separated

Preview Environment Options:

/werft with-local-preview
If enabled this will build install/preview
/werft with-preview
/werft with-large-vm
/werft with-gce-vm
If enabled this will create the environment on GCE infra
with-integration-tests=all
Valid options are all, workspace, webapp, ide, jetbrains, vscode, ssh

/hold

werft-gitpod-dev-com · 2023-04-14T09:32:54Z

started the job as gitpod-build-aledbf-sync-k8s.5 because the annotations in the pull request description changed
(with .werft/ from main)

kylos101

@aledbf awesome Re: upgrading to a more recent version.

Would we also upgrade Dedicated to 1.26?

My recollection is we're staying on 1.24 because of the ephemeral storage limit not working - internal link.

Can you tell if the fix landed in 1.26? I don't see the PR or issue mentioned here.

As this is a major upgrade, how should we test? I am thinking an ephemeral cluster makes sense, along with runtime and load tests.

aledbf · 2023-04-14T12:53:00Z

@kylos101 kubernetes/kubernetes#116793

https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.26.md#changelog-since-v1263

aledbf · 2023-04-14T12:54:20Z

As this is a major upgrade, how should we test? I am thinking an ephemeral cluster makes sense, along with runtime and load tests.

Update the packer image to v1.26.3+k3s1, create an ephemeral cluster, and run the test

kylos101 · 2023-04-14T12:54:25Z

@kylos101 kubernetes/kubernetes#116793

https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.26.md#changelog-since-v1263

Ohhh, I see, so they cherry pick the fix to a release, and the changelog cites the PR of the cherry pick PR. Got it!

aledbf · 2023-04-14T12:55:31Z

@kylos101 those things do not depend on this change. You can already use this change against a preview env or an ephemeral cluster. We do not use features not present in 1.23. We only want the fixes and improvements.

googleapis/google-cloud-go#6857

kylos101 · 2023-04-17T12:37:12Z

Looks like the preview environment was torn down, I'll spin it up real quick again to do some brief testing.

@gtsiolis may we ask you for a review of the files you own? I'm not sure which ones those are, but, you show up as a codeowner.

kylos101

Can start workspaces
ImageBuilder e2e tests still pass
Ran all e2e tests, saw many failures (below)

The error is common between many tests:

could not run copy operation: GroupVersion is required when initializing a RESTClient

Tests failing with it: TestAdditionalRepositories, TestOpenWorkspaceFromPrebuildSerialOnly, TestK3s, TestGitHooks, TestRegularWorkspaceTasks, and TestMountProc fail with the same error, so I assume its a setup problem. Investigating.

edit: the failure is here and originates from here, and can be recreating from a Gitpod in this branch via:

go test -v ./... -kubeconfig=/home/gitpod/.kube/config -namespace=default -run TestMountProc

aledbf · 2023-04-17T18:34:02Z

@kylos101 fixed


gitpod /workspace/gitpod/test (aledbf/sync-k8s) $ go test -v ./... -kubeconfig=/home/gitpod/.kube/config -namespace=default -run TestMountProc
?       github.com/gitpod-io/gitpod/test/pkg/agent/daemon       [no test files]
?       github.com/gitpod-io/gitpod/test/pkg/agent/daemon/api   [no test files]
?       github.com/gitpod-io/gitpod/test/pkg/agent/workspace    [no test files]
?       github.com/gitpod-io/gitpod/test/pkg/agent/workspace/api        [no test files]
?       github.com/gitpod-io/gitpod/test/pkg/integration        [no test files]
?       github.com/gitpod-io/gitpod/test/pkg/integration/common [no test files]
?       github.com/gitpod-io/gitpod/test/pkg/report     [no test files]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/components/content-service       0.508s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/components/database      0.547s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/components/image-builder 0.505s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/components/server        0.534s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/components/ws-daemon     0.511s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/components/ws-manager    0.518s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/ide/jetbrains    0.516s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/ide/ssh  0.511s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/ide/vscode       0.512s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/smoke-test       0.507s [no tests to run]
=== RUN   TestMountProc
=== RUN   TestMountProc/proc_mount
=== RUN   TestMountProc/proc_mount/load_test_proc_mount
=== PAUSE TestMountProc/proc_mount/load_test_proc_mount
=== CONT  TestMountProc/proc_mount/load_test_proc_mount
    workspace.go:261: prepare for a connection with ws-manager
    workspace.go:266: established a connection with ws-manager
    workspace.go:270: attemp to start up the workspace directly: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7, harlequin-puffin-2rhnwm9dex4
    workspace.go:295: successfully sent workspace start request
    workspace.go:304: wait for workspace to be fully up and running
    workspace.go:553: prepare for a connection with ws-manager
    workspace.go:574: established for a connection with ws-manager
    workspace.go:582: check if the status of workspace is in the running phase: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7
    workspace.go:615: status: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7, CREATING
    workspace.go:582: check if the status of workspace is in the running phase: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7
    workspace.go:615: status: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7, CREATING
    workspace.go:582: check if the status of workspace is in the running phase: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7
    workspace.go:615: status: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7, INITIALIZING
    workspace.go:582: check if the status of workspace is in the running phase: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7
    workspace.go:615: status: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7, RUNNING
    workspace.go:639: confirmed that the worksapce is running: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7, RUNNING
    workspace.go:309: successful launch of the workspace
    workspace.go:488: waiting for stopping the workspace: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7
    workspace.go:498: attemp to delete the workspace: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7
    workspace.go:781: confirmed the worksapce is stopped: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7, STOPPED
    workspace.go:522: successfully terminated workspace
--- PASS: TestMountProc (186.79s)
    --- PASS: TestMountProc/proc_mount (0.00s)
        --- PASS: TestMountProc/proc_mount/load_test_proc_mount (186.79s)
PASS
ok      github.com/gitpod-io/gitpod/test/tests/workspace        187.321s

kylos101 · 2023-04-17T19:57:49Z

@kylos101 fixed


gitpod /workspace/gitpod/test (aledbf/sync-k8s) $ go test -v ./... -kubeconfig=/home/gitpod/.kube/config -namespace=default -run TestMountProc
?       github.com/gitpod-io/gitpod/test/pkg/agent/daemon       [no test files]
?       github.com/gitpod-io/gitpod/test/pkg/agent/daemon/api   [no test files]
?       github.com/gitpod-io/gitpod/test/pkg/agent/workspace    [no test files]
?       github.com/gitpod-io/gitpod/test/pkg/agent/workspace/api        [no test files]
?       github.com/gitpod-io/gitpod/test/pkg/integration        [no test files]
?       github.com/gitpod-io/gitpod/test/pkg/integration/common [no test files]
?       github.com/gitpod-io/gitpod/test/pkg/report     [no test files]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/components/content-service       0.508s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/components/database      0.547s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/components/image-builder 0.505s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/components/server        0.534s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/components/ws-daemon     0.511s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/components/ws-manager    0.518s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/ide/jetbrains    0.516s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/ide/ssh  0.511s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/ide/vscode       0.512s [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/gitpod-io/gitpod/test/tests/smoke-test       0.507s [no tests to run]
=== RUN   TestMountProc
=== RUN   TestMountProc/proc_mount
=== RUN   TestMountProc/proc_mount/load_test_proc_mount
=== PAUSE TestMountProc/proc_mount/load_test_proc_mount
=== CONT  TestMountProc/proc_mount/load_test_proc_mount
    workspace.go:261: prepare for a connection with ws-manager
    workspace.go:266: established a connection with ws-manager
    workspace.go:270: attemp to start up the workspace directly: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7, harlequin-puffin-2rhnwm9dex4
    workspace.go:295: successfully sent workspace start request
    workspace.go:304: wait for workspace to be fully up and running
    workspace.go:553: prepare for a connection with ws-manager
    workspace.go:574: established for a connection with ws-manager
    workspace.go:582: check if the status of workspace is in the running phase: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7
    workspace.go:615: status: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7, CREATING
    workspace.go:582: check if the status of workspace is in the running phase: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7
    workspace.go:615: status: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7, CREATING
    workspace.go:582: check if the status of workspace is in the running phase: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7
    workspace.go:615: status: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7, INITIALIZING
    workspace.go:582: check if the status of workspace is in the running phase: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7
    workspace.go:615: status: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7, RUNNING
    workspace.go:639: confirmed that the worksapce is running: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7, RUNNING
    workspace.go:309: successful launch of the workspace
    workspace.go:488: waiting for stopping the workspace: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7
    workspace.go:498: attemp to delete the workspace: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7
    workspace.go:781: confirmed the worksapce is stopped: 34eebbc6-9dc2-4395-a140-cd43b6c5d7e7, STOPPED
    workspace.go:522: successfully terminated workspace
--- PASS: TestMountProc (186.79s)
    --- PASS: TestMountProc/proc_mount (0.00s)
        --- PASS: TestMountProc/proc_mount/load_test_proc_mount (186.79s)
PASS
ok      github.com/gitpod-io/gitpod/test/tests/workspace        187.321s

Awesome, thank you! This'll help prevent more test regressions. The remaining regressions I am fixing in WKS-70.

edit: I reran the workspace tests with this fix, and found the following failing tests (which are pre-existing regressions): TestGitHooks, TestGitHubContexts, TestOpenWorkspaceFromOutdatedPrebuild, so, we're good to go.

kylos101

🚀

roboquat added release-note-none do-not-merge/hold do-not-merge/work-in-progress size/XXL labels Apr 13, 2023

aledbf marked this pull request as ready for review April 14, 2023 09:53

aledbf requested review from a team April 14, 2023 09:53

aledbf requested review from gtsiolis and a team as code owners April 14, 2023 09:53

roboquat removed the do-not-merge/work-in-progress label Apr 14, 2023

github-actions bot added team: webapp Issue belongs to the WebApp team team: workspace Issue belongs to the Workspace team team: IDE team: staff-engineers team: security-infrastructure-and-delivery labels Apr 14, 2023

vulkoingim approved these changes Apr 14, 2023

View reviewed changes

kylos101 reviewed Apr 14, 2023

View reviewed changes

Update k8s dependencies to v0.26.2

600b70c

aledbf added 5 commits April 14, 2023 21:01

Update controller-runtime to v0.14.6

6a95970

Update cloud storage

c9902cd

googleapis/google-cloud-go#6857

Update copy options

123d1cf

Update wolfi image

72c37d4

Remove controller-runtime replace directives

d23b6da

aledbf force-pushed the aledbf/sync-k8s branch from 7fc871a to d23b6da Compare April 14, 2023 21:08

easyCZ approved these changes Apr 17, 2023

View reviewed changes

mustard-mh approved these changes Apr 17, 2023

View reviewed changes

aledbf requested a review from kylos101 April 17, 2023 11:15

gtsiolis approved these changes Apr 17, 2023

View reviewed changes

kylos101 reviewed Apr 17, 2023

View reviewed changes

Fix integration tests

0d81bb2

kylos101 approved these changes Apr 17, 2023

View reviewed changes

kylos101 removed the do-not-merge/hold label Apr 17, 2023

roboquat merged commit 891cd54 into main Apr 17, 2023

roboquat deleted the aledbf/sync-k8s branch April 17, 2023 19:59

roboquat added deployed: workspace Workspace team change is running in production deployed: webapp Meta team change is running in production deployed: IDE IDE change is running in production labels Apr 20, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update k8s dependencies to v0.26.2 #17211

Update k8s dependencies to v0.26.2 #17211

aledbf commented Apr 13, 2023 •

edited

Loading

werft-gitpod-dev-com bot commented Apr 14, 2023

kylos101 left a comment

aledbf commented Apr 14, 2023

aledbf commented Apr 14, 2023

kylos101 commented Apr 14, 2023

aledbf commented Apr 14, 2023

kylos101 commented Apr 17, 2023

kylos101 left a comment •

edited

Loading

aledbf commented Apr 17, 2023

kylos101 commented Apr 17, 2023 •

edited

Loading

kylos101 left a comment

Update k8s dependencies to v0.26.2 #17211

Update k8s dependencies to v0.26.2 #17211

Conversation

aledbf commented Apr 13, 2023 • edited Loading

Description

How to test

Release Notes

Documentation

Build Options:

Preview Environment Options:

werft-gitpod-dev-com bot commented Apr 14, 2023

kylos101 left a comment

Choose a reason for hiding this comment

aledbf commented Apr 14, 2023

aledbf commented Apr 14, 2023

kylos101 commented Apr 14, 2023

aledbf commented Apr 14, 2023

kylos101 commented Apr 17, 2023

kylos101 left a comment • edited Loading

Choose a reason for hiding this comment

aledbf commented Apr 17, 2023

kylos101 commented Apr 17, 2023 • edited Loading

kylos101 left a comment

Choose a reason for hiding this comment

aledbf commented Apr 13, 2023 •

edited

Loading

kylos101 left a comment •

edited

Loading

kylos101 commented Apr 17, 2023 •

edited

Loading