Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] Better solution for helm-chart bump pipeline #3988

Open
sotojn opened this issue Mar 6, 2025 · 5 comments
Open

[ci] Better solution for helm-chart bump pipeline #3988

sotojn opened this issue Mar 6, 2025 · 5 comments
Assignees

Comments

@sotojn
Copy link
Contributor

sotojn commented Mar 6, 2025

We had added 3 required checks to our master branch protections to prevent accidental merges to the master when the pull request hasn't passed all tests.

The issue now is that it breaks the build-and-publish-helm-charts ci pipeline:
https://github.com/terascope/teraslice/actions/runs/13641561351/job/38132568786

This has been validated as the cause because we removed these required checks and re-ran the job and it passed. This is because this pipeline updates an index.yaml file and pushes it directly to the master branch.

After looking into potential solutions, I thought I had a path forward but came to some other issues.

Solution 1:

We can instead update the index.yaml locally and have the change in the PR itself. Then the build-and-publish-helm-charts pipeline would skip the step of updating the index.yaml and proceed to generate the chart tarball and publish it to ghcr.

Problems with solution 1:

  1. The developer would be required to have helm installed when bumping the chart itself or when bumping teraslice.
  2. When generating the chart tarball using helm package, it will always have a unique digest because of some creation timestamp. This ends up being displaced when we update the index.yaml with the helm repo index command. Here is a diff of the index.yaml when updating the chart locally:
apiVersion: v1
entries:
  teraslice-chart:
+  - apiVersion: v2
+    appVersion: v2.13.1
+    created: "2025-03-05T16:29:11.445284-07:00"
+    description: Teraslice -  Distributed computing platform for processing JSON data
+    digest: 9c91e733c60eec48b618910cefc0ee15995fa5c113f6d0d4147c39a322280acf
+    keywords:
+    - teraslice
+    - elasticsearch
+    name: teraslice-chart
+    sources:
+    - https://github.com/terascope/teraslice
+    type: application
+    urls:
+    - oci://ghcr.io/terascope/teraslice-chart:2.1.0/teraslice-chart-2.2.0.tgz
+    version: 2.2.0
  - apiVersion: v2
    appVersion: v2.13.0
    created: "2025-03-03T22:53:53.209431792Z"
    description: Teraslice -  Distributed computing platform for processing JSON data
    digest: 93d4b774e809be81dc2d8671e4cdab3c0b8cdfbd092940ef02660bc641054cf6
    keywords:
    - teraslice
    - elasticsearch
    name: teraslice-chart
    sources:
    - https://github.com/terascope/teraslice
    type: application
    urls:
    - oci://ghcr.io/terascope/teraslice-chart:2.1.0
    version: 2.1.0
  - apiVersion: v2
    appVersion: v2.12.2
    created: "2025-01-24T23:41:09.883296275Z"
    description: Teraslice -  Distributed computing platform for processing JSON data
    digest: e079065bcae45c968c040766c4bcf3d96203c158afe16bf534a15d8df35ddb76
    keywords:
    - teraslice
    - elasticsearch
    name: teraslice-chart
    sources:
    - https://github.com/terascope/teraslice
    type: application
    urls:
    - oci://ghcr.io/terascope/teraslice-chart:2.0.0
    version: 2.0.0
+ generated: "2025-03-05T16:29:11.444595-07:00"

I then deleted the tarball and restored everything, then proceeded to regenerate the tarball and run helm index again, and the digest is in fact different. Meaning when we go to generate the tarball in ci and publish to ghcr, it will have a different digest than the one listed in the index.yaml

@sotojn sotojn self-assigned this Mar 6, 2025
@sotojn
Copy link
Contributor Author

sotojn commented Mar 6, 2025

My other proposed solution is to have a pipeline on a pull request that updates the index.yaml in the case that the chart has been bumped. This pipeline would run potentially before all other tests have ran. Also this job would then add the chart tarball as an artifact to the PR, similar to how we add asset builds to PR's. So then when we merge to master, instead of generating a new tarball, it would download the tarball from the PR and upload to ghcr.

Concerns about this solution: If this action runs, it would make changes to the current PR and tests would gave to run again.

@busma13
Copy link
Contributor

busma13 commented Mar 6, 2025

Was there no way to allow the app that commits to master to override the branch protection rules? or is that just a bad idea?

@sotojn
Copy link
Contributor Author

sotojn commented Mar 6, 2025

Was there no way to allow the app that commits to master to override the branch protection rules? or is that just a bad idea?

I might be able to add a ruleset to see if this works. I'll test it on my test-repo to see

@sotojn
Copy link
Contributor Author

sotojn commented Mar 6, 2025

@sotojn
Copy link
Contributor Author

sotojn commented Mar 6, 2025

We have implemented a new ruleset to the master branch that should resolve this issue. We will see if this resolves this issue on the next release of teraslice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants