Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Infra: Migrate CI to Gradle #885

Merged
merged 89 commits into from
Mar 8, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
25dea60
Misc
Haarolean Oct 25, 2024
1e668f8
Impl authentication page backend
Haarolean Oct 25, 2024
7b6cf2e
Merge remote-tracking branch 'origin/main' into auth_page_be
Haarolean Oct 25, 2024
d91e8b9
Add POST auth endpoint
Haarolean Nov 17, 2024
5ec0abe
Merge branch 'main' into auth_page_be
Haarolean Dec 17, 2024
3e61608
resolves #72
Nilumilak May 20, 2024
09f2070
feat: auth page
Nilumilak Nov 17, 2024
09062ef
feat: auth page
Nilumilak Dec 17, 2024
92b7db6
Fix missing controller tag
Haarolean Dec 17, 2024
8e78c51
feat: auth page
Nilumilak Dec 17, 2024
b8161eb
feat: auth page
Nilumilak Dec 18, 2024
f65874a
Lint
Haarolean Dec 18, 2024
eae23e2
Fixed login page renderer
germanosin Dec 23, 2024
6cf3881
Initial gradle support
germanosin Jan 15, 2025
cbc96ff
Merge main
germanosin Jan 15, 2025
4a6ee2b
Plugins varion catalog
germanosin Jan 15, 2025
7c5fbd1
Plugins varion catalog
germanosin Jan 15, 2025
112ede0
Added jib and nexus publish
germanosin Jan 15, 2025
8085237
Update node version
germanosin Jan 16, 2025
4f568c5
Update default version
germanosin Jan 16, 2025
17e0624
Fixing comments
germanosin Jan 17, 2025
f828fd3
Fixing comments
germanosin Jan 17, 2025
336eb76
Version cleanup, minor refactoring
Haarolean Jan 17, 2025
ac99710
Simplify dep mngmnt, update libs
Haarolean Jan 17, 2025
f7183fa
Dedupe boot version
Haarolean Jan 23, 2025
808807e
Added e2e flags
germanosin Jan 29, 2025
436904b
Switched release flag instead of prod
germanosin Jan 29, 2025
21d2155
rm todo
Haarolean Jan 29, 2025
a0395cf
update workflow trigger paths
Haarolean Jan 30, 2025
9828e76
Update CODEOWNERS
Haarolean Jan 30, 2025
e7cebd0
cache mvn -> gradle
Haarolean Jan 30, 2025
f5b5412
run e2e on release
Haarolean Jan 30, 2025
f7c028b
Add sonar plugin
Haarolean Jan 30, 2025
aa408c5
Update backend_tests.yml, rm sonar as it's run is cloud based now
Haarolean Jan 30, 2025
979c0d0
rename codeql for clarity
Haarolean Jan 30, 2025
c6a1279
upd cve
Haarolean Jan 30, 2025
eeef79e
formatting
Haarolean Jan 30, 2025
2239d51
Fix e2e
Haarolean Feb 19, 2025
7d37d66
Bump spring boot to match main branch
Haarolean Feb 19, 2025
eb39338
fix suite path
Haarolean Feb 20, 2025
45ffbd4
fix build with gradle
Haarolean Feb 25, 2025
0501729
forgor img build
Haarolean Feb 25, 2025
80e1ae1
fix img version
Haarolean Feb 25, 2025
0e898c2
rm redundant goal
Haarolean Feb 25, 2025
5bec734
test with clean
Haarolean Feb 25, 2025
693191c
fix selenide results path
Haarolean Feb 25, 2025
b459469
WIP debug
Haarolean Feb 25, 2025
f908ab1
WIP log chrome
Haarolean Feb 25, 2025
df446da
fix build dirs
Haarolean Feb 25, 2025
c1b9b44
WIP test fix
Haarolean Feb 25, 2025
5bcd1af
WIP test
Haarolean Feb 25, 2025
cafddae
WIP test
Haarolean Feb 25, 2025
f01e218
fix selenium version
Haarolean Feb 26, 2025
2b76c1d
WIP update chrome
Haarolean Feb 26, 2025
9763f72
yo
Haarolean Feb 26, 2025
c398c29
less verbose
Haarolean Feb 26, 2025
e085629
nuke local e2e setup
Haarolean Feb 26, 2025
3b9e548
nuke old redundant files
Haarolean Feb 26, 2025
ca11379
fix frontend build
Haarolean Feb 26, 2025
4d77f6d
fix e2e reports condition
Haarolean Feb 26, 2025
1a48a7d
Revert "fix selenium version"
Haarolean Feb 26, 2025
afbd5eb
Fix current CVEs
Haarolean Feb 26, 2025
6c6dc97
Fix artifacts caching
Haarolean Feb 26, 2025
f39013c
Fix cve flow
Haarolean Feb 26, 2025
bf97110
Fix JAR_FILE arg values
Haarolean Feb 26, 2025
45215a1
Fix artifacts paths
Haarolean Feb 26, 2025
9c9c6f7
Fix allure report link
Haarolean Feb 26, 2025
daa8341
Separate tests & sonar tasks
Haarolean Feb 26, 2025
1171c40
nuke old wf
Haarolean Feb 26, 2025
20c19f8
fix sonar
Haarolean Feb 26, 2025
15be347
rm frontend sonar job
Haarolean Feb 26, 2025
3e97141
refactor
Haarolean Feb 26, 2025
8755a0c
Fix main & release wfs
Haarolean Feb 26, 2025
7bedc73
Fix JAR_FILE again
Haarolean Feb 26, 2025
cc391b3
Fix JAR_FILE again (2)
Haarolean Feb 28, 2025
4ac28b1
WIP WIP WIP test
Haarolean Mar 1, 2025
f55614a
wip rm
Haarolean Mar 1, 2025
e4f438e
wip rm
Haarolean Mar 2, 2025
7ee0be9
fix
Haarolean Mar 2, 2025
6ae0e2d
Revert "wip rm"
Haarolean Mar 2, 2025
c643b09
Revert "wip rm"
Haarolean Mar 2, 2025
b4bfe62
revert
Haarolean Mar 2, 2025
eeb47d5
refactor
Haarolean Mar 2, 2025
2187993
other misc updates
Haarolean Mar 2, 2025
1af1f07
aren't we all
Haarolean Mar 2, 2025
a60c015
Merge remote-tracking branch 'origin/gradle_final' into gradle_ci
Haarolean Mar 8, 2025
a45d5ef
revert unnecessary
Haarolean Mar 8, 2025
33284a5
add vite commit
Haarolean Mar 8, 2025
a6551ef
get back forgotten stuff
Haarolean Mar 8, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@
"features": {
"ghcr.io/devcontainers/features/java:1": {
"version": "none",
"installMaven": "true",
"installGradle": "false"
"installMaven": "false",
"installGradle": "true"
},
"ghcr.io/devcontainers/features/docker-in-docker:2": {}
},
Expand Down
5 changes: 4 additions & 1 deletion .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@


# BACKEND
/pom.xml @kafbat/backend
/build.gradle @kafbat/backend
/gradle.properties @kafbat/backend
/settings.gradle @kafbat/backend
/gradle/ @kafbat/backend
/contract/ @kafbat/backend
/api/ @kafbat/backend
/serde-api/ @kafbat/backend
Expand Down
2 changes: 1 addition & 1 deletion .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
version: 2
updates:

- package-ecosystem: maven
- package-ecosystem: gradle
directory: "/"
schedule:
interval: weekly
Expand Down
77 changes: 0 additions & 77 deletions .github/workflows/aws_ami_publish.yml

This file was deleted.

6 changes: 5 additions & 1 deletion .github/workflows/backend_main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@ on:
push:
branches: [ "main" ]
paths:
- "pom.xml"
- "build.gradle"
- "gradle.properties"
- "settings.gradle"
- "gradle/libs.versions.toml"

- "contract/**"
- "api/**"
- "serde-api/**"
Expand Down
6 changes: 5 additions & 1 deletion .github/workflows/backend_pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,11 @@ on:
pull_request_target:
types: [ "opened", "reopened", "synchronize" ]
paths:
- "pom.xml"
- "build.gradle"
- "gradle.properties"
- "settings.gradle"
- "gradle/libs.versions.toml"

- "contract/**"
- "api/**"
- "serde-api/**"
Expand Down
40 changes: 6 additions & 34 deletions .github/workflows/backend_tests.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: "Backend: build and test"
name: "Backend: tests & sonar"

on:
workflow_call:
Expand Down Expand Up @@ -30,40 +30,12 @@ jobs:
with:
java-version: '21'
distribution: 'zulu'
cache: 'maven'
cache: 'gradle'

- name: Cache SonarCloud packages
uses: actions/cache@v4
with:
path: ~/.sonar/cache
key: ${{ runner.os }}-sonar
restore-keys: ${{ runner.os }}-sonar

- name: "Pull request: Maven tests & sonar"
if: ${{ inputs.event_name == 'pull_request' || inputs.event_name == 'pull_request_target' }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN_BACKEND }}
HEAD_REF: ${{ github.head_ref }}
BASE_REF: ${{ github.base_ref }}
SKIP_SONAR: "true" # TODO remove when public
- name: "Tests"
run: |
./mvnw -B -ntp versions:set -DnewVersion=${{ github.event.pull_request.head.sha }}
./mvnw -B -V -ntp verify org.sonarsource.scanner.maven:sonar-maven-plugin:sonar \
-Dsonar.skip=${SKIP_SONAR} \
-Dsonar.projectKey=io.kafbat:kafbat-ui_backend \
-Dsonar.pullrequest.key=${{ github.event.pull_request.number }} \
-Dsonar.pullrequest.branch=$HEAD_REF \
-Dsonar.pullrequest.base=$BASE_REF
./gradlew :api:test --info

- name: "Main: Maven tests & sonar"
if: ${{ inputs.event_name == 'push' }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN_BACKEND }}
SKIP_SONAR: "true" # TODO remove when public
- name: "Sonar"
run: |
./mvnw -B -ntp versions:set -DnewVersion=$GITHUB_SHA
./mvnw -B -V -ntp verify org.sonarsource.scanner.maven:sonar-maven-plugin:sonar \
-Dsonar.skip=${SKIP_SONAR} \
-Dsonar.projectKey=io.kafbat:kafbat-ui_backend
./gradlew sonar --info
10 changes: 9 additions & 1 deletion .github/workflows/block_merge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,12 @@ jobs:
with:
mode: exactly
count: 0
labels: "status/blocked, status/needs-attention, status/on-hold, status/pending, status/triage, status/pending-backend, status/pending-frontend, status/pending-QA"
labels: >
status/blocked,
status/needs-attention,
status/on-hold,
status/pending,
status/triage,
status/pending-backend,
status/pending-frontend,
status/pending-QA
4 changes: 2 additions & 2 deletions .github/workflows/branch-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
with:
java-version: '21'
distribution: 'zulu'
cache: 'maven'
cache: 'gradle'
- name: Build
id: build
run: |
Expand Down Expand Up @@ -69,7 +69,7 @@ jobs:
push: true
tags: 297478128798.dkr.ecr.eu-central-1.amazonaws.com/kafka-ui:${{ steps.extract_branch.outputs.tag }}
build-args: |
JAR_FILE=api-${{ steps.build.outputs.version }}.jar
JAR_FILE=build/libs/api-${{ steps.build.outputs.version }}.jar
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache
outputs:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/build-public-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
with:
java-version: '21'
distribution: 'zulu'
cache: 'maven'
cache: 'gradle'
- name: Build
id: build
run: |
Expand Down Expand Up @@ -68,7 +68,7 @@ jobs:
push: true
tags: ${{ vars.ECR_REGISTRY }}/${{ github.repository }}:${{ steps.extract_branch.outputs.tag }}
build-args: |
JAR_FILE=api-${{ steps.build.outputs.version }}.jar
JAR_FILE=build/libs/api-${{ steps.build.outputs.version }}.jar
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache
- name: make comment with private deployment link
Expand Down
11 changes: 8 additions & 3 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,13 @@ on:
# The branches below must be a subset of the branches above
branches: [ main ]
paths:
- "build.gradle"
- "gradle.properties"
- "settings.gradle"
- "gradle/libs.versions.toml"

- 'frontend/**'
- "pom.xml"

- "contract/**"
- 'api/**'
- 'serde-api/**'
Expand All @@ -24,7 +29,7 @@ permissions:

jobs:
analyze:
name: Analyze
name: CodeQL Analyze
runs-on: ubuntu-latest
strategy:
fail-fast: false
Expand All @@ -47,7 +52,7 @@ jobs:
with:
java-version: '21'
distribution: 'zulu'
cache: 'maven'
cache: 'gradle'

- name: Autobuild
uses: github/codeql-action/autobuild@v3
Expand Down
17 changes: 9 additions & 8 deletions .github/workflows/cve_checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,16 @@ jobs:
with:
java-version: '21'
distribution: 'zulu'
cache: 'maven'
cache: 'gradle'

- name: Build project
id: build
run: |
./mvnw -B -ntp versions:set -DnewVersion=$GITHUB_SHA
./mvnw -B -V -ntp clean package -DskipTests
export VERSION=$(./mvnw -q -Dexec.executable=echo -Dexec.args='${project.version}' --non-recursive exec:exec)
echo "version=${VERSION}" >> $GITHUB_OUTPUT
./gradlew clean build \
-x test \
-Pbuild-docker-images=true \
-Pinclude-frontend=true \
-Pversion=latest

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
Expand All @@ -60,16 +61,16 @@ jobs:
push: false
load: true
tags: |
ghcr.io/kafbat/kafka-ui:${{ steps.build.outputs.version }}
ghcr.io/kafbat/kafka-ui:latest
build-args: |
JAR_FILE=api-${{ steps.build.outputs.version }}.jar
JAR_FILE=build/libs/api-latest.jar
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache

- name: Run CVE checks
uses: aquasecurity/[email protected]
with:
image-ref: "ghcr.io/kafbat/kafka-ui:${{ steps.build.outputs.version }}"
image-ref: "ghcr.io/kafbat/kafka-ui:latest"
format: "table"
exit-code: "1"

Expand Down
11 changes: 6 additions & 5 deletions .github/workflows/docker_build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,11 @@ jobs:
ref: ${{ github.event.pull_request.head.sha }}
token: ${{ github.token }}

- name: Download maven artifacts
- name: Download artifacts
uses: actions/download-artifact@v4
with:
name: kafbat-ui-${{ inputs.version }}
path: api/target
path: api/build/libs

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
Expand All @@ -44,9 +44,10 @@ jobs:
restore-keys: |
${{ runner.os }}-buildx-

# Build multi platform images and loading them at the same time is not possible with default container runtime : https://github.com/docker/buildx/issues/59
# Build multi-platform images and loading them at the same time is not possible with default container runtime : https://github.com/docker/buildx/issues/59
# So let's use containerd instead as it supports this option
# Also containerd is one of the option to allow preserving provenance attestations :https://docs.docker.com/build/attestations/#creating-attestations
# Also containerd is one of the option to allow preserving provenance attestations:
# https://docs.docker.com/build/attestations/#creating-attestations
- name: Setup docker with containerd
uses: crazy-max/ghaction-setup-docker@v3
with:
Expand All @@ -71,7 +72,7 @@ jobs:
tags: |
kafka-ui:temp
build-args: |
JAR_FILE=api-${{ inputs.version }}.jar
JAR_FILE=build/libs/api-${{ inputs.version }}.jar
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache

Expand Down
Loading
Loading