Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

daemons, ui, webui: add policy package support #236

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

rdimaio
Copy link
Contributor

@rdimaio rdimaio commented Feb 24, 2025

For the daemons, test rendering to ensure the include works:

      initContainers:
      - name: install-policy-packages
        # A slim Python image could be used instead, but using the same image as the base image for the main container ensures that the same Python version is used
        image: almalinux:9
        command:
          - /bin/bash
          - -e
          - -c
          - |
            export PYTHONPATH=/opt/policy_packages/:$PYTHONPATH
            if python3 -c "from importlib.metadata import version; assert version('vo_1_policy_package') == '1.4.0'"; then
                echo "module vo_1_policy_package version 1.4.0 is already installed"
            else
                if [[ 'vo_1_policy_package==1.4.0' == git+* ]]; then
                    dnf install --assumeyes git-all
                fi
                dnf install --assumeyes python-pip
                pip install vo_1_policy_package==1.4.0 --target /opt/policy_packages/
            fi
            if python3 -c "from importlib.metadata import version; assert version('vo_2_policy_package') == '0.1.0'"; then
                echo "module vo_2_policy_package version 0.1.0 is already installed"
            else
                if [[ 'git+https://github.com/vo-2/[email protected]' == git+* ]]; then
                    dnf install --assumeyes git-all
                fi
                dnf install --assumeyes python-pip
                pip install git+https://github.com/vo-2/[email protected] --target /opt/policy_packages/
            fi
        volumeMounts:
        - name: policy-package-volume
          mountPath: /opt/policy_packages/
      containers:
        - name: rucio-daemons
          image: "rucio/rucio-daemons:release-34.2.0"
          imagePullPolicy: Always
          volumeMounts:
            - name: proxy-volume
              mountPath: /opt/proxy
            - name: ca-volume
              mountPath: /opt/certs
            - name: config-common
              mountPath: /opt/rucio/etc/conf.d/10_common.json
              subPath: common.json
            - name: config-component
              mountPath: /opt/rucio/etc/conf.d/20_component.json
              subPath: component.json
            - name: policy-package-volume
              mountPath: /opt/policy_packages/
          ports:
            - name: metrics
              containerPort: 8080
              protocol: TCP
          env:
            - name: RUCIO_OVERRIDE_CONFIGS
              value: "/opt/rucio/etc/conf.d/"
            - name: RUCIO_DAEMON
              value: "abacus-account"
            - name: RUCIO_DAEMON_ARGS
              value: "--threads 1"
            - name: PYTHONPATH
              value: /opt/policy_packages/:${PYTHONPATH}
          resources:
            limits:
              cpu: 700m
              memory: 200Mi
            requests:
              cpu: 700m

@rdimaio rdimaio requested a review from dchristidis February 24, 2025 18:01
@rdimaio rdimaio marked this pull request as draft February 24, 2025 18:04
@rdimaio rdimaio marked this pull request as ready for review February 24, 2025 18:11
@rdimaio
Copy link
Contributor Author

rdimaio commented Feb 25, 2025

Latest revision: bumped chart versions after #233 was merged

@rdimaio rdimaio linked an issue Feb 26, 2025 that may be closed by this pull request
@rdimaio
Copy link
Contributor Author

rdimaio commented Mar 14, 2025

Latest revision: rebased from master

@rdimaio rdimaio marked this pull request as draft March 14, 2025 14:25
@rdimaio
Copy link
Contributor Author

rdimaio commented Mar 14, 2025

Latest revision: refactored PVC logic so that it functions in the same way as server, as done in this PR #241

@rdimaio
Copy link
Contributor Author

rdimaio commented Mar 14, 2025

Test rendering of daemons via helm template test-release charts/rucio-server -f charts/rucio-server/values.yaml --debug :

---
# Source: rucio-daemons/templates/policy-package-storage.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-release-policy-package-volume
spec:
  storageClassName: 
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 100Mi
---
...
# Source: rucio-daemons/templates/abacus-deployment.yaml
...
    spec:
      volumes:
      - name: config-common
        secret:
          secretName: test-release-rucio-daemons.config.common
      - name: config-component
        secret:
          secretName: test-release-rucio-daemons.config.abacus-account
      - name: policy-package-volume
        persistentVolumeClaim:
          claimName: test-release-policy-package-volume
      - name: proxy-volume
        secret:
          secretName: test-release-rucio-x509up
      - name: ca-volume
        secret:
          secretName: test-release-rucio-ca-bundle
            
      initContainers:
      - name: install-policy-packages
        # A slim Python image could be used instead, but using the same image as the base image for the main container ensures that the same Python version is used
        image: almalinux:9
        command:
          - /bin/bash
          - -e
          - -c
          - |
            export PYTHONPATH=/opt/policy_packages/:$PYTHONPATH
            if python3 -c "from importlib.metadata import version; assert version('vo_1_policy_package') == '1.4.0'"; then
                echo "module vo_1_policy_package version 1.4.0 is already installed"
            else
                if [[ 'vo_1_policy_package==1.4.0' == git+* ]]; then
                    dnf install --assumeyes git-all
                fi
                dnf install --assumeyes python-pip
                pip install vo_1_policy_package==1.4.0 --target /opt/policy_packages/
            fi
            if python3 -c "from importlib.metadata import version; assert version('vo_2_policy_package') == '0.1.0'"; then
                echo "module vo_2_policy_package version 0.1.0 is already installed"
            else
                if [[ 'git+https://github.com/vo-2/[email protected]' == git+* ]]; then
                    dnf install --assumeyes git-all
                fi
                dnf install --assumeyes python-pip
                pip install git+https://github.com/vo-2/[email protected] --target /opt/policy_packages/
            fi
        volumeMounts:
        - name: policy-package-volume
          mountPath: /opt/policy_packages/

@rdimaio rdimaio marked this pull request as ready for review March 14, 2025 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Policy package deployment: using an init container
1 participant