Skip to content

Commit

Permalink
Allow list syntax for exchangeManager baseDir targets
Browse files Browse the repository at this point in the history
This change will allow using list syntax of exchangeManager base-directories which is more common and understandable with Helm charts.

Trino allows specifying multiple spooling targets for exchageManager separated by commas when using main cloud providers or any S3 compatible protocol (example: s3://bucket1,s3://bucket2,s3://bucket3). With current trinodb/chart we have to specify them within values as a single line for baseDir:

server:
  exchangeManager:
    name: "filesystem"
    baseDir: "s3://exchange-spooling-bucket-1,s3://exchange-spooling-bucket-2"

Which translates to:

  exchange-manager.properties: |
    exchange-manager.name=filesystem
    exchange.base-directories=s3://exchange-spooling-bucket-1,s3://exchange-spooling-bucket-2

This change allows a more clean listing within values:

server:
  exchangeManager:
    name: "filesystem"
    baseDir:
    - "s3://exchange-spooling-bucket-1"
    - "s3://exchange-spooling-bucket-2"

Which translates to the same:

  exchange-manager.properties: |
    exchange-manager.name=filesystem
    exchange.base-directories=s3://exchange-spooling-bucket-1,s3://exchange-spooling-bucket-2

It will not break any existing listings as stating them the same way as before in a single line will result in exchange.base-directories within a single line.
Documentation for setting up fault tolerant execution: https://trino.io/docs/current/admin/fault-tolerant-execution.html

Chart documentation has been also updated. Default usage example has been changed to a listed view with a minor clean up of the overall description.
  • Loading branch information
laimis9133 authored and nineinchnick committed Jan 28, 2025
1 parent bcf5409 commit 1a9cb3b
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 6 deletions.
2 changes: 1 addition & 1 deletion charts/trino/templates/configmap-coordinator.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ data:
{{- if .Values.server.exchangeManager }}
exchange-manager.properties: |
exchange-manager.name={{ .Values.server.exchangeManager.name }}
exchange.base-directories={{ .Values.server.exchangeManager.baseDir }}
exchange.base-directories={{ join "," .Values.server.exchangeManager.baseDir }}
{{- range $configValue := .Values.additionalExchangeManagerProperties }}
{{ $configValue }}
{{- end }}
Expand Down
2 changes: 1 addition & 1 deletion charts/trino/templates/configmap-worker.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ data:
{{- if .Values.server.exchangeManager }}
exchange-manager.properties: |
exchange-manager.name={{ .Values.server.exchangeManager.name }}
exchange.base-directories={{ .Values.server.exchangeManager.baseDir }}
exchange.base-directories={{ join "," .Values.server.exchangeManager.baseDir }}
{{- range $configValue := .Values.additionalExchangeManagerProperties }}
{{ $configValue }}
{{- end }}
Expand Down
9 changes: 5 additions & 4 deletions charts/trino/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -62,16 +62,17 @@ server:
exchangeManager: {}
# server.exchangeManager -- Mandatory [exchange manager
# configuration](https://trino.io/docs/current/admin/fault-tolerant-execution.html#id1).
# Used to set the name and location(s) of the spooling storage destination. To enable fault-tolerant execution,
# set the `retry-policy` property in `additionalConfigProperties`. Additional exchange manager configurations can be
# added to `additionalExchangeManagerProperties`.
# Used to set the name and location(s) of spooling data storage. For multiple destinations use a list or a comma separated URI locations.

Check failure on line 65 in charts/trino/values.yaml

View workflow job for this annotation

GitHub Actions / lint

65:140 [trailing-spaces] trailing spaces
# To enable fault-tolerant execution, set the `retry-policy` property in `additionalConfigProperties`.

Check failure on line 66 in charts/trino/values.yaml

View workflow job for this annotation

GitHub Actions / lint

66:105 [trailing-spaces] trailing spaces
# Additional exchange manager configurations can be added to `additionalExchangeManagerProperties`.
# @raw
# Example:
# ```yaml
# server:
# exchangeManager:
# name: "filesystem"
# baseDir: "/tmp/trino-local-file-system-exchange-manager"
# baseDir:
# - "/tmp/trino-local-file-system-exchange-manager"
# additionalConfigProperties:
# - retry-policy=TASK
# additionalExchangeManagerProperties:
Expand Down

0 comments on commit 1a9cb3b

Please sign in to comment.