Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

My zones input becomes invalid on GCP Batch for Cromwell v88 #7708

Closed
yihming opened this issue Mar 12, 2025 · 1 comment
Closed

My zones input becomes invalid on GCP Batch for Cromwell v88 #7708

yihming opened this issue Mar 12, 2025 · 1 comment

Comments

@yihming
Copy link

yihming commented Mar 12, 2025

Hello,

In Cromwell v87, my workflow has the following default values for zones runtime attribute:

zones: "us-central1-a us-central1-b us-central1-c us-central1-f us-east1-b us-east1-c us-east1-d us-west1-a us-west1-b us-west1-c"

And when submitted to my GCP Batch which is deployed in us-west1, it worked smoothly with zones overwritten by the Batch configuration.

However, in Cromwell v88, the same job triggers the following error at server side:

2025-03-11 22:41:49 cromwell-system-akka.dispatchers.engine-dispatcher-29 INFO  - WorkflowManagerActor: Workflow 4257dce0-247f-45c3-9961-5f1db9bfb073 failed (during ExecutingWorkflowState): cromwell.backend.google.batch.api.BatchApiRequestManager$UserBatchApiException: Unable to complete Batch request due to a problem with the request (io.grpc.StatusRuntimeException: INVALID_ARGUMENT: job.allocation_policy field is invalid. all specified locations end up in more than one regions [us-central1 us-east1 us-west1].).

I was able to go around it by explicitly specifying zones: "us-west1-a us-west1-b us-west1-c" in my input JSON file. However, I'm still wondering if there is any way to fix it so that it works as usual.

Or is it true that since Cromwell v88, multi-regon zones value would always be considered invalid?

@jgainerdewar
Copy link
Collaborator

This is due to a limitation of GCP Batch - when specifying a location for the job to run in, we can only request a region or a list of zones in a single region.

https://cloud.google.com/batch/docs/reference/rest/v1/projects.locations.jobs#LocationPolicy.FIELDS.allowed_locations

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants