chore(deployment): reduce cpu requests to realistic requirements allowing more previews #3715
Conversation
…wing more previews
There was a problem hiding this comment.
We're currently limited in Loculus preview deployments because of having too much CPU requests. The requests are way higher than actual utilization, which means we could actually have more previews without any issues - if we simply reduced overly high CPU requests.
This wasn't the case earlier today when I had to remove previews - my preview was running but various components dying due to OOM issues. I'm not saying that's always true - I presume you are seeing an issue that has prompted this - but presumably we're relatively balanced
There was a problem hiding this comment.
Hmm, there were a bunch of events saying "failed scheduling due to insufficient CPU" throughout the day and we were above 93% for CPU utilization earlier
There was a problem hiding this comment.
Sure (but in my specific case it was memory that was crashing it - I could see it in argoCD, so both were likely limiting)
| @@ -1763,6 +1763,5 @@ defaultResources: | |||
| cpu: "20m" | |||
| limits: | |||
| memory: "1Gi" | |||
| cpu: "20m" | |||
There was a problem hiding this comment.
huh - this isn't allowed - seems like a bug in the schema - but your original 1000m seems good for now
There was a problem hiding this comment.
You have to do both or neithe?
There was a problem hiding this comment.
Schema complains:
Is that a correct complaint or a false positive complaint?
There was a problem hiding this comment.
seems like a bug in the schema
We're currently limited in Loculus preview deployments because of having too much CPU requests. The requests are way higher than actual utilization, which means we could actually have more previews without any issues - if we simply reduced overly high CPU requests.
This PR does that - in particular ingest and databases asked for more CPU than necessary.