argocd progressing state "forever" #5620
Comments
|
I just rolled out the same to a new cluster in aws.. with autosync disabled.. and it is spinning on sync status 'unknown'.. and spinning is "confusing".. is it doing something or not.. and it actually shows an error.. will try and see if it behaves better here after I press sync. |
|
as soon as I fixed the error (missing helm repo in argocd repos) - it switched to Missing and unsynced as it should. |
|
I can see kind of the same behavior with |
|
my environment is manual sync. But occasionally there will be there is no problem with kustomize configuration. But APP HEALTH is always Progressing |
|
Seeing a very simple Kibana app stuck in 'Progressing' on ArgoCD v2.1.3+d855831. |
|
I'm having the same problem, is there a way to get this stuttering status through the api? |
|
facing the same issue with github runner , everything inside the cluster seems running & health , argocd ui also says healthy & syced but pod status keeps on saying progressing , |
|
i have same issue is there any solution? |
|
Same here, a solution would be great. |
|
Same here. |
|
Same here, with Kiali Operator and CR, with latest version of argocd v2.5.2+148d8da. |
1 similar comment
|
Same here, with Kiali Operator and CR, with latest version of argocd v2.5.2+148d8da. |
|
fwiw I am seeing the same issue with a Prometheus setup with v2.5.4 |
|
This is also happening on the live demo site :) https://cd.apps.argoproj.io/applications/prometheus-operator?operation=false&resource=kind%3APrometheus |
|
Shows from the The status of the resource in the |
|
Same issue here with Prometheus Operator v0.52.0 using ArgoCD v2.5.4+86b2dde Edit: couple minutes after sync, the state became |
|
In our case we were using kube-prometheus-stack helm chart version 30.2.0 where status sub-resource is not enabled in Prometheus CRD Argocd uses status section to check whether the resource is healthy of Prometheus custom resource Upgrading kube-prometheus-stack helm chart to version 43.2.1 where the status sub-resource is enabled has resolved this issue |
|
I've seen the same issue and fixed it that way: #11782. |
|
the same for me in argo v2.4.10+2ccc17a |
|
I'm also stuck with this, for a very simple dummy application. Argo CD version: 2.8.0. Here's the app object: Progressing application
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"argoproj.io/v1alpha1","kind":"Application","metadata":{"annotations":{},"finalizers":["resources-finalizer.argocd.argoproj.io"],"name":"app","namespace":"foo"},"spec":{"destination":{"name":"my-cluster","namespace":"foo"},"project":"foo","source":{"path":"manifests/app","repoURL":"[email protected]:foo/bar.git","targetRevision":"main"},"syncPolicy":{"automated":{"prune":true,"selfHeal":true}}}}
creationTimestamp: "2023-08-30T13:50:14Z"
finalizers:
- resources-finalizer.argocd.argoproj.io
generation: 412
name: app
namespace: foo
resourceVersion: "61647168"
uid: 4cdab6b5-926a-4c87-baec-a9623c2e8928
spec:
destination:
name: my-cluster
namespace: foo
project: foo
source:
path: manifests/app
repoURL: [email protected]:foo/bar.git
targetRevision: main
syncPolicy:
automated:
prune: true
selfHeal: true
status:
controllerNamespace: argocd
health:
status: Progressing
history:
- deployStartedAt: "2023-08-30T16:35:17Z"
deployedAt: "2023-08-30T16:35:17Z"
id: 0
revision: 3f2e52c5ae1d9fa94d470d915e2556c94f3bff94
source:
path: manifests/app
repoURL: [email protected]:foo/bar.git
targetRevision: main
operationState:
finishedAt: "2023-08-30T16:35:17Z"
message: successfully synced (all tasks run)
operation:
initiatedBy:
automated: true
retry:
limit: 5
sync:
prune: true
revision: 3f2e52c5ae1d9fa94d470d915e2556c94f3bff94
phase: Succeeded
startedAt: "2023-08-30T16:35:17Z"
syncResult:
resources:
- group: ""
hookPhase: Running
kind: PersistentVolumeClaim
message: persistentvolumeclaim/dbdisk-pvc configured
name: dbdisk-pvc
namespace: foo
status: Synced
syncPhase: Sync
version: v1
- group: ""
hookPhase: Running
kind: Service
message: service/mariadb configured
name: mariadb
namespace: foo
status: Synced
syncPhase: Sync
version: v1
- group: ""
hookPhase: Running
kind: Service
message: service/webserver configured
name: webserver
namespace: foo
status: Synced
syncPhase: Sync
version: v1
- group: ""
hookPhase: Running
kind: Pod
message: pod/dbadmin configured
name: dbadmin
namespace: foo
status: Synced
syncPhase: Sync
version: v1
- group: ""
hookPhase: Running
kind: Pod
message: pod/webserver configured
name: webserver
namespace: foo
status: Synced
syncPhase: Sync
version: v1
- group: ""
hookPhase: Running
kind: Pod
message: pod/db1 configured
name: db1
namespace: foo
status: Synced
syncPhase: Sync
version: v1
- group: networking.k8s.io
hookPhase: Running
kind: Ingress
message: ingress.networking.k8s.io/foo configured
name: foo
namespace: foo
status: Synced
syncPhase: Sync
version: v1
revision: 3f2e52c5ae1d9fa94d470d915e2556c94f3bff94
source:
path: manifests/app
repoURL: [email protected]:foo/bar.git
targetRevision: main
reconciledAt: "2023-08-31T09:20:34Z"
resources:
- health:
status: Healthy
kind: PersistentVolumeClaim
name: dbdisk-pvc
namespace: foo
status: Synced
version: v1
- health:
status: Progressing
kind: Pod
name: db1
namespace: foo
status: Synced
version: v1
- health:
status: Healthy
kind: Pod
name: dbadmin
namespace: foo
status: Synced
version: v1
- health:
status: Healthy
kind: Pod
name: webserver
namespace: foo
status: Synced
version: v1
- health:
status: Healthy
kind: Service
name: mariadb
namespace: foo
status: Synced
version: v1
- health:
status: Healthy
kind: Service
name: webserver
namespace: foo
status: Synced
version: v1
- group: networking.k8s.io
health:
status: Healthy
kind: Ingress
name: foo
namespace: foo
status: Synced
version: v1
sourceType: Directory
summary:
externalURLs:
- https://foo.example.com/
images:
- gillos/pytest1
- mariadb
- phpmyadmin
sync:
comparedTo:
destination:
name: my-cluster
namespace: foo
source:
path: manifests/app
repoURL: [email protected]:foo/bar.git
targetRevision: main
revision: d83d871617efcb68f499ee5abe7763da0c9758bd
status: SyncedHere's the pod: Pod
apiVersion: v1
kind: Pod
metadata:
annotations:
argocd.argoproj.io/tracking-id: foo_app:/Pod:foo/db1
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{"argocd.argoproj.io/tracking-id":"foo_app:/Pod:foo/db1"},"creationTimestamp":null,"labels":{"app":"mariadb"},"name":"db1","namespace":"foo"},"spec":{"containers":[{"env":[{"name":"MYSQL_ROOT_PASSWORD","value":"hej123"}],"image":"mariadb","name":"db1","resources":{},"volumeMounts":[{"mountPath":"/var/lib/mysql","name":"storage-volume"}]}],"dnsPolicy":"ClusterFirst","restartPolicy":"Never","volumes":[{"name":"storage-volume","persistentVolumeClaim":{"claimName":"dbdisk-pvc"}}]}}
creationTimestamp: "2023-08-30T14:19:50Z"
labels:
app: mariadb
name: db1
namespace: foo
resourceVersion: "61172805"
uid: 6d09c766-fab5-4a78-a0b5-cc8c134de418
spec:
containers:
image: mariadb
imagePullPolicy: Always
name: db1
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/lib/mysql
name: storage-volume
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-d9zp7
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: rancher-node026
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Never
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- name: storage-volume
persistentVolumeClaim:
claimName: dbdisk-pvc
- name: kube-api-access-d9zp7
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2023-08-30T14:19:50Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2023-08-30T14:20:13Z"
status: "True"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2023-08-30T14:20:13Z"
status: "True"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2023-08-30T14:19:50Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: containerd://4de82d4577345858cc851877dbd9db050aa944d372ea1e181eecb65dc81fc795
image: docker.io/library/mariadb:latest
imageID: docker.io/library/mariadb@sha256:a104070983c2a9ab542d6142de858457dd15d2cabd7ac26e4ca3891d7721e73e
lastState: {}
name: db1
ready: true
restartCount: 0
started: true
state:
running:
startedAt: "2023-08-30T14:20:13Z"
hostIP: 93.184.216.34
phase: Running
podIP: 10.42.10.70
podIPs:
- ip: 10.42.10.70
qosClass: BestEffort
startTime: "2023-08-30T14:19:50Z"But I see nothing wrong with the pod in question. Is there anything I can probe in order to debug this further?
I've opened a new issue for my very particular case: #15317 |
|
For those that installed Traefik via Helm, you can update your values.yml:
Add below:
Here is what should look like: Then upgrade the helm (this command might be different for you)
App Health should become healthy within seconds. |
If you are trying to resolve an environment-specific issue or have a one-off question about the edge case that does not require a feature then please consider asking a question in argocd slack channel.
Checklist:
argocd version.Describe the bug
After having installed kube-prometheus-stack helm chart via argocd (on a small microk8s cluster) - it runs perfectly fine.
I had first set wrong storageclass name - so pvc was hanging.. I fixed that and had to manually delete the hanging PVC (I thought argocd would have fixed that? or atleast marked it as an issue.. I didn't see any issue about that in argocd UI.
All instances are now running and in k9s I see the pods fine and all healthy and responding.
argocd UI says "Synced" - but Health state check keeps hanging for hours (has not finished yet) in "progressing"..
Where do I see details on this? I see no errors in logs from argocd-server nor from application-controller.. Shouldn't argocd react on a progress check hanging/spinning for so long? or maybe its a UI problem showing wrong status? (reloading does not change status though).
To Reproduce
install kube-prometheus-stack chart - v13.13.0 with promtheus pvc enabled and wrong classname - and then correct that classname once installation fails.. (I guess)
Expected behavior
ArgoCD should have showed some sort of failure on the "health check" - and details on whats up.. instead of never timing out.
Screenshots
if you want a screenshot of the Health spinnner - say so. :)
The other applications I have in same argocd instance are all showing Green health just fine and argocd runs fine it seems.
Version
Logs
Nothing seems to be relevant.. I see no errors.
The text was updated successfully, but these errors were encountered: