Azure Monitor pods memory consumption issue

[brgrz]

Has anyone been notified of the AKS / Azure Monitor pod memory consumption issue which has been opened for months on the AKS repo?

Cross referencing this here:
Azure/AKS#3443

Any work being done on this? kube-system namespace pods taking up more than half of memory footprint in clusters where Azure Monitor/Container Insights are enabled.

[ganga1980]

Hi, @brgrz - We have done resource optimizations and @pfrcks working on rolling out the changes.

[andrewdinunzio]

Do the resource optimizations apply for the ama-logs' ama-logs-prometheus container's CPU usage? We've noticed consistent CPU throttling on that.

[github-actions]

This issue is stale because it has been open 7 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This issue was closed because it has been stalled for 12 days with no activity.

[akakira]

We have also found this to be an issue. The memory "Limits" to the monitoring pod are set like so:
cpu: 5
memory: 8Gi
Requests:
cpu: 10m
memory: 50Mi
The monitor consistently runs over 1gb.
kubectl top pods -n kube-system --sort-by memory
NAME CPU(cores) MEMORY(bytes)
ama-metrics-operator-targets-cdf598c95-q8kgm 9m 1188Mi
ama-metrics-654dc7bdcc-mxjpl 24m 316Mi
ama-logs-xtx9t 28m 309Mi
ama-metrics-node-grd44 19m 288Mi
ama-logs-rktd7 24m 286Mi
ama-metrics-node-d9k5x 23m 285Mi
ama-metrics-node-4khgz 20m 275Mi
ama-logs-rqtlk 23m 273Mi
ama-logs-rs-86466dbfb8-t8cdz 20m 232Mi