[ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path #211822
+590
−61
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jennypavlova
added
the
Team:obs-ux-infra_services
jennypavlova
force-pushed
the
211783-obsux-apm-otel-runtime-metrics-show-dashboards-with-different-ingest-path
branch
from
91f169e to
12f364d
Compare
jennypavlova
force-pushed
the
211783-obsux-apm-otel-runtime-metrics-show-dashboards-with-different-ingest-path
branch
from
87776b4 to
e59ed91
Compare
…ashboards-with-different-ingest-path
|
Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services) |
|
/ci |
src/platform/packages/shared/kbn-elastic-agent-utils/src/agent_guards.ts
Outdated
Show resolved
Hide resolved
…ashboards-with-different-ingest-path
…_guards.ts Co-authored-by: Cauê Marcondes <[email protected]>
jennypavlova
added
release_note:enhancement
backport:prev-minor
AlexanderWert
approved these changes
Mar 5, 2025
…ashboards-with-different-ingest-path
…ashboards-with-different-ingest-path
|
Starting backport for target branches: 8.16, 8.17, 8.18, 8.x, 9.0 https://github.com/elastic/kibana/actions/runs/13699127640 |
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]Module Count
Public APIs missing comments
Async chunks
Unknown metric groupsAPI count
async chunk count
History
|
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Mar 6, 2025
…ngest path (elastic#211822) Closes elastic#211783 Part of elastic#195857 ## Summary This PR expands the logic to get the dashboard files based on the agent. We have many different ways to ingest data so we want to add more metrics dashboards to the APM metrics tab. The different ingest paths we have: Classic APM Agent --> APM Server --> ES Vanilla OTel SDKs --> APM Server --> ES EDOT OTel SDKs --> APM Server --> ES Classic APM Agent --> EDOT Collector --> ES Vanilla OTel SDKs. --> EDOT Collector --> ES EDOT OTel SDKs --> EDOT Collector --> ES We agreed on having a dashboard filename pattern to make showing the correct dashboard easier described [here](elastic#195857 (comment)) First, we determine if the ingest path is through APM Server or EDOT Collector by checking the `telemtry.sdk` fields. ## TODOs / Reviewer notes - [ ] Currently, we have a fallback to metrics charts which is valid only if we have APM agent so this PR adds an empty state message: "Runtime metrics are not available for this Agent / SDK type." in case there is no dashboard for the service language. To be improved in elastic#211774 and will be updated in this PR when ready - I will still open it for review as the other logic can be reviewed - The dashboards are to be updated (by the agent team so not part of the changes here) ## Testing: - Using e2e PoC - The available dashboard cases can be found in [loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40) - Cases to be checked: - OTel native with Vanilla OTel SDKs with available dashboard (example case file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`) <img width="1903" alt="image" src="https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb" /> - APM server with Vanilla OTel SDKs service with available dashboard (example case file: `classic_apm-otel_other-nodejs`, `...-java`, `...-dotnet`)  - APM server with Classic APM Agent (example case file: `classic_apm-apm-nodejs`, `...-java`) <img width="962" alt="image" src="https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e" /> - OTel native with Vanilla OTel SDKs without available dashboard (empty state case example: python service)  - APM server with Vanilla OTel SDKs service without available dashboard (empty state) <img width="1910" alt="image" src="https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281" /> - APM server with Classic APM Agent without available dashboard (Current metrics fallback) <img width="1914" alt="image" src="https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75" /> -⚠️ OTel native Dashboards are still not available (at the time of adding the description) --------- Co-authored-by: Sergi Romeu <[email protected]> Co-authored-by: Cauê Marcondes <[email protected]> Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit f195570)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Mar 6, 2025
…ngest path (elastic#211822) Closes elastic#211783 Part of elastic#195857 ## Summary This PR expands the logic to get the dashboard files based on the agent. We have many different ways to ingest data so we want to add more metrics dashboards to the APM metrics tab. The different ingest paths we have: Classic APM Agent --> APM Server --> ES Vanilla OTel SDKs --> APM Server --> ES EDOT OTel SDKs --> APM Server --> ES Classic APM Agent --> EDOT Collector --> ES Vanilla OTel SDKs. --> EDOT Collector --> ES EDOT OTel SDKs --> EDOT Collector --> ES We agreed on having a dashboard filename pattern to make showing the correct dashboard easier described [here](elastic#195857 (comment)) First, we determine if the ingest path is through APM Server or EDOT Collector by checking the `telemtry.sdk` fields. ## TODOs / Reviewer notes - [ ] Currently, we have a fallback to metrics charts which is valid only if we have APM agent so this PR adds an empty state message: "Runtime metrics are not available for this Agent / SDK type." in case there is no dashboard for the service language. To be improved in elastic#211774 and will be updated in this PR when ready - I will still open it for review as the other logic can be reviewed - The dashboards are to be updated (by the agent team so not part of the changes here) ## Testing: - Using e2e PoC - The available dashboard cases can be found in [loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40) - Cases to be checked: - OTel native with Vanilla OTel SDKs with available dashboard (example case file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`) <img width="1903" alt="image" src="https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb" /> - APM server with Vanilla OTel SDKs service with available dashboard (example case file: `classic_apm-otel_other-nodejs`, `...-java`, `...-dotnet`)  - APM server with Classic APM Agent (example case file: `classic_apm-apm-nodejs`, `...-java`) <img width="962" alt="image" src="https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e" /> - OTel native with Vanilla OTel SDKs without available dashboard (empty state case example: python service)  - APM server with Vanilla OTel SDKs service without available dashboard (empty state) <img width="1910" alt="image" src="https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281" /> - APM server with Classic APM Agent without available dashboard (Current metrics fallback) <img width="1914" alt="image" src="https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75" /> -⚠️ OTel native Dashboards are still not available (at the time of adding the description) --------- Co-authored-by: Sergi Romeu <[email protected]> Co-authored-by: Cauê Marcondes <[email protected]> Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit f195570)
kibanamachine
added a commit
that referenced
this pull request
Mar 6, 2025
…rent ingest path (#211822) (#213373) # Backport This will backport the following commits from `main` to `8.x`: - [[ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path (#211822)](#211822) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"jennypavlova","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-03-06T12:52:44Z","message":"[ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path (#211822)\n\nCloses #211783\nPart of https://github.com/elastic/kibana/issues/195857\n\n## Summary\n\nThis PR expands the logic to get the dashboard files based on the agent.\nWe have many different ways to ingest data so we want to add more\nmetrics dashboards to the APM metrics tab. The different ingest paths we\nhave:\nClassic APM Agent --> APM Server --> ES\nVanilla OTel SDKs --> APM Server --> ES\nEDOT OTel SDKs --> APM Server --> ES\nClassic APM Agent --> EDOT Collector --> ES\nVanilla OTel SDKs. --> EDOT Collector --> ES\nEDOT OTel SDKs --> EDOT Collector --> ES\nWe agreed on having a dashboard filename pattern to make showing the\ncorrect dashboard easier described\n[here](https://github.com/elastic/kibana/issues/195857#issue-2580733648)\n\nFirst, we determine if the ingest path is through APM Server or EDOT\nCollector by checking the `telemtry.sdk` fields.\n\n## TODOs / Reviewer notes\n- [ ] Currently, we have a fallback to metrics charts which is valid\nonly if we have APM agent so this PR adds an empty state message:\n\"Runtime metrics are not available for this Agent / SDK type.\" in case\nthere is no dashboard for the service language. To be improved in\nhttps://github.com//issues/211774 and will be updated in\nthis PR when ready - I will still open it for review as the other logic\ncan be reviewed\n- The dashboards are to be updated (by the agent team so not part of the\nchanges here)\n\n## Testing: \n- Using e2e PoC \n- The available dashboard cases can be found in\n[loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40)\n- Cases to be checked:\n- OTel native with Vanilla OTel SDKs with available dashboard (example\ncase file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`)\n \n<img width=\"1903\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb\"\n/>\n\n- APM server with Vanilla OTel SDKs service with available dashboard\n(example case file: `classic_apm-otel_other-nodejs`, `...-java`,\n`...-dotnet`)\n \n\n\n\n- APM server with Classic APM Agent (example case file:\n`classic_apm-apm-nodejs`, `...-java`)\n \n<img width=\"962\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e\"\n/>\n\n- OTel native with Vanilla OTel SDKs without available dashboard (empty\nstate case example: python service)\n \n\n\n\n- APM server with Vanilla OTel SDKs service without available dashboard\n(empty state)\n \n<img width=\"1910\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281\"\n/>\n\n- APM server with Classic APM Agent without available dashboard (Current\nmetrics fallback)\n \n<img width=\"1914\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75\"\n/>\n\n-⚠️ OTel native Dashboards are still not available (at the time of\nadding the description)\n\n---------\n\nCo-authored-by: Sergi Romeu <[email protected]>\nCo-authored-by: Cauê Marcondes <[email protected]>\nCo-authored-by: kibanamachine <[email protected]>","sha":"f19557004904b490fcb69d18c1ea5f1ef9634ee0","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","backport:prev-minor","backport:prev-major","Team:obs-ux-infra_services","v9.1.0"],"title":"[ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path","number":211822,"url":"https://github.com/elastic/kibana/pull/211822","mergeCommit":{"message":"[ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path (#211822)\n\nCloses #211783\nPart of https://github.com/elastic/kibana/issues/195857\n\n## Summary\n\nThis PR expands the logic to get the dashboard files based on the agent.\nWe have many different ways to ingest data so we want to add more\nmetrics dashboards to the APM metrics tab. The different ingest paths we\nhave:\nClassic APM Agent --> APM Server --> ES\nVanilla OTel SDKs --> APM Server --> ES\nEDOT OTel SDKs --> APM Server --> ES\nClassic APM Agent --> EDOT Collector --> ES\nVanilla OTel SDKs. --> EDOT Collector --> ES\nEDOT OTel SDKs --> EDOT Collector --> ES\nWe agreed on having a dashboard filename pattern to make showing the\ncorrect dashboard easier described\n[here](https://github.com/elastic/kibana/issues/195857#issue-2580733648)\n\nFirst, we determine if the ingest path is through APM Server or EDOT\nCollector by checking the `telemtry.sdk` fields.\n\n## TODOs / Reviewer notes\n- [ ] Currently, we have a fallback to metrics charts which is valid\nonly if we have APM agent so this PR adds an empty state message:\n\"Runtime metrics are not available for this Agent / SDK type.\" in case\nthere is no dashboard for the service language. To be improved in\nhttps://github.com//issues/211774 and will be updated in\nthis PR when ready - I will still open it for review as the other logic\ncan be reviewed\n- The dashboards are to be updated (by the agent team so not part of the\nchanges here)\n\n## Testing: \n- Using e2e PoC \n- The available dashboard cases can be found in\n[loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40)\n- Cases to be checked:\n- OTel native with Vanilla OTel SDKs with available dashboard (example\ncase file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`)\n \n<img width=\"1903\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb\"\n/>\n\n- APM server with Vanilla OTel SDKs service with available dashboard\n(example case file: `classic_apm-otel_other-nodejs`, `...-java`,\n`...-dotnet`)\n \n\n\n\n- APM server with Classic APM Agent (example case file:\n`classic_apm-apm-nodejs`, `...-java`)\n \n<img width=\"962\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e\"\n/>\n\n- OTel native with Vanilla OTel SDKs without available dashboard (empty\nstate case example: python service)\n \n\n\n\n- APM server with Vanilla OTel SDKs service without available dashboard\n(empty state)\n \n<img width=\"1910\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281\"\n/>\n\n- APM server with Classic APM Agent without available dashboard (Current\nmetrics fallback)\n \n<img width=\"1914\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75\"\n/>\n\n-⚠️ OTel native Dashboards are still not available (at the time of\nadding the description)\n\n---------\n\nCo-authored-by: Sergi Romeu <[email protected]>\nCo-authored-by: Cauê Marcondes <[email protected]>\nCo-authored-by: kibanamachine <[email protected]>","sha":"f19557004904b490fcb69d18c1ea5f1ef9634ee0"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211822","number":211822,"mergeCommit":{"message":"[ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path (#211822)\n\nCloses #211783\nPart of https://github.com/elastic/kibana/issues/195857\n\n## Summary\n\nThis PR expands the logic to get the dashboard files based on the agent.\nWe have many different ways to ingest data so we want to add more\nmetrics dashboards to the APM metrics tab. The different ingest paths we\nhave:\nClassic APM Agent --> APM Server --> ES\nVanilla OTel SDKs --> APM Server --> ES\nEDOT OTel SDKs --> APM Server --> ES\nClassic APM Agent --> EDOT Collector --> ES\nVanilla OTel SDKs. --> EDOT Collector --> ES\nEDOT OTel SDKs --> EDOT Collector --> ES\nWe agreed on having a dashboard filename pattern to make showing the\ncorrect dashboard easier described\n[here](https://github.com/elastic/kibana/issues/195857#issue-2580733648)\n\nFirst, we determine if the ingest path is through APM Server or EDOT\nCollector by checking the `telemtry.sdk` fields.\n\n## TODOs / Reviewer notes\n- [ ] Currently, we have a fallback to metrics charts which is valid\nonly if we have APM agent so this PR adds an empty state message:\n\"Runtime metrics are not available for this Agent / SDK type.\" in case\nthere is no dashboard for the service language. To be improved in\nhttps://github.com//issues/211774 and will be updated in\nthis PR when ready - I will still open it for review as the other logic\ncan be reviewed\n- The dashboards are to be updated (by the agent team so not part of the\nchanges here)\n\n## Testing: \n- Using e2e PoC \n- The available dashboard cases can be found in\n[loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40)\n- Cases to be checked:\n- OTel native with Vanilla OTel SDKs with available dashboard (example\ncase file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`)\n \n<img width=\"1903\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb\"\n/>\n\n- APM server with Vanilla OTel SDKs service with available dashboard\n(example case file: `classic_apm-otel_other-nodejs`, `...-java`,\n`...-dotnet`)\n \n\n\n\n- APM server with Classic APM Agent (example case file:\n`classic_apm-apm-nodejs`, `...-java`)\n \n<img width=\"962\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e\"\n/>\n\n- OTel native with Vanilla OTel SDKs without available dashboard (empty\nstate case example: python service)\n \n\n\n\n- APM server with Vanilla OTel SDKs service without available dashboard\n(empty state)\n \n<img width=\"1910\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281\"\n/>\n\n- APM server with Classic APM Agent without available dashboard (Current\nmetrics fallback)\n \n<img width=\"1914\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75\"\n/>\n\n-⚠️ OTel native Dashboards are still not available (at the time of\nadding the description)\n\n---------\n\nCo-authored-by: Sergi Romeu <[email protected]>\nCo-authored-by: Cauê Marcondes <[email protected]>\nCo-authored-by: kibanamachine <[email protected]>","sha":"f19557004904b490fcb69d18c1ea5f1ef9634ee0"}}]}] BACKPORT--> Co-authored-by: jennypavlova <[email protected]>
kibanamachine
added a commit
that referenced
this pull request
Mar 6, 2025
…rent ingest path (#211822) (#213374) # Backport This will backport the following commits from `main` to `9.0`: - [[ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path (#211822)](#211822) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"jennypavlova","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-03-06T12:52:44Z","message":"[ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path (#211822)\n\nCloses #211783\nPart of https://github.com/elastic/kibana/issues/195857\n\n## Summary\n\nThis PR expands the logic to get the dashboard files based on the agent.\nWe have many different ways to ingest data so we want to add more\nmetrics dashboards to the APM metrics tab. The different ingest paths we\nhave:\nClassic APM Agent --> APM Server --> ES\nVanilla OTel SDKs --> APM Server --> ES\nEDOT OTel SDKs --> APM Server --> ES\nClassic APM Agent --> EDOT Collector --> ES\nVanilla OTel SDKs. --> EDOT Collector --> ES\nEDOT OTel SDKs --> EDOT Collector --> ES\nWe agreed on having a dashboard filename pattern to make showing the\ncorrect dashboard easier described\n[here](https://github.com/elastic/kibana/issues/195857#issue-2580733648)\n\nFirst, we determine if the ingest path is through APM Server or EDOT\nCollector by checking the `telemtry.sdk` fields.\n\n## TODOs / Reviewer notes\n- [ ] Currently, we have a fallback to metrics charts which is valid\nonly if we have APM agent so this PR adds an empty state message:\n\"Runtime metrics are not available for this Agent / SDK type.\" in case\nthere is no dashboard for the service language. To be improved in\nhttps://github.com//issues/211774 and will be updated in\nthis PR when ready - I will still open it for review as the other logic\ncan be reviewed\n- The dashboards are to be updated (by the agent team so not part of the\nchanges here)\n\n## Testing: \n- Using e2e PoC \n- The available dashboard cases can be found in\n[loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40)\n- Cases to be checked:\n- OTel native with Vanilla OTel SDKs with available dashboard (example\ncase file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`)\n \n<img width=\"1903\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb\"\n/>\n\n- APM server with Vanilla OTel SDKs service with available dashboard\n(example case file: `classic_apm-otel_other-nodejs`, `...-java`,\n`...-dotnet`)\n \n\n\n\n- APM server with Classic APM Agent (example case file:\n`classic_apm-apm-nodejs`, `...-java`)\n \n<img width=\"962\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e\"\n/>\n\n- OTel native with Vanilla OTel SDKs without available dashboard (empty\nstate case example: python service)\n \n\n\n\n- APM server with Vanilla OTel SDKs service without available dashboard\n(empty state)\n \n<img width=\"1910\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281\"\n/>\n\n- APM server with Classic APM Agent without available dashboard (Current\nmetrics fallback)\n \n<img width=\"1914\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75\"\n/>\n\n-⚠️ OTel native Dashboards are still not available (at the time of\nadding the description)\n\n---------\n\nCo-authored-by: Sergi Romeu <[email protected]>\nCo-authored-by: Cauê Marcondes <[email protected]>\nCo-authored-by: kibanamachine <[email protected]>","sha":"f19557004904b490fcb69d18c1ea5f1ef9634ee0","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","backport:prev-minor","backport:prev-major","Team:obs-ux-infra_services","v9.1.0"],"title":"[ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path","number":211822,"url":"https://github.com/elastic/kibana/pull/211822","mergeCommit":{"message":"[ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path (#211822)\n\nCloses #211783\nPart of https://github.com/elastic/kibana/issues/195857\n\n## Summary\n\nThis PR expands the logic to get the dashboard files based on the agent.\nWe have many different ways to ingest data so we want to add more\nmetrics dashboards to the APM metrics tab. The different ingest paths we\nhave:\nClassic APM Agent --> APM Server --> ES\nVanilla OTel SDKs --> APM Server --> ES\nEDOT OTel SDKs --> APM Server --> ES\nClassic APM Agent --> EDOT Collector --> ES\nVanilla OTel SDKs. --> EDOT Collector --> ES\nEDOT OTel SDKs --> EDOT Collector --> ES\nWe agreed on having a dashboard filename pattern to make showing the\ncorrect dashboard easier described\n[here](https://github.com/elastic/kibana/issues/195857#issue-2580733648)\n\nFirst, we determine if the ingest path is through APM Server or EDOT\nCollector by checking the `telemtry.sdk` fields.\n\n## TODOs / Reviewer notes\n- [ ] Currently, we have a fallback to metrics charts which is valid\nonly if we have APM agent so this PR adds an empty state message:\n\"Runtime metrics are not available for this Agent / SDK type.\" in case\nthere is no dashboard for the service language. To be improved in\nhttps://github.com//issues/211774 and will be updated in\nthis PR when ready - I will still open it for review as the other logic\ncan be reviewed\n- The dashboards are to be updated (by the agent team so not part of the\nchanges here)\n\n## Testing: \n- Using e2e PoC \n- The available dashboard cases can be found in\n[loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40)\n- Cases to be checked:\n- OTel native with Vanilla OTel SDKs with available dashboard (example\ncase file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`)\n \n<img width=\"1903\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb\"\n/>\n\n- APM server with Vanilla OTel SDKs service with available dashboard\n(example case file: `classic_apm-otel_other-nodejs`, `...-java`,\n`...-dotnet`)\n \n\n\n\n- APM server with Classic APM Agent (example case file:\n`classic_apm-apm-nodejs`, `...-java`)\n \n<img width=\"962\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e\"\n/>\n\n- OTel native with Vanilla OTel SDKs without available dashboard (empty\nstate case example: python service)\n \n\n\n\n- APM server with Vanilla OTel SDKs service without available dashboard\n(empty state)\n \n<img width=\"1910\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281\"\n/>\n\n- APM server with Classic APM Agent without available dashboard (Current\nmetrics fallback)\n \n<img width=\"1914\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75\"\n/>\n\n-⚠️ OTel native Dashboards are still not available (at the time of\nadding the description)\n\n---------\n\nCo-authored-by: Sergi Romeu <[email protected]>\nCo-authored-by: Cauê Marcondes <[email protected]>\nCo-authored-by: kibanamachine <[email protected]>","sha":"f19557004904b490fcb69d18c1ea5f1ef9634ee0"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211822","number":211822,"mergeCommit":{"message":"[ObsUX] [APM] [OTel] Runtime metrics show dashboards with different ingest path (#211822)\n\nCloses #211783\nPart of https://github.com/elastic/kibana/issues/195857\n\n## Summary\n\nThis PR expands the logic to get the dashboard files based on the agent.\nWe have many different ways to ingest data so we want to add more\nmetrics dashboards to the APM metrics tab. The different ingest paths we\nhave:\nClassic APM Agent --> APM Server --> ES\nVanilla OTel SDKs --> APM Server --> ES\nEDOT OTel SDKs --> APM Server --> ES\nClassic APM Agent --> EDOT Collector --> ES\nVanilla OTel SDKs. --> EDOT Collector --> ES\nEDOT OTel SDKs --> EDOT Collector --> ES\nWe agreed on having a dashboard filename pattern to make showing the\ncorrect dashboard easier described\n[here](https://github.com/elastic/kibana/issues/195857#issue-2580733648)\n\nFirst, we determine if the ingest path is through APM Server or EDOT\nCollector by checking the `telemtry.sdk` fields.\n\n## TODOs / Reviewer notes\n- [ ] Currently, we have a fallback to metrics charts which is valid\nonly if we have APM agent so this PR adds an empty state message:\n\"Runtime metrics are not available for this Agent / SDK type.\" in case\nthere is no dashboard for the service language. To be improved in\nhttps://github.com//issues/211774 and will be updated in\nthis PR when ready - I will still open it for review as the other logic\ncan be reviewed\n- The dashboards are to be updated (by the agent team so not part of the\nchanges here)\n\n## Testing: \n- Using e2e PoC \n- The available dashboard cases can be found in\n[loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40)\n- Cases to be checked:\n- OTel native with Vanilla OTel SDKs with available dashboard (example\ncase file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`)\n \n<img width=\"1903\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb\"\n/>\n\n- APM server with Vanilla OTel SDKs service with available dashboard\n(example case file: `classic_apm-otel_other-nodejs`, `...-java`,\n`...-dotnet`)\n \n\n\n\n- APM server with Classic APM Agent (example case file:\n`classic_apm-apm-nodejs`, `...-java`)\n \n<img width=\"962\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e\"\n/>\n\n- OTel native with Vanilla OTel SDKs without available dashboard (empty\nstate case example: python service)\n \n\n\n\n- APM server with Vanilla OTel SDKs service without available dashboard\n(empty state)\n \n<img width=\"1910\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281\"\n/>\n\n- APM server with Classic APM Agent without available dashboard (Current\nmetrics fallback)\n \n<img width=\"1914\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75\"\n/>\n\n-⚠️ OTel native Dashboards are still not available (at the time of\nadding the description)\n\n---------\n\nCo-authored-by: Sergi Romeu <[email protected]>\nCo-authored-by: Cauê Marcondes <[email protected]>\nCo-authored-by: kibanamachine <[email protected]>","sha":"f19557004904b490fcb69d18c1ea5f1ef9634ee0"}}]}] BACKPORT--> Co-authored-by: jennypavlova <[email protected]>
jennypavlova
added a commit
to jennypavlova/kibana
that referenced
this pull request
Mar 7, 2025
…ngest path (elastic#211822) Closes elastic#211783 Part of elastic#195857 ## Summary This PR expands the logic to get the dashboard files based on the agent. We have many different ways to ingest data so we want to add more metrics dashboards to the APM metrics tab. The different ingest paths we have: Classic APM Agent --> APM Server --> ES Vanilla OTel SDKs --> APM Server --> ES EDOT OTel SDKs --> APM Server --> ES Classic APM Agent --> EDOT Collector --> ES Vanilla OTel SDKs. --> EDOT Collector --> ES EDOT OTel SDKs --> EDOT Collector --> ES We agreed on having a dashboard filename pattern to make showing the correct dashboard easier described [here](elastic#195857 (comment)) First, we determine if the ingest path is through APM Server or EDOT Collector by checking the `telemtry.sdk` fields. ## TODOs / Reviewer notes - [ ] Currently, we have a fallback to metrics charts which is valid only if we have APM agent so this PR adds an empty state message: "Runtime metrics are not available for this Agent / SDK type." in case there is no dashboard for the service language. To be improved in elastic#211774 and will be updated in this PR when ready - I will still open it for review as the other logic can be reviewed - The dashboards are to be updated (by the agent team so not part of the changes here) ## Testing: - Using e2e PoC - The available dashboard cases can be found in [loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40) - Cases to be checked: - OTel native with Vanilla OTel SDKs with available dashboard (example case file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`) <img width="1903" alt="image" src="https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb" /> - APM server with Vanilla OTel SDKs service with available dashboard (example case file: `classic_apm-otel_other-nodejs`, `...-java`, `...-dotnet`)  - APM server with Classic APM Agent (example case file: `classic_apm-apm-nodejs`, `...-java`) <img width="962" alt="image" src="https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e" /> - OTel native with Vanilla OTel SDKs without available dashboard (empty state case example: python service)  - APM server with Vanilla OTel SDKs service without available dashboard (empty state) <img width="1910" alt="image" src="https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281" /> - APM server with Classic APM Agent without available dashboard (Current metrics fallback) <img width="1914" alt="image" src="https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75" /> -⚠️ OTel native Dashboards are still not available (at the time of adding the description) --------- Co-authored-by: Sergi Romeu <[email protected]> Co-authored-by: Cauê Marcondes <[email protected]> Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit f195570) # Conflicts: # src/platform/packages/shared/kbn-apm-synthtrace-client/src/lib/otel/index.ts
jennypavlova
added a commit
to jennypavlova/kibana
that referenced
this pull request
Mar 7, 2025
…ngest path (elastic#211822) Closes elastic#211783 Part of elastic#195857 ## Summary This PR expands the logic to get the dashboard files based on the agent. We have many different ways to ingest data so we want to add more metrics dashboards to the APM metrics tab. The different ingest paths we have: Classic APM Agent --> APM Server --> ES Vanilla OTel SDKs --> APM Server --> ES EDOT OTel SDKs --> APM Server --> ES Classic APM Agent --> EDOT Collector --> ES Vanilla OTel SDKs. --> EDOT Collector --> ES EDOT OTel SDKs --> EDOT Collector --> ES We agreed on having a dashboard filename pattern to make showing the correct dashboard easier described [here](elastic#195857 (comment)) First, we determine if the ingest path is through APM Server or EDOT Collector by checking the `telemtry.sdk` fields. ## TODOs / Reviewer notes - [ ] Currently, we have a fallback to metrics charts which is valid only if we have APM agent so this PR adds an empty state message: "Runtime metrics are not available for this Agent / SDK type." in case there is no dashboard for the service language. To be improved in elastic#211774 and will be updated in this PR when ready - I will still open it for review as the other logic can be reviewed - The dashboards are to be updated (by the agent team so not part of the changes here) ## Testing: - Using e2e PoC - The available dashboard cases can be found in [loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40) - Cases to be checked: - OTel native with Vanilla OTel SDKs with available dashboard (example case file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`) <img width="1903" alt="image" src="https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb" /> - APM server with Vanilla OTel SDKs service with available dashboard (example case file: `classic_apm-otel_other-nodejs`, `...-java`, `...-dotnet`)  - APM server with Classic APM Agent (example case file: `classic_apm-apm-nodejs`, `...-java`) <img width="962" alt="image" src="https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e" /> - OTel native with Vanilla OTel SDKs without available dashboard (empty state case example: python service)  - APM server with Vanilla OTel SDKs service without available dashboard (empty state) <img width="1910" alt="image" src="https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281" /> - APM server with Classic APM Agent without available dashboard (Current metrics fallback) <img width="1914" alt="image" src="https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75" /> -⚠️ OTel native Dashboards are still not available (at the time of adding the description) --------- Co-authored-by: Sergi Romeu <[email protected]> Co-authored-by: Cauê Marcondes <[email protected]> Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit f195570) # Conflicts: # packages/kbn-elastic-agent-utils/src/agent_ingestion_path.ts # packages/kbn-elastic-agent-utils/src/agent_sdk_name_and_language.test.ts # packages/kbn-elastic-agent-utils/src/agent_sdk_name_and_language.ts # src/platform/packages/shared/kbn-apm-synthtrace-client/src/lib/otel/index.ts # x-pack/plugins/observability_solution/apm/public/components/app/metrics/index.test.tsx # x-pack/plugins/observability_solution/apm/public/components/app/metrics/static_dashboard/dashboards/get_dashboard_file_name.test.ts # x-pack/plugins/observability_solution/apm/public/components/app/metrics/static_dashboard/dashboards/get_dashboard_file_name.ts
jennypavlova
added a commit
to jennypavlova/kibana
that referenced
this pull request
Mar 7, 2025
…ngest path (elastic#211822) Closes elastic#211783 Part of elastic#195857 ## Summary This PR expands the logic to get the dashboard files based on the agent. We have many different ways to ingest data so we want to add more metrics dashboards to the APM metrics tab. The different ingest paths we have: Classic APM Agent --> APM Server --> ES Vanilla OTel SDKs --> APM Server --> ES EDOT OTel SDKs --> APM Server --> ES Classic APM Agent --> EDOT Collector --> ES Vanilla OTel SDKs. --> EDOT Collector --> ES EDOT OTel SDKs --> EDOT Collector --> ES We agreed on having a dashboard filename pattern to make showing the correct dashboard easier described [here](elastic#195857 (comment)) First, we determine if the ingest path is through APM Server or EDOT Collector by checking the `telemtry.sdk` fields. ## TODOs / Reviewer notes - [ ] Currently, we have a fallback to metrics charts which is valid only if we have APM agent so this PR adds an empty state message: "Runtime metrics are not available for this Agent / SDK type." in case there is no dashboard for the service language. To be improved in elastic#211774 and will be updated in this PR when ready - I will still open it for review as the other logic can be reviewed - The dashboards are to be updated (by the agent team so not part of the changes here) ## Testing: - Using e2e PoC - The available dashboard cases can be found in [loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40) - Cases to be checked: - OTel native with Vanilla OTel SDKs with available dashboard (example case file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`) <img width="1903" alt="image" src="https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb" /> - APM server with Vanilla OTel SDKs service with available dashboard (example case file: `classic_apm-otel_other-nodejs`, `...-java`, `...-dotnet`)  - APM server with Classic APM Agent (example case file: `classic_apm-apm-nodejs`, `...-java`) <img width="962" alt="image" src="https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e" /> - OTel native with Vanilla OTel SDKs without available dashboard (empty state case example: python service)  - APM server with Vanilla OTel SDKs service without available dashboard (empty state) <img width="1910" alt="image" src="https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281" /> - APM server with Classic APM Agent without available dashboard (Current metrics fallback) <img width="1914" alt="image" src="https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75" /> -⚠️ OTel native Dashboards are still not available (at the time of adding the description) --------- Co-authored-by: Sergi Romeu <[email protected]> Co-authored-by: Cauê Marcondes <[email protected]> Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit f195570) # Conflicts: # packages/kbn-elastic-agent-utils/src/agent_guards.ts # packages/kbn-elastic-agent-utils/src/agent_ingestion_path.ts # packages/kbn-elastic-agent-utils/src/agent_sdk_name_and_language.test.ts # packages/kbn-elastic-agent-utils/src/agent_sdk_name_and_language.ts # src/platform/packages/shared/kbn-apm-synthtrace-client/src/lib/otel/index.ts # x-pack/plugins/observability_solution/apm/public/components/app/metrics/index.test.tsx # x-pack/plugins/observability_solution/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts # x-pack/plugins/observability_solution/apm/public/components/app/metrics/static_dashboard/dashboards/get_dashboard_file_name.test.ts # x-pack/plugins/observability_solution/apm/public/components/app/metrics/static_dashboard/dashboards/get_dashboard_file_name.ts # x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/services/agent.spec.ts
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
jennypavlova
added a commit
to jennypavlova/kibana
that referenced
this pull request
Mar 7, 2025
…ngest path (elastic#211822) Closes elastic#211783 Part of elastic#195857 ## Summary This PR expands the logic to get the dashboard files based on the agent. We have many different ways to ingest data so we want to add more metrics dashboards to the APM metrics tab. The different ingest paths we have: Classic APM Agent --> APM Server --> ES Vanilla OTel SDKs --> APM Server --> ES EDOT OTel SDKs --> APM Server --> ES Classic APM Agent --> EDOT Collector --> ES Vanilla OTel SDKs. --> EDOT Collector --> ES EDOT OTel SDKs --> EDOT Collector --> ES We agreed on having a dashboard filename pattern to make showing the correct dashboard easier described [here](elastic#195857 (comment)) First, we determine if the ingest path is through APM Server or EDOT Collector by checking the `telemtry.sdk` fields. ## TODOs / Reviewer notes - [ ] Currently, we have a fallback to metrics charts which is valid only if we have APM agent so this PR adds an empty state message: "Runtime metrics are not available for this Agent / SDK type." in case there is no dashboard for the service language. To be improved in elastic#211774 and will be updated in this PR when ready - I will still open it for review as the other logic can be reviewed - The dashboards are to be updated (by the agent team so not part of the changes here) ## Testing: - Using e2e PoC - The available dashboard cases can be found in [loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40) - Cases to be checked: - OTel native with Vanilla OTel SDKs with available dashboard (example case file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`) <img width="1903" alt="image" src="https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb" /> - APM server with Vanilla OTel SDKs service with available dashboard (example case file: `classic_apm-otel_other-nodejs`, `...-java`, `...-dotnet`)  - APM server with Classic APM Agent (example case file: `classic_apm-apm-nodejs`, `...-java`) <img width="962" alt="image" src="https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e" /> - OTel native with Vanilla OTel SDKs without available dashboard (empty state case example: python service)  - APM server with Vanilla OTel SDKs service without available dashboard (empty state) <img width="1910" alt="image" src="https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281" /> - APM server with Classic APM Agent without available dashboard (Current metrics fallback) <img width="1914" alt="image" src="https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75" /> -⚠️ OTel native Dashboards are still not available (at the time of adding the description) --------- Co-authored-by: Sergi Romeu <[email protected]> Co-authored-by: Cauê Marcondes <[email protected]> Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit f195570) # Conflicts: # packages/kbn-elastic-agent-utils/src/agent_guards.ts # packages/kbn-elastic-agent-utils/src/agent_ingestion_path.ts # packages/kbn-elastic-agent-utils/src/agent_sdk_name_and_language.test.ts # packages/kbn-elastic-agent-utils/src/agent_sdk_name_and_language.ts # src/platform/packages/shared/kbn-apm-synthtrace-client/src/lib/otel/index.ts # x-pack/plugins/observability_solution/apm/public/components/app/metrics/index.test.tsx # x-pack/plugins/observability_solution/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts # x-pack/plugins/observability_solution/apm/public/components/app/metrics/static_dashboard/dashboards/get_dashboard_file_name.test.ts # x-pack/plugins/observability_solution/apm/public/components/app/metrics/static_dashboard/dashboards/get_dashboard_file_name.ts # x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/services/agent.spec.ts
jennypavlova
added a commit
to jennypavlova/kibana
that referenced
this pull request
Mar 7, 2025
…ngest path (elastic#211822) Closes elastic#211783 Part of elastic#195857 ## Summary This PR expands the logic to get the dashboard files based on the agent. We have many different ways to ingest data so we want to add more metrics dashboards to the APM metrics tab. The different ingest paths we have: Classic APM Agent --> APM Server --> ES Vanilla OTel SDKs --> APM Server --> ES EDOT OTel SDKs --> APM Server --> ES Classic APM Agent --> EDOT Collector --> ES Vanilla OTel SDKs. --> EDOT Collector --> ES EDOT OTel SDKs --> EDOT Collector --> ES We agreed on having a dashboard filename pattern to make showing the correct dashboard easier described [here](elastic#195857 (comment)) First, we determine if the ingest path is through APM Server or EDOT Collector by checking the `telemtry.sdk` fields. ## TODOs / Reviewer notes - [ ] Currently, we have a fallback to metrics charts which is valid only if we have APM agent so this PR adds an empty state message: "Runtime metrics are not available for this Agent / SDK type." in case there is no dashboard for the service language. To be improved in elastic#211774 and will be updated in this PR when ready - I will still open it for review as the other logic can be reviewed - The dashboards are to be updated (by the agent team so not part of the changes here) ## Testing: - Using e2e PoC - The available dashboard cases can be found in [loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40) - Cases to be checked: - OTel native with Vanilla OTel SDKs with available dashboard (example case file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`) <img width="1903" alt="image" src="https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb" /> - APM server with Vanilla OTel SDKs service with available dashboard (example case file: `classic_apm-otel_other-nodejs`, `...-java`, `...-dotnet`)  - APM server with Classic APM Agent (example case file: `classic_apm-apm-nodejs`, `...-java`) <img width="962" alt="image" src="https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e" /> - OTel native with Vanilla OTel SDKs without available dashboard (empty state case example: python service)  - APM server with Vanilla OTel SDKs service without available dashboard (empty state) <img width="1910" alt="image" src="https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281" /> - APM server with Classic APM Agent without available dashboard (Current metrics fallback) <img width="1914" alt="image" src="https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75" /> -⚠️ OTel native Dashboards are still not available (at the time of adding the description) --------- Co-authored-by: Sergi Romeu <[email protected]> Co-authored-by: Cauê Marcondes <[email protected]> Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit f195570) # Conflicts: # packages/kbn-elastic-agent-utils/src/agent_ingestion_path.ts # packages/kbn-elastic-agent-utils/src/agent_sdk_name_and_language.test.ts # packages/kbn-elastic-agent-utils/src/agent_sdk_name_and_language.ts # src/platform/packages/shared/kbn-apm-synthtrace-client/src/lib/otel/index.ts # x-pack/plugins/observability_solution/apm/public/components/app/metrics/index.test.tsx # x-pack/plugins/observability_solution/apm/public/components/app/metrics/static_dashboard/dashboards/get_dashboard_file_name.test.ts # x-pack/plugins/observability_solution/apm/public/components/app/metrics/static_dashboard/dashboards/get_dashboard_file_name.ts
jennypavlova
added a commit
to jennypavlova/kibana
that referenced
this pull request
Mar 7, 2025
…ngest path (elastic#211822) Closes elastic#211783 Part of elastic#195857 ## Summary This PR expands the logic to get the dashboard files based on the agent. We have many different ways to ingest data so we want to add more metrics dashboards to the APM metrics tab. The different ingest paths we have: Classic APM Agent --> APM Server --> ES Vanilla OTel SDKs --> APM Server --> ES EDOT OTel SDKs --> APM Server --> ES Classic APM Agent --> EDOT Collector --> ES Vanilla OTel SDKs. --> EDOT Collector --> ES EDOT OTel SDKs --> EDOT Collector --> ES We agreed on having a dashboard filename pattern to make showing the correct dashboard easier described [here](elastic#195857 (comment)) First, we determine if the ingest path is through APM Server or EDOT Collector by checking the `telemtry.sdk` fields. ## TODOs / Reviewer notes - [ ] Currently, we have a fallback to metrics charts which is valid only if we have APM agent so this PR adds an empty state message: "Runtime metrics are not available for this Agent / SDK type." in case there is no dashboard for the service language. To be improved in elastic#211774 and will be updated in this PR when ready - I will still open it for review as the other logic can be reviewed - The dashboards are to be updated (by the agent team so not part of the changes here) ## Testing: - Using e2e PoC - The available dashboard cases can be found in [loadDashboardFile](https://github.com/jennypavlova/kibana/blob/91f169e19a3fa3f83ca60eb078159c026f9617af/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts#L40) - Cases to be checked: - OTel native with Vanilla OTel SDKs with available dashboard (example case file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`) <img width="1903" alt="image" src="https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb" /> - APM server with Vanilla OTel SDKs service with available dashboard (example case file: `classic_apm-otel_other-nodejs`, `...-java`, `...-dotnet`)  - APM server with Classic APM Agent (example case file: `classic_apm-apm-nodejs`, `...-java`) <img width="962" alt="image" src="https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e" /> - OTel native with Vanilla OTel SDKs without available dashboard (empty state case example: python service)  - APM server with Vanilla OTel SDKs service without available dashboard (empty state) <img width="1910" alt="image" src="https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281" /> - APM server with Classic APM Agent without available dashboard (Current metrics fallback) <img width="1914" alt="image" src="https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75" /> -⚠️ OTel native Dashboards are still not available (at the time of adding the description) --------- Co-authored-by: Sergi Romeu <[email protected]> Co-authored-by: Cauê Marcondes <[email protected]> Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit f195570) # Conflicts: # src/platform/packages/shared/kbn-apm-synthtrace-client/src/lib/otel/index.ts
kibanamachine
added
the
backport missing
|
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #211783
Part of #195857
Summary
This PR expands the logic to get the dashboard files based on the agent. We have many different ways to ingest data so we want to add more metrics dashboards to the APM metrics tab. The different ingest paths we have:
Classic APM Agent --> APM Server --> ES
Vanilla OTel SDKs --> APM Server --> ES
EDOT OTel SDKs --> APM Server --> ES
Classic APM Agent --> EDOT Collector --> ES
Vanilla OTel SDKs. --> EDOT Collector --> ES
EDOT OTel SDKs --> EDOT Collector --> ES
We agreed on having a dashboard filename pattern to make showing the correct dashboard easier described here
First, we determine if the ingest path is through APM Server or EDOT Collector by checking the
telemtry.sdkfields.TODOs / Reviewer notes
Testing:
otel_native-otel_other-nodejs,...-java,...-dotnet)classic_apm-otel_other-nodejs,...-java,...-dotnet)classic_apm-apm-nodejs,...-java)