[Clone] [Security Manager Replacement] Native Java Agent #17517

kumargu · 2025-03-05T07:11:20Z

Clone of original implementation of Java agent

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2025-03-05T07:35:44Z

❌ Gradle check result for 1b6692e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-05T08:18:00Z

❌ Gradle check result for 2fcdd44: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-05T09:38:16Z

❌ Gradle check result for 21e708c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-05T15:34:35Z

❌ Gradle check result for e8aab30: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-05T16:15:31Z

❌ Gradle check result for dbbef0e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-05T18:37:03Z

❌ Gradle check result for 1923e2d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-05T19:03:31Z

❌ Gradle check result for 5cb1c07: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-05T19:31:52Z

❌ Gradle check result for 51328a0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-06T08:57:44Z

❌ Gradle check result for 447c4a8: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-06T09:30:12Z

❌ Gradle check result for 88f3d6f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-06T11:06:57Z

❌ Gradle check result for ae3e3c2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-06T11:58:04Z

❌ Gradle check result for a5c15b4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-06T14:09:31Z

❌ Gradle check result for 5a51029: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-06T14:40:06Z

❌ Gradle check result for bfe165f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

kumargu · 2025-03-06T16:26:51Z

❌ Gradle check result for bfe165f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@peterzhuamazon would you know if I can run this check locally and fix it faster? thanks!

github-actions · 2025-03-12T18:36:44Z

❌ Gradle check result for c4db178: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-12T19:27:54Z

❌ Gradle check result for 03761a6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-12T22:06:04Z

❌ Gradle check result for f832563: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

reta · 2025-03-13T01:21:28Z

libs/agent-sm/agent/src/main/java/org/opensearch/javaagent/SocketChannelInterceptor.java

@@ -52,19 +52,23 @@ public static void intercept(@Advice.AllArguments Object[] args, @Origin Method
        if (args[0] instanceof InetSocketAddress address) {
            if (!AgentPolicy.isTrustedHost(address.getHostString())) {
                final String host = address.getHostString() + ":" + address.getPort();
-
-                final SocketPermission permission = new SocketPermission(host, "connect,resolve");
+                final SocketPermission connectResolve = new SocketPermission("*", "connect,resolve");


@kumargu I think this is not the right fix: the host comes from policy and has to be matched.

you are right. I overlooked it while copying it from the policy.

github-actions · 2025-03-13T07:25:47Z

✅ Gradle check result for c772cc5: SUCCESS

codecov · 2025-03-13T07:27:04Z

Codecov Report

Attention: Patch coverage is 21.01785% with 1195 lines in your changes missing coverage. Please review.

Project coverage is 72.30%. Comparing base (2ee8660) to head (1483b8b).
Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
...va/org/opensearch/secure_sm/policy/PolicyFile.java	24.82%	405 Missing and 28 partials ⚠️
.../org/opensearch/secure_sm/policy/PolicyParser.java	21.00%	350 Missing and 26 partials ⚠️
...ava/org/opensearch/secure_sm/policy/ParseUtil.java	7.14%	189 Missing and 6 partials ⚠️
...java/org/opensearch/secure_sm/policy/Password.java	0.00%	60 Missing ⚠️
...va/org/opensearch/secure_sm/policy/PolicyUtil.java	15.78%	31 Missing and 1 partial ⚠️
.../src/main/java/org/opensearch/javaagent/Agent.java	0.00%	25 Missing ⚠️
...opensearch/javaagent/SocketChannelInterceptor.java	0.00%	24 Missing ⚠️
...rg/opensearch/javaagent/bootstrap/AgentPolicy.java	0.00%	19 Missing ⚠️
.../opensearch/secure_sm/policy/PropertyExpander.java	60.86%	12 Missing and 6 partials ⚠️
...pensearch/javaagent/StackCallerChainExtractor.java	0.00%	6 Missing ⚠️
... and 4 more

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #17517      +/-   ##
============================================
- Coverage     72.43%   72.30%   -0.13%     
- Complexity    65694    65877     +183     
============================================
  Files          5311     5322      +11     
  Lines        304937   306408    +1471     
  Branches      44226    44574     +348     
============================================
+ Hits         220872   221555     +683     
- Misses        65912    66717     +805     
+ Partials      18153    18136      -17

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

kumargu · 2025-03-13T08:00:26Z

@reta thank-you! for pointing out the issue. looks like the all-the-gradle-check passed.

github-actions · 2025-03-13T10:40:57Z

❕ Gradle check result for 1483b8b: UNSTABLE

TEST FAILURES:

      4 org.opensearch.repositories.s3.S3BlobStoreRepositoryTests.classMethod
      2 org.opensearch.snapshots.DedicatedClusterSnapshotRestoreIT.testSnapshotWithStuckNode
      1 org.opensearch.action.admin.indices.create.CreateIndexIT.testCreateAndDeleteIndexConcurrently
      1 org.opensearch.action.admin.indices.create.CreateIndexIT.classMethod

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Signed-off-by: Gulshan Kumar <[email protected]>

peterzhuamazon · 2025-03-13T11:43:39Z

Commit sha: e36c1e5, Author: Gulshan, Committer: Gulshan; Expected "Gulshan [email protected]", but got "Gulshan Kumar [email protected]".

github-actions · 2025-03-13T12:06:48Z

❌ Gradle check result for e36c1e5: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

peterzhuamazon · 2025-03-13T12:24:09Z

CHANGELOG-3.0.md

@@ -25,7 +25,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Add filter function for AbstractQueryBuilder, BoolQueryBuilder, ConstantScoreQueryBuilder([#17409](https://github.com/opensearch-project/OpenSearch/pull/17409))
 - [Star Tree] [Search] Resolving keyword & numeric bucket aggregation with metric aggregation using star-tree ([#17165](https://github.com/opensearch-project/OpenSearch/pull/17165))
 - Added error handling support for the pull-based ingestion ([#17427](https://github.com/opensearch-project/OpenSearch/pull/17427))
-
+- A Java Agent for intercepting socket operation and enforcing a security policy to it.


Suggested change

- A Java Agent for intercepting socket operation and enforcing a security policy to it.

- A Java Agent for intercepting socket operation and enforcing a security policy to it ([#17517](https://github.com/opensearch-project/OpenSearch/pull/17517))

reta · 2025-03-13T14:17:01Z

libs/agent-sm/agent/src/main/java/org/opensearch/javaagent/SocketChannelInterceptor.java

-
-                final SocketPermission permission = new SocketPermission(host, "connect,resolve");
+                final SocketPermission connectResolve = new SocketPermission(host, "connect,resolve");
+                final SocketPermission listenResolve = new SocketPermission(host, "listen,resolve");


@kumargu I am not sure what exactly you are trying to fix here: we are intercepting SocketChannel:connect only which connects to remote address. The listen is not applicable here, we are not be listening on a socket (this is ServerSocketChannel responsibility). Could you please clarify this change?

I was trying to fix this test failure. Looking at the trace, I think it fails with while attempting "listen & resolve" from localhost:0

I had a similar assumption that we are only dealing with SocketChannel:connect but all my efforts to just play around with the policy file didn't work-out and this particular does seem to work.

Thanks @kumargu , I will take a look shortly

kumargu force-pushed the kumargu_java_agent_v1 branch from 2fcdd44 to 21e708c Compare March 5, 2025 09:09

kumargu force-pushed the kumargu_java_agent_v1 branch from 21e708c to e8aab30 Compare March 5, 2025 15:00

kumargu force-pushed the kumargu_java_agent_v1 branch from e8aab30 to dbbef0e Compare March 5, 2025 15:50

kumargu force-pushed the kumargu_java_agent_v1 branch from dbbef0e to 1923e2d Compare March 5, 2025 18:13

kumargu force-pushed the kumargu_java_agent_v1 branch from 1923e2d to 5cb1c07 Compare March 5, 2025 18:42

kumargu force-pushed the kumargu_java_agent_v1 branch from 5cb1c07 to 51328a0 Compare March 5, 2025 19:08

kumargu force-pushed the kumargu_java_agent_v1 branch from 51328a0 to 447c4a8 Compare March 6, 2025 08:23

kumargu force-pushed the kumargu_java_agent_v1 branch from 447c4a8 to 88f3d6f Compare March 6, 2025 09:01

kumargu force-pushed the kumargu_java_agent_v1 branch 3 times, most recently from 12d3cc3 to ae3e3c2 Compare March 6, 2025 10:36

kumargu force-pushed the kumargu_java_agent_v1 branch from ae3e3c2 to a5c15b4 Compare March 6, 2025 11:35

kumargu force-pushed the kumargu_java_agent_v1 branch from a5c15b4 to 5a51029 Compare March 6, 2025 13:44

kumargu force-pushed the kumargu_java_agent_v1 branch from 5a51029 to bfe165f Compare March 6, 2025 14:16

kumargu force-pushed the kumargu_java_agent_v1 branch from bfe165f to 1bc341c Compare March 6, 2025 17:56

opensearch-ci-bot mentioned this pull request Mar 12, 2025

[AUTOCUT] Gradle Check Flaky Test Report for SecureReactorNetty4HttpServerTransportTests #17486

Open

kumargu force-pushed the kumargu_java_agent_v1 branch from 6570221 to c4db178 Compare March 12, 2025 18:02

kumargu force-pushed the kumargu_java_agent_v1 branch from c4db178 to aa61d93 Compare March 12, 2025 18:52

kumargu force-pushed the kumargu_java_agent_v1 branch 2 times, most recently from c8805a6 to f832563 Compare March 12, 2025 20:01

opensearch-ci-bot mentioned this pull request Mar 13, 2025

[AUTOCUT] Gradle Check Flaky Test Report for MinimumClusterManagerNodesIT #14289

Open

reta reviewed Mar 13, 2025

View reviewed changes

kumargu closed this Mar 13, 2025

kumargu deleted the kumargu_java_agent_v1 branch March 13, 2025 05:02

kumargu reopened this Mar 13, 2025

kumargu force-pushed the kumargu_java_agent_v1 branch from f832563 to c772cc5 Compare March 13, 2025 06:30

kumargu force-pushed the kumargu_java_agent_v1 branch 3 times, most recently from d5e1c00 to 1483b8b Compare March 13, 2025 09:49

Fix tests

e36c1e5

Signed-off-by: Gulshan Kumar <[email protected]>

kumargu force-pushed the kumargu_java_agent_v1 branch from 1483b8b to e36c1e5 Compare March 13, 2025 11:17

peterzhuamazon reviewed Mar 13, 2025

View reviewed changes

This was referenced Mar 13, 2025

[AUTOCUT] Gradle Check Flaky Test Report for S3BlobStoreRepositoryTests #14299

Open

[AUTOCUT] Gradle Check Flaky Test Report for IndicesRequestCacheCleanupIT #15822

Open

[AUTOCUT] Gradle Check Flaky Test Report for ResourceAwareTasksTests #14293

Open

reta reviewed Mar 13, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Clone] [Security Manager Replacement] Native Java Agent #17517

[Clone] [Security Manager Replacement] Native Java Agent #17517

kumargu commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 6, 2025

github-actions bot commented Mar 6, 2025

github-actions bot commented Mar 6, 2025

github-actions bot commented Mar 6, 2025

github-actions bot commented Mar 6, 2025

github-actions bot commented Mar 6, 2025

kumargu commented Mar 6, 2025

github-actions bot commented Mar 12, 2025

github-actions bot commented Mar 12, 2025

github-actions bot commented Mar 12, 2025

reta Mar 13, 2025

kumargu Mar 13, 2025

github-actions bot commented Mar 13, 2025

codecov bot commented Mar 13, 2025 •

edited

Loading

kumargu commented Mar 13, 2025 •

edited

Loading

github-actions bot commented Mar 13, 2025

peterzhuamazon commented Mar 13, 2025 •

edited

Loading

github-actions bot commented Mar 13, 2025

peterzhuamazon Mar 13, 2025

kumargu Mar 13, 2025

reta Mar 13, 2025

kumargu Mar 13, 2025

kumargu Mar 13, 2025

reta Mar 13, 2025

kumargu Mar 13, 2025

	- A Java Agent for intercepting socket operation and enforcing a security policy to it.
	- A Java Agent for intercepting socket operation and enforcing a security policy to it ([#17517](https://github.com/opensearch-project/OpenSearch/pull/17517))

[Clone] [Security Manager Replacement] Native Java Agent #17517

Are you sure you want to change the base?

[Clone] [Security Manager Replacement] Native Java Agent #17517

Conversation

kumargu commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

github-actions bot commented Mar 6, 2025

github-actions bot commented Mar 6, 2025

github-actions bot commented Mar 6, 2025

github-actions bot commented Mar 6, 2025

github-actions bot commented Mar 6, 2025

github-actions bot commented Mar 6, 2025

kumargu commented Mar 6, 2025

github-actions bot commented Mar 12, 2025

github-actions bot commented Mar 12, 2025

github-actions bot commented Mar 12, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Mar 13, 2025

codecov bot commented Mar 13, 2025 • edited Loading

Codecov Report

kumargu commented Mar 13, 2025 • edited Loading

github-actions bot commented Mar 13, 2025

peterzhuamazon commented Mar 13, 2025 • edited Loading

github-actions bot commented Mar 13, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Mar 13, 2025 •

edited

Loading

kumargu commented Mar 13, 2025 •

edited

Loading

peterzhuamazon commented Mar 13, 2025 •

edited

Loading