Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor io_uring unit test #2099

Merged
merged 12 commits into from
Feb 22, 2022
Merged

refactor io_uring unit test #2099

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
19acf1a
refactor io_uring unit test
sullis Feb 19, 2022
532495e
introduce matrix.gradle-command
sullis Feb 20, 2022
0704b0e
use Bash shell in ci-prb.yml
sullis Feb 20, 2022
ad17f86
Revert "use Bash shell in ci-prb.yml"
sullis Feb 20, 2022
22ea5fc
fix matrix in ci-prb.yml
sullis Feb 20, 2022
bb1ec0d
set matrix gradle-command in ci-prq.yml
sullis Feb 20, 2022
827ed00
modify ci-release.yml and ci-snapshot.yml
sullis Feb 20, 2022
05affa6
restore ci-prq.yml
sullis Feb 22, 2022
8e5ec29
linux specific build step in GitHub Actions
sullis Feb 22, 2022
53600a4
improve assertions in ioUringIsNotAvailableOnMacOs
sullis Feb 22, 2022
ec2c0bd
cleanup ioUringIsAvailableOnLinux
sullis Feb 22, 2022
13cb2b9
remove unused import
sullis Feb 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions .github/workflows/ci-prb.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,18 @@ jobs:
with:
minimum-size: 8GB
maximum-size: 16GB
- name: Build and Test
- name: Build and Test (Linux)
if: runner.os == 'Linux'
env:
CI: true
JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
run: ./gradlew --parallel clean test
run: sudo env "PATH=$PATH" bash -c "ulimit -l 65536 && ulimit -a && ./gradlew --no-daemon --parallel clean test"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it required to execute ulimit in the same bash -c command as ./gradlew?
I was thinking about a separate optional step, something like:

- name: Configure Linux for io_uring
  if: runner.os == 'Linux'
  run: sudo ulimit -l 65536 && ulimit -a

That way there is no repetition for the gradle command and tests won't run with root privileges.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ulimit is discussed here: actions/runner-images#4683

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the context!

- name: Build and Test (non-Linux)
if: runner.os != 'Linux'
env:
CI: true
JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
run: ./gradlew --no-daemon --parallel clean test
- name: Upload Test Results
if: always()
uses: actions/upload-artifact@v2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ jobs:
fi

# Execute the gradlew command to publish the build
./gradlew --parallel -PreleaseBuild=true$FIRST_GRADLE_TARGETS && ./gradlew -PreleaseBuild=true$SECOND_GRADLE_TARGETS
sudo env "PATH=$PATH" "FIRST_GRADLE_TARGETS=$FIRST_GRADLE_TARGETS" "SECOND_GRADLE_TARGETS=$SECOND_GRADLE_TARGETS" bash -c "ulimit -l 65536 && ulimit -a && ./gradlew --no-daemon --parallel -PreleaseBuild=true$FIRST_GRADLE_TARGETS && ./gradlew --no-daemon -PreleaseBuild=true$SECOND_GRADLE_TARGETS"
- name: Publish Test Results
if: always()
uses: scacap/action-surefire-report@482f012643ed0560e23ef605a79e8e87ca081648
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-snapshot.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ jobs:
fi

# Execute the gradlew command to publish the build
./gradlew --parallel$FIRST_GRADLE_TARGETS && ./gradlew$SECOND_GRADLE_TARGETS
sudo env "PATH=$PATH" "FIRST_GRADLE_TARGETS=$FIRST_GRADLE_TARGETS" "SECOND_GRADLE_TARGETS=$SECOND_GRADLE_TARGETS" bash -c "ulimit -l 65536 && ulimit -a && ./gradlew --no-daemon --parallel$FIRST_GRADLE_TARGETS && ./gradlew --no-daemon$SECOND_GRADLE_TARGETS"
- name: Publish Test Results
if: always()
uses: scacap/action-surefire-report@482f012643ed0560e23ef605a79e8e87ca081648
Expand Down
2 changes: 2 additions & 0 deletions servicetalk-http-netty/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,8 @@ dependencies {
testImplementation "io.netty:netty-transport-native-unix-common"
testImplementation "io.netty:netty-tcnative-boringssl-static"
testImplementation "io.netty.incubator:netty-incubator-transport-native-io_uring:$nettyIoUringVersion"
testImplementation "io.netty.incubator:netty-incubator-transport-native-io_uring:$nettyIoUringVersion:linux-x86_64"
testImplementation "io.netty.incubator:netty-incubator-transport-native-io_uring:$nettyIoUringVersion:linux-aarch_64"
testImplementation "org.junit.jupiter:junit-jupiter-api"
testImplementation "org.junit.jupiter:junit-jupiter-params"
testImplementation "org.hamcrest:hamcrest:$hamcrestVersion"
Expand Down
34 changes: 30 additions & 4 deletions servicetalk-http-netty/src/test/java/io/servicetalk/http/netty/IoUringTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,32 +16,55 @@
package io.servicetalk.http.netty;

import io.servicetalk.http.api.BlockingHttpClient;
import io.servicetalk.http.api.HttpRequest;
import io.servicetalk.http.api.HttpResponse;
import io.servicetalk.transport.api.ServerContext;
import io.servicetalk.transport.netty.internal.EventLoopAwareNettyIoExecutor;
import io.servicetalk.transport.netty.internal.IoUringUtils;
import io.servicetalk.transport.netty.internal.NettyIoExecutors;

import io.netty.incubator.channel.uring.IOUring;
import io.netty.incubator.channel.uring.IOUringEventLoopGroup;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.condition.EnabledOnOs;

import static io.servicetalk.http.api.HttpResponseStatus.OK;
import static io.servicetalk.http.api.HttpSerializers.textSerializerUtf8;
import static io.servicetalk.http.netty.TestServiceStreaming.SVC_ECHO;
import static io.servicetalk.transport.netty.internal.AddressUtils.localAddress;
import static io.servicetalk.transport.netty.internal.AddressUtils.serverHostAndPort;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.instanceOf;
import static org.hamcrest.Matchers.is;
import static org.junit.jupiter.api.Assumptions.assumeTrue;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.condition.OS.LINUX;
import static org.junit.jupiter.api.condition.OS.MAC;

class IoUringTest {

@Test
void test() throws Exception {
@EnabledOnOs(value = { MAC })
void ioUringIsNotAvailableOnMacOs() {
assertFalse(IOUring.isAvailable());
try {
IoUringUtils.tryIoUring(false);
assertFalse(IoUringUtils.isAvailable());
IoUringUtils.tryIoUring(true);
assertFalse(IoUringUtils.isAvailable());
} finally {
IoUringUtils.tryIoUring(false);
}
}

@Test
@EnabledOnOs(value = { LINUX })
void ioUringIsAvailableOnLinux() throws Exception {
EventLoopAwareNettyIoExecutor ioUringExecutor = null;
try {
IoUringUtils.tryIoUring(true);
assumeTrue(IoUringUtils.isAvailable());
assertTrue(IoUringUtils.isAvailable());
IOUring.ensureAvailability();

ioUringExecutor = NettyIoExecutors.createIoExecutor(2, "io-uring");
assertThat(ioUringExecutor.eventLoopGroup(), is(instanceOf(IOUringEventLoopGroup.class)));
Expand All @@ -52,13 +75,16 @@ void test() throws Exception {
BlockingHttpClient client = HttpClients.forSingleAddress(serverHostAndPort(serverContext))
.ioExecutor(ioUringExecutor)
.buildBlocking()) {
HttpResponse response = client.request(client.get(SVC_ECHO));
HttpRequest request = client.post(SVC_ECHO).payloadBody("bonjour!", textSerializerUtf8());
HttpResponse response = client.request(request);
assertThat(response.status(), is(OK));
assertThat(response.payloadBody(textSerializerUtf8()), is("bonjour!"));
}
} finally {
IoUringUtils.tryIoUring(false);
if (ioUringExecutor != null) {
ioUringExecutor.closeAsync().toFuture().get();
assertTrue(ioUringExecutor.eventLoopGroup().isShutdown());
}
}
}
Expand Down