Various bulk benchmark improvements #1193
Conversation
Matthias247
force-pushed
the
bench
branch
2 times, most recently
from
26c83b6 to
803f2af
Compare
In order to extend the bulk benchmark further, this change moves most common functions out of the bulk.rs binary and into a lib.rs and stats.rs file. Functionally there are no relevant changes.
This changes the bulk benchmark to support server->client in addition to client->server transfers. The default direction is set to server->client, since this is more typically used.
|
Nice work! |
bench/src/bin/bulk.rs
Outdated
| drain_stream(&mut recv_stream, opt.read_unordered).await?; | ||
| send_data_on_stream(&mut send_stream, opt.download_size).await?; |
There was a problem hiding this comment.
Should these be concurrent? Sequential seems of limited value, since you could get similar data from just running two tests. Alternatively, use unidirectional streams in both directions, with handlers spawned independently?
There was a problem hiding this comment.
I thought about it. But that would require adding either a join! which adds overhead due to polling non-ready tasks, or adding more subtasks or futures-unordered. In the end I decided its not worth it since testing both directions at the same time doesn't seem super useful, and sequential still gives useful values since at least the per-stream metrics only cover the actual transfer time inside the stream.
There was a problem hiding this comment.
If we use unidirectional streams in each direction, I don't think the actual number of subtasks needs to change, since streams for an unused transfer direction can be dynamically omitted.
That said, I don't see a clear requirement for the capability, so 🤷, nonblocking.
This makes it a bit easier to specify large arguments
This allows to specify a "--client" parameter in the benchmark, which specifies the amount of clients which perform a test concurrently. Each client uses its own thread and single-threaded runtime. Stats are currently printed for each individual client and not aggregated.
This adds a set of improvements to the bulk benchmark:
10G)The first commit in the set refactors the executable a bit in order to avoid having a single huge file and should make the other changes a bit easier to follow.
Here's a invocation which gives a summary about the new functionality: