Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IF: Process votes in a dedicated thread pool #24

Merged
merged 42 commits into from
Apr 19, 2024
Merged
Changes from 1 commit
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
7fc69ac
GH-3 Move vote processing off net threads into a dedicated thread pool
heifner Apr 10, 2024
c3a9c95
GH-3 Track num_messages per connection
heifner Apr 11, 2024
6b0c4c4
GH-3 Make vote processor thread pool size configurable including disa…
heifner Apr 11, 2024
fba5561
GH-3 Fix issue with not re-locking after unlock
heifner Apr 11, 2024
1dcf7c1
GH-3 Shutdown nodeos on vote thread pool exception
heifner Apr 11, 2024
295f7d4
GH-3 Add unittest for vote_processor. Modify vote_processor to make i…
heifner Apr 11, 2024
18c2fca
GH-3 std::latch not available on all platforms; use std::atomic and f…
heifner Apr 11, 2024
d5cef78
GH-3 ci/cd is slow, allow more time.
heifner Apr 11, 2024
da0106b
GH-3 Make test more robust by checking that node has received a hands…
heifner Apr 11, 2024
bf545e3
GH-3 Optimization: Don't emit duplicate votes
heifner Apr 11, 2024
d1fe636
GH-3 Fix: num_messages.clear() called before decrement
heifner Apr 11, 2024
f3ff3e8
GH-3 Avoid vote_message copies by using shared_ptr
heifner Apr 12, 2024
ae02196
GH-3 Avoid vote_message copies by using shared_ptr
heifner Apr 12, 2024
a6e4de8
GH-3 Fix tests now that duplicates are not signaled
heifner Apr 12, 2024
382f648
GH-3 Fix use of packed_transaction_ptr over shared_ptr<packed_transac…
heifner Apr 12, 2024
508b584
GH-3 There are three producers A,B,C if the test happens to hit this …
heifner Apr 12, 2024
4d70ab3
GH-3 Test failed on asan run because it took 55ms to load the WASM. I…
heifner Apr 12, 2024
23b5739
Merge remote-tracking branch 'spring/savanna' into GH-3-process-votes
heifner Apr 12, 2024
1641a07
GH-3 Report known lib instead of fork db root as these log statements…
heifner Apr 12, 2024
4b4eca6
GH-3 Change default --vote-threads to 0 to be disabled by default. EO…
heifner Apr 15, 2024
895cef7
GH-3 --vote-threads needed for all producers
heifner Apr 15, 2024
796dc5b
Merge remote-tracking branch 'spring/savanna' into GH-3-process-votes
heifner Apr 16, 2024
c3bdd8a
GH-12 Add vote threads needed for unittests
heifner Apr 16, 2024
e4010d2
GH-12 Add vote threads
heifner Apr 16, 2024
bf8fe18
GH-12 Add vote threads to tests
heifner Apr 16, 2024
8bfbe0a
GH-3 Modify named_thread_pool to be a no-op if given 0 for num_threads.
heifner Apr 16, 2024
117f02d
GH-3 Check for stopped in inner loop
heifner Apr 16, 2024
027d278
GH-3 Add better descriptions
heifner Apr 16, 2024
8f26a50
GH-3 Default vote-threads to 4 when a block producer
heifner Apr 16, 2024
6cc6820
GH-3 Simplify by adding a vote_signal_t type
heifner Apr 16, 2024
166a580
GH-3 Use chain pluging accept_votes to init p2p_accept_votes
heifner Apr 16, 2024
83434d1
GH-3 Add vote-threads to all nodes so bridge nodes process votes
heifner Apr 16, 2024
042ce30
GH-3 Simplify vote_processor by processing votes on a first-come-firs…
heifner Apr 17, 2024
d80def5
GH-3 Fix use of iterator after erase
heifner Apr 17, 2024
317dd61
GH-3 Remove unneeded if
heifner Apr 18, 2024
955a72a
GH-3 Move emit to controller.hpp and use in vote_processor
heifner Apr 18, 2024
4a759b3
GH-3 Use same value as default for producer
heifner Apr 18, 2024
51503c4
GH-3 Use unordered_map
heifner Apr 18, 2024
b694aad
GH-3 Add better log message
heifner Apr 18, 2024
344b778
GH-3 Do not clear num_messages if there are votes in the index to be …
heifner Apr 18, 2024
ecc8cd4
GH-3 More descriptive emit logs
heifner Apr 18, 2024
8d3a838
GH-3 Fix spelling
heifner Apr 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
GH-3 Use unordered_map
heifner committed Apr 18, 2024
commit 51503c404c275f94e762891fc2914ae794235c11
13 changes: 7 additions & 6 deletions libraries/chain/include/eosio/chain/vote_processor.hpp
Original file line number Diff line number Diff line change
@@ -10,14 +10,16 @@
#include <boost/multi_index/mem_fun.hpp>
#include <boost/multi_index/ordered_index.hpp>

#include <unordered_map>

namespace eosio::chain {

/**
* Process votes in a dedicated thread pool.
*/
class vote_processor_t {
// Even 3000 vote structs are less than 1MB per connection.
// 2500 is should never be reached unless a specific connection is sending garbage.
// 2500 should never be reached unless a specific connection is sending garbage.
static constexpr size_t max_votes_per_connection = 2500;
// If we have not processed a vote in this amount of time, give up on it.
static constexpr fc::microseconds too_old = fc::seconds(5);
@@ -51,8 +53,8 @@ class vote_processor_t {
std::mutex mtx;
vote_index_type index;
block_state_ptr last_bsp;
// connection, count of messages
std::map<uint32_t, uint16_t> num_messages;
// connection, count of messages
std::unordered_map<uint32_t, uint16_t> num_messages;

std::atomic<block_num_type> lib{0};
std::atomic<block_num_type> largest_known_block_num{0};
@@ -99,19 +101,18 @@ class vote_processor_t {
index.insert(vote{.connection_id = connection_id, .received = now, .msg = msg});
}

// called with locked mtx
// called with locked mtx, returns with a locked mutex
void process_any_queued_for_later(std::unique_lock<std::mutex>& g) {
if (index.empty())
return;
remove_too_old();
remove_before_lib();
auto& idx = index.get<by_last_received>();
std::vector<vote> unprocessed;
unprocessed.reserve(std::min<size_t>(21u, idx.size())); // maybe increase if we increase # of finalizers from 21
for (auto i = idx.begin(); i != idx.end();) {
if (stopped)
return;
vote v = *i;
vote v = std::move(*i);
idx.erase(i);
auto bsp = get_block(v.msg->block_id, g);
// g is unlocked
2 changes: 1 addition & 1 deletion plugins/net_plugin/net_plugin.cpp
Original file line number Diff line number Diff line change
@@ -535,7 +535,7 @@ namespace eosio {

void on_accepted_block_header( const signed_block_ptr& block, const block_id_type& id );
void on_accepted_block();
void on_voted_block ( uint32_t connection_id, vote_status stauts, const vote_message_ptr& vote );
void on_voted_block( uint32_t connection_id, vote_status stauts, const vote_message_ptr& vote );

void transaction_ack(const std::pair<fc::exception_ptr, packed_transaction_ptr>&);
void on_irreversible_block( const block_id_type& id, uint32_t block_num );