Fix issue #125: sign block with old key if signing key updated in same block #1203

abitmore · 2018-07-29T21:08:53Z

PR for #125.

This PR also contains a fix about block size check, and a fix about removing item from fork db.

Things to be done:

test case for loading object_database from disk which contains signing keys not in genesis
test case for witness_update_operation
test case for witness_create_operation
~~test case for witness plugin~~ update: I have done some manual tests, it's too much efforts to create such test cases.

When signing key is updated in the same block, producing will fail.

abitmore · 2018-07-29T22:17:01Z

Travis complains:

2795751ms th_a object_database.cpp:106 open ] Opening object database from /tmp/graphene-tmp/e16c-550d-aba8-ade6 ...
chain_test: /home/travis/build/bitshares/bitshares-core/libraries/db/include/graphene/db/simple_index.hpp:67: const graphene::db::object& graphene::db::simple_index::insert(graphene::db::object&&) [with T = graphene::chain::global_property_object]: Assertion `!_objects[instance]' failed.
Running 214 test cases...
unknown location(0): fatal error in "change_signing_key": signal: SIGABRT (application abort requested)
/home/travis/build/bitshares/bitshares-core/tests/common/database_fixture.cpp(271): last checkpoint

Related code:

bitshares-core/tests/tests/block_tests.cpp

Lines 327 to 331 in 37e72ad

    
           // close the database, flush all data to disk 
        
           db.close(); 
        
           // reopen database, all data should be unchanged 
        
           db.open(data_dir.path(), make_genesis, "TEST");

Failed to reopen database file after closed? Any idea how to fix?

pmconrad · 2018-07-31T13:56:29Z

The database is not meant to be reopened after close. Also see discussion in #436 #687 #689 .

abitmore · 2018-07-31T20:56:46Z

Destroyed db object and created a new one. Travis-CI no longer complains.

moved `init_witness_key_cache()` call from `plugin_startup()` to `plugin_initialize()`.

pmconrad · 2018-08-19T15:59:52Z

libraries/chain/include/graphene/chain/database.hpp

      private:
         void                  _apply_block( const signed_block& next_block );
         processed_transaction _apply_transaction( const signed_transaction& trx );
         void                  _cancel_bids_and_revive_mpa( const asset_object& bitasset, const asset_bitasset_data_object& bad );

+         /// For tracking signing keys of specified witnesses, only update when applied a block or popped a block
+         flat_map< witness_id_type, optional<public_key_type> > _witness_key_cache;


IMO it is a bad idea to introduce block-dependent state in database that is not covered by object_db. This state must be maintained manually, which is fragile. I think the problem at hand can be solved in a more simple way.

pmconrad · 2018-08-19T16:04:44Z

libraries/plugins/witness/witness.cpp

@@ -250,7 +253,7 @@ block_production_condition::block_production_condition_enum witness_plugin::mayb
   }

   fc::time_point_sec scheduled_time = db.get_slot_time( slot );
-   graphene::chain::public_key_type scheduled_key = scheduled_witness( db ).signing_key;
+   graphene::chain::public_key_type scheduled_key = *db.find_witness_key_from_cache( scheduled_witness ); // should be valid


I think instead of using the cache, witness_plugin could simply connect to the database::applied_block signal and use that to retrieve the latest keys of the witness(es) it is interested in.

Essentially you mean to move the cache to witness plugin. Because there is no signal when a block is popped out, in order to deal with chain reorganization, the cache should be in (extended) chain state. Then, in order to make it work, need to either replay the whole chain to enable the plugin, or add code to refresh the cache (copy data from main chain state to the extended part) on startup. Then, in case when there is a reorganization to the startup head block, need to make sure the cache don't get emptied.

I didn't see it's simpler so far.

The witness plugin is only interested in current head block. Blocks are only popped when switching forks, which means there will be block pushes immediately after the pop, which means the cache will automatically be updated in that case.
This is slightly different from the cache in database itself - that should always be up to date, also after popping blocks.

Overall, I think even if the resulting code is not significantly simpler, the containment in the witness plugin is a big plus.

I agree that ideally it's better to put the logic in witness plugin because block generation is actually witness plugin's business.

Blocks are only popped when switching forks, which means there will be block pushes immediately after the pop, which means the cache will automatically be updated in that case.

One scenario to consider: if put the cache in witness plugin, if we code the plugin to track applied operations only, after a block is popped out, the cache may become inconsistent if the popped out block contains a witness_update_operation, when pushing a new block, if there is no the same witness_update_operation in the new block, the cache would still be inconsistent. To solve this, we need additional code to detect if a chain reorganization has occurred and refresh the cache accordingly, if not simply refresh the whole cache on every block. However, when refreshing the cache, we need to fetch data from main chain state, since signalling is a multi-threading thing, the main state could have changed when fetching data, which means the cache is not 100% reliable.

By putting the cache in database class we can guarantee its consistency. Perhaps there will still be edge cases, for example, witness plugin tries to generate a block in the middle when database is pushing a block.

Another thing is we don't have unit test for witness plugin, so adding more code there means more work / risks.

The witness plugin typically needs to keep track of the keys of only one witness. Refreshing this after each pushed block should have minimal cost, and only for the node in question.

I still don't like to refresh from the plugin by querying chain state directly, because chain state may have become dirty already when refreshing. For example, this scenario:

there was a witness_update_operation in _pending_transactions

a block is pushed to database without that operation

that operation got pushed to database soon after finished pushing the block

witness plugin received the "applied_block" signal

witness plugin tries to refresh local cache, but got updated key (dirty data).

That said, ideally we need to keep a clean state or a subset in database for plugins or other modules to use. This PR is trying in this way, IMHO we can merge it now, then refactor it to a more generic module later when necessary.

I think the applied_block signal is processed in sync right after the block has been applied. There shouldn't be any tx's pushed in between.
https://github.com/bitshares/bitshares-core/blob/2.0.180612/libraries/chain/db_block.cpp#L550

Oh, right, I forgot that. The state should be clean when processing applied_block event.

By the way, why we need to sleep here and there in test cases? IIRC sometimes chain_test will segfault without sleeping. E.G.

bitshares-core/tests/tests/market_rounding_tests.cpp

Line 76 in 95906d8

fc::usleep(fc::milliseconds(200)); // sleep a while to execute callback in another thread

I think it's required in some places where API callbacks happen. These are asynchronous.
Shouldn't be necessary in market_rounding_tests IMO.

pmconrad · 2018-08-19T16:06:07Z

libraries/chain/db_block.cpp

   if( !(skip & skip_witness_signature) )
-      FC_ASSERT( witness_obj.signing_key == block_signing_private_key.get_public_key() );
+   {
+      auto signing_key = find_witness_key_from_cache( witness_id );


Instead of using the cache, simply move the existing FC_ASSERT down a couple of lines, after _pending_tx_session.reset();

It's cheaper to check here than checking after called _pending_tx_session.reset();.

This will only hurt the node that is trying to produce without proper keys.

Even for nodes with proper keys, it's still slightly cheaper because the cache would be much smaller than object database.

For nodes with keys it may be cheaper, but the global cache is a penalty for all nodes.

abitmore · 2018-08-29T00:04:19Z

Moved the signing key cache to witness plugin as recommended by @pmconrad.

Resolved conflicts: libraries/chain/db_block.cpp

which means we assume it will be no more than 3 bytes after serialization. 21 bits means at maximum 2^21-1=2,097,151 transactions in a block, it's practically safe although theoretically the size is unsigned_int which can be 56 bits at maximum.

pmconrad

Looks good, thanks! (And much simpler than before, IMO.)

The PR contains several unrelated changes. That's OK, IMO - no need to create separate PRs for minor code improvements
We need to keep this balanced though. Perhaps not quite as many unrelated things in the future.

pmconrad · 2018-08-30T13:25:13Z

libraries/chain/db_block.cpp

+   {
+      // Note: if this check failed (which won't happen in normal situations),
+      // we would have temporarily broken the invariant that
+      // _pending_tx_session is the result of applying _pending_tx.


At this point _pending_tx_session has been cleared, so the comment makes no sense.

Yes, _pending_tx_session has been cleared, but _pending_tx hasn't, so they're inconsistent. That's why I wrote the comment.

Thanks. Never thought of this as an invariant, but I guess it's ok.

pmconrad · 2018-08-30T13:31:01Z

libraries/chain/db_block.cpp

@@ -532,7 +538,17 @@ void database::_apply_block( const signed_block& next_block )
   uint32_t skip = get_node_properties().skip_flags;
   _applied_ops.clear();

-   FC_ASSERT( (skip & skip_merkle_check) || next_block.transaction_merkle_root == next_block.calculate_merkle_root(), "", ("next_block.transaction_merkle_root",next_block.transaction_merkle_root)("calc",next_block.calculate_merkle_root())("next_block",next_block)("id",next_block.id()) );
+   if( !(skip & skip_block_size_check) )


Should at least soft-fork protect this.

pmconrad · 2018-08-30T13:47:51Z

tests/tests/block_tests.cpp

+         // should fail to push if it's too large
+         if( fc::raw::pack_size(maybe_large_block) > gpo.parameters.maximum_block_size )
+         {
+            BOOST_CHECK_THROW( db.push_block(maybe_large_block), fc::exception );


The test should ensure that this is triggered a couple of times. Perhaps increment a counter each time and BOOST_CHECK_EQUAL the counter after the loop ends. (Not sure how often it should trigger...)

Added a check to make sure it has been triggered at least once. I think it's enough. Didn't use BOOST_CHECK_EQUAL because it would be a pain when trying to tweak the loop in the future.

pmconrad · 2018-08-30T19:23:25Z

Ok, I won't insist on the soft fork.

abitmore added 3 commits July 29, 2018 20:35

Test case to reproduce #125 block producing issue

b8bafdd

When signing key is updated in the same block, producing will fail.

Cache witness signing key for block producing #125

ac49b7f

Added comments

37e72ad

abitmore added this to the Future Non-Consensus-Changing Release milestone Jul 29, 2018

abitmore mentioned this pull request Jul 29, 2018

When signing a block that updates the signing witness's signing key, use correct signing key #125

Closed

abitmore modified the milestones: Future Non-Consensus-Changing Release, 201810 - Non-Consensus-Changing Release Jul 29, 2018

abitmore mentioned this pull request Jul 30, 2018

CLI wallet: avoid directly overwriting wallet file on exit #1109 #1195

Merged

Refactor changing key test, try reopen db

f28c6bf

abitmore added 3 commits August 18, 2018 23:56

Initialize witness key cache at correct time

b89bea1

moved `init_witness_key_cache()` call from `plugin_startup()` to `plugin_initialize()`.

Added skip_flags to create_witness test fixture

2563f4b

Added tests for witness_create_op and key cache

bf0d807

abitmore force-pushed the 125-signing-key-check branch from 449745f to bf0d807 Compare August 19, 2018 14:25

pmconrad reviewed Aug 19, 2018

View reviewed changes

abitmore self-assigned this Aug 27, 2018

abitmore added 3 commits August 28, 2018 17:57

Merge remote-tracking branch 'origin/develop' into 125-signing-key-check

24ff443

Moved witness signing key cache to witness plugin

108e6a5

Updated test cases for witness signing key cache

4915156

Merge branch 'develop' into 125-signing-key-check

8336f86

Resolved conflicts: libraries/chain/db_block.cpp

abitmore removed their assignment Aug 29, 2018

abitmore added 6 commits August 29, 2018 09:01

Added test case for block size check

549337e

Respect skip_fork_db when failed to push a block

67166ca

fork_db: update head when removing a block

756f0ef

Updated block_tests to try to push large blocks

e211373

Added more logging about number of trx in a block

1c2bc76

pmconrad suggested changes Aug 30, 2018

View reviewed changes

abitmore added 2 commits August 30, 2018 11:05

More tests about pushing large blocks

dc56019

Updated block_size_test to test fork db correctly

39089ae

pmconrad approved these changes Aug 30, 2018

View reviewed changes

abitmore merged commit 8189b45 into develop Aug 31, 2018

abitmore deleted the 125-signing-key-check branch August 31, 2018 01:30

TheTaconator mentioned this pull request Oct 15, 2018

Release Notes for BitShares Core 201810 #1381

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix issue #125: sign block with old key if signing key updated in same block #1203

Fix issue #125: sign block with old key if signing key updated in same block #1203

abitmore commented Jul 29, 2018 •

edited

Loading

abitmore commented Jul 29, 2018

pmconrad commented Jul 31, 2018

abitmore commented Jul 31, 2018

pmconrad Aug 19, 2018

pmconrad Aug 19, 2018

abitmore Aug 19, 2018

pmconrad Aug 19, 2018

abitmore Aug 19, 2018

pmconrad Aug 20, 2018

abitmore Aug 22, 2018

pmconrad Aug 23, 2018

abitmore Aug 23, 2018

pmconrad Aug 27, 2018

pmconrad Aug 19, 2018

abitmore Aug 19, 2018

pmconrad Aug 19, 2018

abitmore Aug 19, 2018

pmconrad Aug 27, 2018

abitmore commented Aug 29, 2018

pmconrad left a comment

pmconrad Aug 30, 2018

abitmore Aug 30, 2018 •

edited

Loading

pmconrad Aug 30, 2018

pmconrad Aug 30, 2018

pmconrad Aug 30, 2018

abitmore Aug 30, 2018

pmconrad commented Aug 30, 2018

Fix issue #125: sign block with old key if signing key updated in same block #1203

Fix issue #125: sign block with old key if signing key updated in same block #1203

Conversation

abitmore commented Jul 29, 2018 • edited Loading

abitmore commented Jul 29, 2018

pmconrad commented Jul 31, 2018

abitmore commented Jul 31, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abitmore commented Aug 29, 2018

pmconrad left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abitmore Aug 30, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pmconrad commented Aug 30, 2018

abitmore commented Jul 29, 2018 •

edited

Loading

abitmore Aug 30, 2018 •

edited

Loading