Integrate State Proof keys #2990

id-ms · 2021-10-04T08:35:06Z

Summary

This PR adds the support for the State Proof Keys.

Test Plan

Unit tests added.

cmd/goal/account.go

crypto/hashes.go

crypto/sig_abstractions.go

codecov-commenter · 2021-10-04T13:16:46Z

Codecov Report

Merging #2990 (617f69f) into master (c8c6fb7) will increase coverage by 0.31%.
The diff coverage is 52.73%.

@@            Coverage Diff             @@
##           master    #2990      +/-   ##
==========================================
+ Coverage   47.74%   48.06%   +0.31%     
==========================================
  Files         370      381      +11     
  Lines       60262    62061    +1799     
==========================================
+ Hits        28774    29829    +1055     
- Misses      28180    28815     +635     
- Partials     3308     3417     +109

Impacted Files	Coverage Δ
agreement/abstractions.go	`50.00% <ø> (ø)`
cmd/goal/account.go	`12.88% <0.00%> (-0.24%)`	⬇️
compactcert/db.go	`73.68% <ø> (ø)`
compactcert/worker.go	`90.00% <ø> (ø)`
crypto/digest.go	`0.00% <0.00%> (ø)`
crypto/merklearray/worker.go	`100.00% <ø> (ø)`
daemon/algod/api/server/v1/handlers/handlers.go	`0.62% <0.00%> (ø)`
daemon/algod/api/server/v2/account.go	`77.55% <0.00%> (-0.80%)`	⬇️
daemon/algod/api/server/v2/handlers.go	`0.00% <0.00%> (ø)`
data/accountManager.go	`0.00% <0.00%> (ø)`
... and 78 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c8c6fb7...617f69f. Read the comment docs.

data/account/participation.go

data/transactions/transaction.go

shiqizng · 2021-10-07T18:28:10Z

data/account/participation.go

@@ -121,6 +130,11 @@ func (part Participation) GenerateRegistrationTransaction(fee basics.MicroAlgos,
 			SelectionPK: part.VRF.PK,
 		},
 	}
+	if cert := part.BlockProofSigner(); cert != nil {
+		if cparams.EnableBlockProofKeyregCheck {


what happens when !cparams.EnableBlockProofKeyregCheck and cert != nil?

Nothing I guess. The BlockProof key will not be registered in the transaction since it's not yet supported on this protocol version.

crypto/merklekeystore/keystore.go

test/e2e-go/upgrades/stateproof_test.go

crypto/util.go

data/accountManager.go

data/bookkeeping/txn_merkle_test.go

ledger/compactcert_test.go

crypto/merklearray/array.go

crypto/curve25519_wrapper.go

crypto/digest.go

crypto/sig_abstractions.go

ledger/eval.go

ledger/voters.go

crypto/merklekeystore/keystore.go

shiqizng · 2021-10-11T19:45:03Z

crypto/merklearray/merkle.go

 		if err != nil {
 			return nil, err
 		}
 	}

 	// Confirm that we got the same root hash
 	if len(pl) != 1 {
-		return nil, fmt.Errorf("internal error: partial layer produced %d hashes", len(pl))
+		return nil, fmt.Errorf("internal error: partial Layer produced %d hashes", len(pl))
 	}

 	if validateProof {


validateProof is a constant set as false at line 135, when will this block if executed?

It will be executed when the feature is enabled. This should definitely change though, you are right.
Probably the a global configuration value needs to replace it, or perhaps some flag from the protocol params.
Will check on that.

data/account/participation.go

shiqizng · 2021-10-11T20:44:09Z

gen/walletData.go

@@ -28,7 +28,7 @@ import (
 // instance (because we have no ctors...)
 var DefaultGenesis = GenesisData{
 	FirstPartKeyRound: 0,
-	LastPartKeyRound:  3000000,
+	LastPartKeyRound:  3000,


reason for this change?

Temporary for testing reasons (3,000,000 fails in the CICD tests and it seems excessive).

libgoal/participation.go

shiqizng · 2021-10-12T14:38:48Z

crypto/compactcert/verifier.go

@@ -48,26 +47,29 @@ func (v *Verifier) Verify(c *Cert) error {
 		return fmt.Errorf("cert signed weight %d <= proven weight %d", c.SignedWeight, v.ProvenWeight)
 	}



is there a unit test for this method?

There should be in the future (when the compactcert feature is ready to be released), but as for now it is tested by compactcert.TestWorkerAllSigs() and compactcert.TestWorkerPartialSigs().

ledger/accountdb_test.go

shiqizng · 2021-10-12T19:33:47Z

ledger/compactcert.go

-		SecKQ:        proto.CompactCertSecKQ,
+		Msg:               hdr,
+		ProvenWeight:      provenWeight,
+		SigRound:          hdr.Round,


Why is the sigRound param changed? Does it impact the existing implementation?

Yes, in the old implementation the voting key was used to sign the compact cert (and the commitment was to the next round key , since the ephemeral voting key is disposed after use). Now we use completely different keys and scheme so this hack is no longer needed.

…orand/go-algorand into feature/dilithium-scheme-integration

@jannotti

…gression ## Summary My previous PR was causing a minor regression. Thanks to @jannotti comment, I think that I was able to fix the regression, and added a proper testing for that. ## Test Plan New benchmark result ( no inner method anymore ) ``` goos: darwin goarch: amd64 pkg: github.com/algorand/go-algorand/data/bookkeeping cpu: Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz BenchmarkTxnMerkleToRaw/copy-8 1000000000 1.129 ns/op 0 B/op 0 allocs/op BenchmarkTxnMerkleToRaw/append-8 243079730 4.886 ns/op 0 B/op 0 allocs/op PASS ok github.com/algorand/go-algorand/data/bookkeeping 3.183s ```

…ntest

Co-authored-by: Will Winder <[email protected]>

## Summary Instead of performing a check in `SignerContext.GetVerifier`, I've added a unit test to verify that the output of the merklearray root has the same length as the Verifier object. ## Test Plan Unit test.

regression was rolled back.

## Summary Create an update path that would enable the following features: 1. Batch Verification (#3031) 2. State Proof Keys (#2990) 3. TEAL 6 (#3397) 4. Expired Participation Keys (#2924) 5. Fix rewards calculation bug (#3403) ## Test Plan Unit tests added.

id-ms changed the title ~~[WIP] State proof integration~~ [WIP] dilithium-scheme-integration Oct 4, 2021

id-ms commented Oct 4, 2021

View reviewed changes

cmd/goal/account.go Outdated Show resolved Hide resolved

id-ms commented Oct 4, 2021

View reviewed changes

crypto/hashes.go Outdated Show resolved Hide resolved

id-ms commented Oct 4, 2021

View reviewed changes

crypto/sig_abstractions.go Outdated Show resolved Hide resolved

shiqizng reviewed Oct 7, 2021

View reviewed changes

algonautshant reviewed Oct 8, 2021

View reviewed changes

test/e2e-go/upgrades/stateproof_test.go Outdated Show resolved Hide resolved

algonautshant reviewed Oct 8, 2021

View reviewed changes

test/e2e-go/upgrades/stateproof_test.go Show resolved Hide resolved

algonautshant reviewed Oct 8, 2021

View reviewed changes

shiqizng reviewed Oct 11, 2021

View reviewed changes

crypto/merklekeystore/keystore.go Outdated Show resolved Hide resolved

shiqizng reviewed Oct 11, 2021

View reviewed changes

data/account/participation.go Outdated Show resolved Hide resolved

shiqizng reviewed Oct 11, 2021

View reviewed changes

libgoal/participation.go Outdated Show resolved Hide resolved

shiqizng reviewed Oct 12, 2021

View reviewed changes

jasonpaulos self-requested a review October 12, 2021 15:01

shiqizng reviewed Oct 12, 2021

View reviewed changes

ledger/accountdb_test.go Outdated Show resolved Hide resolved

shiqizng reviewed Oct 12, 2021

View reviewed changes

algonautshant force-pushed the feature/dilithium-scheme-integration branch from 04dd916 to c96eb57 Compare October 13, 2021 21:46

algonathan added 11 commits October 14, 2021 15:16

WIP: adding round commitment to the merklekeystore keys

c9c3d03

added const to EphemeralPublicKey ToBeHashed

4805692

extended unit tests

8de8ebd

fix: 'make sanity'

b0d8e51

api change

8e75eaf

fix: lexicographic order

994d938

fix: import style convention

0f741fa

removed unused variable

6020752

changed name of ephemralpk

0625577

fix: msgpack for merklekeystore.Verifier

238f10c

changed signature test in merklekeystore

2b5f0b1

Merge branch 'feature/dilithium-scheme-integration' of github.com:alg…

0d35440

…orand/go-algorand into feature/dilithium-scheme-integration

tsachiherman force-pushed the feature/dilithium-scheme-integration branch from 0d35440 to bd7fcc8 Compare February 9, 2022 19:22

tsachiherman and others added 10 commits February 9, 2022 15:00

fix broken test

aea9374

make test consts non exported

7db102d

update e2e_subs test

ed6efa5

changes

674688d

Merge branch 'feature/dilithium-scheme-integration' into tsachi/sigle…

005e3f1

…ntest

replace error strings with error type

01fdce0

Update crypto/merklearray/merkle.go

617f69f

Co-authored-by: Will Winder <[email protected]>

update per peer review

74d31b9

Merge pull request #3609 from tsachiherman/tsachi/siglentest

8d10050

## Summary Instead of performing a check in `SignerContext.GetVerifier`, I've added a unit test to verify that the output of the merklearray root has the same length as the Verifier object. ## Test Plan Unit test.

tsachiherman previously approved these changes Feb 10, 2022

View reviewed changes

reuse code in GetStateProofForRound

268ee6d

Aharonee dismissed tsachiherman’s stale review via 268ee6d February 10, 2022 15:27

Aharonee added 5 commits February 10, 2022 17:29

docstring

e4302da

rename

1dc185c

replace error strings with type

3fa00c3

newline for goal output

9cc0800

newline for goal output

83f96ab

tsachiherman approved these changes Feb 10, 2022

View reviewed changes

tsachiherman merged commit 1504f5f into master Feb 10, 2022

tsachiherman mentioned this pull request Feb 10, 2022

V31 consensus upgrade #3553

Merged

onetechnical mentioned this pull request Feb 11, 2022

go-algorand 3.4.0-beta #3613

Merged

egieseke mentioned this pull request Feb 16, 2022

Script to generate go-algorand release notes #3520

Closed

michaeldiamant mentioned this pull request Feb 17, 2022

Support TEAL 6 txn fields LastLog, StateProofPK and opcodes divw, itxnas, gitxnas algorand/pyteal#174

Merged

onetechnical mentioned this pull request Mar 1, 2022

go-algorand 3.4.2-stable #3700

Merged

algorandskiy deleted the feature/dilithium-scheme-integration branch November 3, 2022 15:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate State Proof keys #2990

Integrate State Proof keys #2990

id-ms commented Oct 4, 2021 •

edited by tsachiherman

Loading

codecov-commenter commented Oct 4, 2021 •

edited

Loading

shiqizng Oct 7, 2021

Aharonee Oct 21, 2021

shiqizng Oct 11, 2021

Aharonee Oct 25, 2021

shiqizng Oct 11, 2021

Aharonee Oct 20, 2021

shiqizng Oct 12, 2021

Aharonee Oct 27, 2021

shiqizng Oct 12, 2021

Aharonee Oct 27, 2021 •

edited

Loading

		@@ -48,26 +47,29 @@ func (v Verifier) Verify(c Cert) error {
		return fmt.Errorf("cert signed weight %d <= proven weight %d", c.SignedWeight, v.ProvenWeight)
		}

Integrate State Proof keys #2990

Integrate State Proof keys #2990

Conversation

id-ms commented Oct 4, 2021 • edited by tsachiherman Loading

Summary

Test Plan

codecov-commenter commented Oct 4, 2021 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Aharonee Oct 27, 2021 • edited Loading

Choose a reason for hiding this comment

id-ms commented Oct 4, 2021 •

edited by tsachiherman

Loading

codecov-commenter commented Oct 4, 2021 •

edited

Loading

Aharonee Oct 27, 2021 •

edited

Loading