[Utility] Local Proof of Stake CLI

[Olshansk]

Objective

Create an entry point to interact with the local proof of stake blockchain actors for testing, development and automation purposes.

Origin Document

In order to test, develop and drive consensus in V1, we developed the Pocket v1 client to send a few simple transactions such as printing node state, triggering next view , etc...

Similar to Tendermint's abci-cli tool, we need to expand the functionality of the pocket-dev-cli to fit our needs for the time being.

Goals / Deliverables

• Determine if the existing client suffices to fit these needs (i.e. passing params for addresses and values)
• Define and document the list of commands to be added (e.g. stake, unstake, pause, unpause, send, etc...)
• Update the node so the request is propagated appropriately
• If needed, create the appropriate interfaces in the utility module (or other) to pass through or mock the request
  Bonus:
• Start rough notes for an RPC spec
• Compile a listen of examples and references of other RPC specs for next steps

General issue checklist

• CLI: Update the appropriate CHANGELOG (tracked in [Utility] CLI Usage instructions and specification #193)
• CLI: Update the README (tracked in [Utility] CLI Usage instructions and specification #193)
• RPC: Update the appropriate CHANGELOG
• RPC: Update the README
• Node: Update the appropriate CHANGELOG
• Node: Update the README
• Update the source code tree explanation
• If applicable, add or update a state, sequence or flowchart diagram using mermaid
• Update any relevant global documentation & references
• Document small issues / TODOs along the way

Non-goals

• Create a fully-fledged RPC specification similar to Tendermint
• Have functional end-to-end utility messages being submitted

Testing Methodology

• Utility specific tests: make test_utility_*
• All tests: make test_all
• LocalNet: verify a LocalNet is still functioning correctly by following the instructions at docs/development/README.md
  Remove

Creator: @Olshansk
Co-Owners: @andrewnguyen22

[deblasis]

Hi there! I'll start having a look at this sometime today if that's OK :)

Goals / Deliverables

1) Determine if the existing client suffices to fit these needs (i.e. passing params for addresses and values)

The existing client provides access to 4 debug commands triggered by "broadcasting" the corresponding message.
utility/proto/message.proto describes the messages that the new/updated utility CLI should be capable of handling and indeed it looks like we should be able to handle inputs in the form of promptui prompts but also we need to handle arguments passed as flags via the CLI for automation and usage without having to wait for a user's input in general.

That said I believe that the current client should be refactored using cobra and that the current behaviour should be made available to the user under a subcommand:
eg:

./client debug

The documentation and also stuff like make client_connect should be updated to point to that subcommand.

Potentially, we could also provide direct access to the current options (ResetToGenesis, PrintNodeState, TriggerNextView, TogglePacemakerMode) via the CLI:

eg:

./client debug ResetToGenesis

I trust your judgement on this. From an implementation cost POV it's quite trivial.

2) Define and document the list of commands to be added (e.g. stake, unstake, pause, unpause, send, etc...)

As I mentioned in 1) I believe that the utility related commands should be the ones that broadcast the corresponding messages in utility/proto/message.proto, therefore (at the time of writing we are at commit 3f42f13):

• Send
• Stake
• EditStake
• Unstake
• Unpause
• ChangeParameter
• DoubleSgn

I have a couple of questions:

1. Curiously I can't find a MessagePause what am I missing?
2. if DoubleSign is in scope, how do you feel about a JSON to describe the Vote type for vote_a and vote_b via the CLI?
3. since the majority of these messages have an ActorType, from an UX perspective we need to define what would be the most correct form, semantically speaking.
   If we want to send an amount of 10 from 0xfromaddress to 0xtoaddress for the actorType App
   we could have

./client utility send --from 0xfromaddress --to 0xtoaddress --amount 10 --actor_type App

or

./client utility App send --from 0xfromaddress --to 0xtoaddress --amount 10

or

./client utility send App --from 0xfromaddress --to 0xtoaddress --amount 10

I am experienced enough to have a preference, so I am looking for your feedback/guidance on the best UX and therefore design here.

That's all for now I guess, I'll wait for your input about this before proceeding any further.

👋 :)

[Olshansk]

1) Determine if the existing client suffices to fit these needs (i.e. passing params for addresses and values)

That said I believe that the current client should be refactored using cobra and that the current behaviour should be made available to the user under a subcommand

Sounds good to me. For reference, the pocket V0 CLI (which forked Tendermint) uses cobra, and ignite cli also uses cobra in its newer version. I think we can use both of those references for other commands we will need in the future.

Reference: RPC spec for v0: https://docs.pokt.network/core/specs/rpc-spec

That said I believe that the current client should be refactored using cobra and that the current behaviour should be made available to the user under a subcommand:

I like this idea. I wanted to "protocol development" CLI be different and easier to use for development (i.e. using promptui) but didn't think through how to have a single entrpoint. Your suggestion of just having ./client debug is 👌.

Potentially, we could also provide direct access to the current options (ResetToGenesis, PrintNodeState, TriggerNextView, TogglePacemakerMode) via the CLI

Seems a bit overkill for now, so let's hold off on that.

2) Define and document the list of commands to be added (e.g. stake, unstake, pause, unpause, send, etc...)

Curiously I can't find a MessagePause what am I missing?

Not missing anything. I feel like it should be added to message.proto because I can't find it there either... cc @andrewnguyen22 for context.

I think the original idea was that nodes can get paused for potentially byzantine behaviour for a short time, but do not get slashed for some number of blocks. Therefore, "pause" is a protocol level state change, but unpause is something the operator does manually.

With that being said, the spec does enable operators to quickly & temporarily pause their nodes.

The following commands will enable us to get a pocket-specific proof-of-stake :

• Send
• Stake
• EditStake
• Unstake
• ChangeParameter
• The one other request I'd make is being able to send a raw transaction (i.e. a serialized proto) which will enable automation as we add more request types. See Pocket v0 app/cmd/cli/accounts.go for reference.

if DoubleSign is in scope, how do you feel about a JSON to describe the Vote type for vote_a and vote_b via the CLI?

Let's keep it out of scope for this specific PR

since the majority of these messages have an ActorType, from an UX perspective we need to define what would be the most correct form, semantically speaking.

I decided to reference ignite and pocket V0:

I would say that semantically utility is something that should be hidden from node operators or app users as it is an implementation-specific module and may change. Here is how I would do it:

Validator specific staking:

./client Validator stake --address 0xval_address_where_priv_key_is_on_file --amount 10

Sending is general to any account:

./client Account send --from 0xval_address_where_priv_key_is_on_file --to 0xtoaddress --amount 10

That's all for now I guess, I'll wait for your input about this before proceeding any further.

Thank you for the amazing questions! I'm also going to sync up offline and set up a time for us to discuss this in more detail, along with @andrewnguyen22, in the #v1-dev channel on discord.

[andrewnguyen22]

Not missing anything.

Checkout /utility/doc/changelog.md

• Removed pause messages and functionality as it is out of scope for the current POS iteration

I think the original idea was that nodes can get paused for potentially byzantine behaviour for a short time, but do not get slashed for some number of blocks. Therefore, "pause" is a protocol level state change, but unpause is something the operator does manually.

Nodes get slashed and then removed from consensus with a 'pause' which in the first iteration will be automatic (like Cosmos). In the subsequent implementation where we add the 'Pocket Specific' stuff, we'll add a manual 'Pause' message for operators who want to temporarily take their node offline without being slashed.

./client Account send --from 0xval_address_where_priv_key_is_on_file --to 0xtoaddress --amount 10

I don't think these flags are necessary. Just make them Cobra parameters

[deblasis]

Thanks for the answers guys!

WIP PR coming soon, I have a non-blocker question:

1. Private key handling:

When you refer to 0xval_address_where_priv_key_is_on_file how do you want me to handle that input? I saw that pocket-core basically uses a keybase that we can query given an address to find the PK but apparently, unless I am blind, there's no similar concept in V1.

[andrewnguyen22]

Hmm we do have a keybase for V1

#150

For now just take it as a --path_to_private_key_file and make the private key plain text json for now

[deblasis]

Hi @andrewnguyen22 and @Olshansk!

Forgive me for the delay, unfortunately, I have been dragged into other things.
This is the first batch of changes in an MVP kind-of fashion, at least I guess it's gonna tell where I am going with this.

Blatantly inspired by V0 and ignite.

Please have a quick look and point out anything that sounds just wrong apart from not implemented yet :)

I understand that this might become a blocker soon so please let me know how you would prefer me to proceed (or not proceed) further.

[Olshansk]

Forgive me for the delay, unfortunately, I have been dragged into other things.
I understand that this might become a blocker soon so please let me know how you would prefer me to proceed (or not proceed) further.

Thanks @deblasis! Going to take a look and leave comments. Also ccing @adshmh who will likely help with the RPC spec and CLI moving forward.

Blatantly inspired by V0 and ignite.

I think that's the right approach here

[deblasis]

Thanks @deblasis! Going to take a look and leave comments. Also ccing @adshmh who will likely help with the RPC spec and CLI moving forward.

Awesome! Thank you @Olshansk! (nice to meet you @adshmh ! )

I am pushing updates here and there to address the TODOs.

A couple of questions (non blockers):

1. Regarding the PrivateKey we should currently read from file, should I read a plain text PrivateKey or should I port encryption and passphrase handling like in V0 (basically what's called ArmoredJson and everything around it?)
2. Since a PrivateKey embeds an address, should I simply ignore the <from> argument in commands when a private key is provided via flag for now?

[Olshansk]

@deblasis

Regarding the PrivateKey we should currently read from file, should I read a plain text PrivateKey or should I port encryption and passphrase handling like in V0 (basically what's called ArmoredJson and everything around it?)

Use a plain text for now. Could you create a separate github issue to make sure we don't forget to use ArmoredJson in the future?

Since a PrivateKey embeds an address, should I simply ignore the argument in commands when a private key is provided via flag for now?

This is heavily dependent on the implementation of the keybase, and I do think that the placeholder + TODO you have in the code so far is the perfect approach to unblock our work. I personally think we should merge it as is to main and iterate once the other dependencies make progress.

---

Are you still making any modifications on the PRs or should we review and tend to the questions / TODOs throughout? It's look like a really solid foundation for what we need 💯

[deblasis]

Use a plain text for now. Could you create a separate github issue to make sure we don't forget to use ArmoredJson in the future?

Done: #184 (please check and provide brutally honest feedback 🙂)

This is heavily dependent on the implementation of the keybase, and I do think that the placeholder + TODO you have in the code so far is the perfect approach to unblock our work. I personally think we should merge it as is to main and iterate once the other dependencies make progress.

Awesome!

Are you still making any modifications on the PRs or should we review and tend to the questions / TODOs throughout? It's look like a really solid foundation for what we need 💯

Yes, I am still making changes, tomorrow you should get another batch of changes. I want to make sure that you guys like the command propagation so I was thinking that I'll probably implement one command end to end, ask for a review and then depending on your feedback adjust and complete all of them.

ADDED: actually, I just read that end to end is a non-goal so I will speed this up for faster iteration, anyway, nothing really changes since my ETA is still today (29/08)

[deblasis]

@Olshansk: I have updated #176 #177 and also #169 which has some extra changes (and also all commits from the other 2 PRs) to basically connect the two parts and provide a complete overview of the changes to address this issue.

In #177 I have only written the "Stake" command and depending on your feedback I'll make the relevant changes and apply them to the other commands as well if necessary (trying to work in an MVP-like way)

[deblasis]

@Olshansk , @andrewnguyen22:

Hi guys, please find below something that probably I should have shared at the beginning not now...

Next time I'll try to write down more words, in the form of specs/research/questions before diving into VSCode.

Please let me know if the list is exhaustive enough and/or if you would prefer this in a different format.

Compile a list of examples and references of other RPC specs for next steps:

• Ethereum - Docs (Spec)

  🏷️ JSON-RPC 2.0 standard, uses OpenRPC.

  🚄 Transport: HTTP API, WebSocket and Unix Domain Sockets.

• Algorand - Docs (Spec)

  🏷️ not JSON-RPC 2.0 but similar approach (arguments are in array form in the body and the endpoint represents the method call), secure (HTTPS + API token).

  🚄 Transport: HTTP API.

• Solana Docs

  🏷️ JSON-RPC 2.0 standard.

  🚄 Transport: HTTP API, PubSub WebSocket.

• Cosmos - Docs

  🏷️ not JSON-RPC 2.0, secure (HTTPS + basic auth).

  🚄 Transport: HTTP API (using gRPC-Gateway with OpenApi generation via protoc plugin), gRPC.

• Polkadot Docs

  🏷️ JSON-RPC 2.0, secure (HTTPS + basic auth), SDK (javascript library), Sidecar to provide HTTP API

  🚄 Transport: HTTP API, WebSockets.

• Avalance Docs

  🏷️ JSON-RPC 2.0, secure (HTTPS + JWT token)

  🚄 Transport: HTTP API.

• Ripple Docs

  🏷️ not JSON-RPC 2.0, secure (HTTPS + JWT token)

  🚄 Transport: HTTP API, WebSockets.

• Monero Docs

  🏷️ JSON-RPC 2.0, secure (HTTPS + basic auth)

  🚄 Transport: HTTP API, WebSockets.

Takeaways

• ✅ It looks like that the ubiquitousness of HTTP APIs is here to stay.

• ❌ The JSON-RPC 2.0 specification is pretty much implemented in the majority of cases and perhaps we should consider doing the same

• ❌ We should start thinking about security as well at some point

• ⁉️ Implementing other transport methods could be important. "Build it and they will come"? It would be probably nice to ask node operators from V0 for what they feel like is missing in the RPC and interaction with the nodes in general.

    Guide
    ❌ | ⁉️ = potential action item for next steps
  

[Olshansk]

Next time I'll try to write down more words, in the form of specs/research/questions before diving into VSCode

@andrewnguyen22 When are you going to join the VSCode club?

@deblasis This summary & takeaways are amazing! We should definitely review these in a call as a team and capture it somewhere else. Perhaps in a doc alongside the readme or in a github discussion (https://github.com/pokt-network/pocket/discussions) so we can continue the research/conversation even after this PR is closed.