Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Persistence] Enable multiple persistence contexts on PostgresContext #128

Closed
9 tasks
andrewnguyen22 opened this issue Jul 27, 2022 · 0 comments · Fixed by #140
Closed
9 tasks

[Persistence] Enable multiple persistence contexts on PostgresContext #128

andrewnguyen22 opened this issue Jul 27, 2022 · 0 comments · Fixed by #140
Assignees
@Olshansk @andrewnguyen22
Labels
core Core infrastructure - protocol related persistence Persistence specific changes
Milestone
V1 Persistence Fo...

Comments

@andrewnguyen22
Copy link
Contributor

Objective

Enable the lifecycle of block processing and production by creating a transactional architecture around persistence contexts instead of writing directly to the database on every operation.

Origin Document

Background

Multiple persistence contexts enable a proper lifecycle for validators and fullnodes as they maintain 'ephemeral' states in situations like block validation/processing and block production.

Goals / Deliverables

  • Deliver completed code
  • Document the architecture and intended use
  • Test the transactional nature within the persistence module

General milestone checklist

  • Update all the relevant CHANGELOGs
  • Update all the relevant READMEs
  • Update the source code tree explanation
  • Add or update a state, sequence or flowchart diagram using mermaid
  • Create followup milestones + issues
  • Document small TODO along the way

Creator: @andrewnguyen22
@andrewnguyen22 andrewnguyen22 added the core Core infrastructure - protocol related label Jul 27, 2022
@andrewnguyen22 andrewnguyen22 self-assigned this Jul 27, 2022
@andrewnguyen22 andrewnguyen22 changed the title [Persistence] Enable multiple persistence contexts [Persistence] Enable multiple persistence contexts on PostgresContext Jul 27, 2022
This was referenced Jul 28, 2022
Merged
Merged
@Olshansk Olshansk self-assigned this Aug 3, 2022
@Olshansk Olshansk added persistence Persistence specific changes priority:high labels Aug 3, 2022
@Olshansk Olshansk moved this to In Progress in V1 Dashboard Aug 3, 2022
@andrewnguyen22 andrewnguyen22 mentioned this issue Aug 4, 2022
Closed
10 tasks
Olshansk added a commit that referenced this issue Aug 8, 2022
@Olshansk
Merge branch 'main' into issue-#128
9bebb69
andrewnguyen22 pushed a commit that referenced this issue Aug 10, 2022
Merge remote-tracking branch 'origin/main' into issue-#128
8dbb9e0
@andrewnguyen22 andrewnguyen22 mentioned this issue Aug 14, 2022
Merged
13 tasks
andrewnguyen22 pushed a commit that referenced this issue Aug 14, 2022
Merge remote-tracking branch 'origin/main' into issue-#128
2d84eef
andrewnguyen22 pushed a commit that referenced this issue Aug 14, 2022
Merge branch 'issue-#128' into issue-#154
7fc6dab
This was referenced Aug 14, 2022
Merged
Merged
andrewnguyen22 pushed a commit that referenced this issue Aug 15, 2022
@Olshansk
[Persistence] PrePersistence deprecation & Block lifecycle improvemen…
4358d64 
…ts (Issue #128 && Issue #105) (#140)

# Objective
closes #128 
- Enable the lifecycle of `block` processing and production by creating a transactional architecture around persistence contexts instead of writing directly to the database on every operation.

closes #105  and closes #104
- Deprecate and remove all `PrePersistence` related code.

# Origin Document
### Background

#128
_Multiple persistence contexts enable a proper lifecycle for validators and full nodes as they maintain 'ephemeral' states in situations like block validation/processing and block production._

Pretty early on - it was discovered that though ephemeral states are needed, there seems to be little validity in having multiple `write` contexts in parallel - rather enable write contexts sequentially (only one at a time) and enable multiple read contexts. This is currently not strictly enforced, rather needs discussion on where to delegate this responsibility (Consensus, Utility, or Persistence module).

#105 
[[Persistence] V1 Persistence Foundation Integration Issue](#104)

# Goals / Deliverables
- [x] Deliver completed code
- [ ] Document the architecture and intended use
- [x] Test the transactional nature within the persistence module
- [x] Deprecation of the PrePersistence Module and all related code, docs, resources, etc...

## General milestone checklist
- [x] Update all the relevant CHANGELOGs
- [x] Update all the relevant READMEs
- [ ] Update the source code tree explanation
- [ ] Add or update a state, sequence or flowchart diagram using [mermaid](https://mermaid-js.github.io/mermaid/)
- [x] Create followup milestones + issues
- [x] Document small TODO along the way

## Follow-up Work

All follow-up work is being tracked in #149

---

## Checklist
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have tested my changes using the available tooling
- [x] If applicable, I have made corresponding changes to related or global README
- [ ] If applicable, I have added added new diagrams using [mermaid.js](https://mermaid-js.github.io)
- [x] If applicable, I have added tests that prove my fix is effective or that my feature works

--- 

Co-authored-by: Daniel Olshansky <[email protected]>
Co-authored-by: Andrew Nguyen <[email protected]>
Co-authored-by: Andrew Nguyen <[email protected]>
Co-authored-by: Daniel Olshansky <[email protected]>
Repository owner moved this from In Progress to Done in V1 Dashboard Aug 15, 2022
Olshansk added a commit that referenced this issue Aug 24, 2022
@Olshansk
[Configuration] Genesis and Node Configurations (Issue#154) (#166)
eb9d94a 
#### Intention
Currently merge ready with #128 but not considering #165 since it's under active development.

## Description
- Deprecated old placeholder genesis_state and genesis_config
- Added utility_genesis_state to genesis_state
- Added consensus_genesis_state to genesis_state
- Added genesis_time to consensus_genesis_state
- Added chainID to consensus_genesis_state
- Added max_block_bytes to consensus_genesis_state
- Added accounts and pools to utility_genesis_state
- Added validators to utility_genesis_state
- Added applications to utility_genesis_state
- Added service_nodes to utility_genesis_state
- Added fishermen to utility_genesis_state
- Deprecated shared/config/
- Added new shared config proto3 structure
- Added base_config to config
- Added utility_config to config
- Added consensus_config to config
- Added persistence_config to config
- Added p2p_config to config
- Added telemetry_config to config
- Opened followup issue #163
- Added config and genesis generator to build package
- Deprecated old build files
- Use new config and genesis files for make compose_and_watch 
- Use new config and genesis files for make client_start && make client_connect

closes #154

## Type of change
Please mark the options that are relevant.

- [x] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] Documentation
- [ ] Other (<_REPLACE_ME_WITH_DETAILS_>)

## How Has This Been Tested?
make test_all
&&
make compose_and_watch

- [x] `make test_all`
- [x] [LocalNet](https://github.com/pokt-network/pocket/blob/main/docs/development/README.md)

## Checklist
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have tested my changes using the available tooling
- [x] If applicable, I have made corresponding changes to related or global README
- [ ] If applicable, I have added added new diagrams using [mermaid.js](https://mermaid-js.github.io)
- [ ] If applicable, I have added tests that prove my fix is effective or that my feature works


--- 

Co-authored-by: Daniel Olshansky <[email protected]>
Co-authored-by: Andrew Nguyen <[email protected]>
Co-authored-by: Andrew Nguyen <[email protected]>
Co-authored-by: Daniel Olshansky <[email protected]>
This was referenced Sep 12, 2022
Closed
Closed
Closed
Closed
Closed
@andrewnguyen22 andrewnguyen22 mentioned this issue Oct 19, 2022
Merged
17 tasks
andrewnguyen22 pushed a commit that referenced this issue Oct 19, 2022
issue-#210 (#312)
714df28 
## Description

A general cleanup issue is needed to tackle TODO's and ensure the persistence module is usable/readable by consolidating `Actor` with `BaseActor` as part of #172. 

Follows issue-#128, issue-#105, issue-#147 and issue-#148 the persistence module is messier and more difficult to understand than the developers would want for organic external contribution.

## Issue

Fixes #210 

## Type of change

Please mark the relevant option(s):

- [ ] New feature, functionality or library
- [ ] Bug fix
- [x] Code health or cleanup
- [ ] Major breaking change
- [ ] Documentation
- [ ] Other <!-- add details here if it a different type of change -->

## List of changes

- [x] Consolidate `Actor` in `persistence/schema/base_actor.go` with `BaseActor` in `persistence/schema/base_actor.go`.


## Testing

- [x] `make develop_test`
- [x] [LocalNet](https://github.com/pokt-network/pocket/blob/main/docs/development/README.md) w/ all of the steps outlined in the `README`

## Required Checklist

- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have tested my changes using the available tooling
- [x] I have updated the corresponding CHANGELOG

### If Applicable Checklist
- [x] I have updated the corresponding README(s); local and/or global
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] I have added, or updated, [mermaid.js](https://mermaid-js.github.io) diagrams in the corresponding README(s)
- [ ] I have added, or updated, documentation and [mermaid.js](https://mermaid-js.github.io) diagrams in `shared/docs/*` if I updated `shared/*`README(s)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Olshansk Olshansk

@andrewnguyen22 andrewnguyen22

Labels
core Core infrastructure - protocol related persistence Persistence specific changes
Projects
V1 Dashboard
Status: Done
Milestone
V1 Persistence Foundation
2 participants
@Olshansk @andrewnguyen22