feat: add total reward for epoch #204

neocybereth · 2024-12-13T02:28:34Z

No description provided.

Fraccaman · 2024-12-13T08:44:17Z

this looks nice! thanks for the PR. You are missing a migration to add the epoch field to the rewards table. You can check it here https://diesel.rs/guides/getting-started#setup-diesel-for-your-project

Fraccaman · 2024-12-16T10:10:45Z

Ehi! you are still missing the migration, you need to add a folder here https://github.com/anoma/namada-indexer/tree/main/orm/migrations (you can follow the guide https://diesel.rs/guides/getting-started#setup-diesel-for-your-project to auto generate the folder for you). The migration is needed to modify the rewards table schema and add the epoch field

neocybereth · 2024-12-16T20:09:06Z

Oof! Sorry about that, coming right up!

I think this should do it:

orm/migrations/2024-12-16-231256_add_epoch_pos_rewards_table/up.sql

rewards/src/schema.rs

shared/src/balance.rs

Fraccaman · 2024-12-23T15:32:20Z

you might also want to update the webserver? I mean, update/create a new endpoint

orm/src/schema.rs

joel-u410 · 2024-12-23T22:43:41Z

orm/migrations/2024-12-16-231256_add_epoch_pos_rewards_table/up.sql

+ALTER TABLE pos_rewards ADD COLUMN epoch INTEGER NOT NULL DEFAULT 0;
+-- Now we can safely drop the default
+ALTER TABLE pos_rewards ALTER COLUMN epoch DROP DEFAULT;


While testing this, I ran into a problem where the pos_rewards table didn't have an updated UNIQUE constraint which incorporates the epoch column.

I pushed a commit that fixes that: 725f9c9

Feel free to add it to your branch also.

Updated that here:

1244bab

joel-u410 · 2024-12-24T13:42:14Z

orm/src/pos_rewards.rs

            validator_id,
+            raw_amount: BigDecimal::from(reward.amount),


Note - this is assigning to raw_amount the decimal value of nam (i.e. 12.34567 not 1234567 unam) -- which is different than the previous behavior of the rewards crawler. I noticed it in my testing because the raw_amount column has type numeric(78,0) which only stores integer values, and the reward amounts were being truncated.

0957a4a

Fixed here with:

raw_amount: BigDecimal::from_str(&reward.amount.to_string()).unwrap(),

neocybereth · 2024-12-30T23:25:52Z

you might also want to update the webserver? I mean, update/create a new endpoint

fae5e1f all the webserver endpoints are in.

basically it takes a delegator address, validator address and epoch and gives back the rewards. The validator address is used to find a validator id to look up the reward. Again, my rust sucks so let me know if there's anything off.

brentstone · 2025-01-02T06:03:37Z

Does this PR depend on anoma/namada#4196?

neocybereth · 2025-01-02T23:51:28Z

Does this PR depend on anoma/namada#4196?

I'm 99% sure yes, but I'm also a noob to this codebase so @joel-u410 or @Fraccaman could confirm for sure.

joel-u410 · 2025-01-06T17:05:18Z

Does this PR depend on anoma/namada#4196?

Hi @brentstone @neocybereth -- no, currently it does not, but after anoma/namada#4196 is merged I would like this one to be updated to take advantage of it.

Right now, in this branch, it only queries "current epoch" rewards, and just assigns the current epoch number to their records -- after anoma/namada#4196 is merged I would want to change this to use the epoch argument to be more precise about which epoch is being requested when rewards are indexed.

(I have a Draft commit where I've started working on that, including adding support for --backfill-from to rewards -- see #193 for context on the --backfill-from feature)

joel-u410 · 2025-01-09T23:43:47Z

Just realized something here ... delete_claimed_rewards is now going to delete things it shouldn't.

Instead of deleting, it should probably insert rows with raw_amount = 0 to indicate they've been claimed.

Which will need to be distinguished somehow from the pos_rewards entry in the same epoch.

We'll probably need to add a column -- either block_height perhaps (to indicate the block where claim occurred) or maybe just a boolean claimed column would work.

joel-u410 · 2025-01-10T01:04:54Z

@neocybereth I'm looking at the existing /pos/reward/:address endpoint and it also needs to be updated to return only the latest reward -- right now it's returning every reward from every epoch in a giant list 😁

neocybereth · 2025-01-10T12:03:29Z

@joel-u410 epic, tahnk you for the heads up! What's your go to testing methodology here? I'm just shooting in the dark and YOLO-ing atm and surely there's gotta be a better way :D

neocybereth · 2025-01-11T22:59:41Z

Does this PR depend on anoma/namada#4196?

Hi @brentstone @neocybereth -- no, currently it does not, but after anoma/namada#4196 is merged I would like this one to be updated to take advantage of it.

Right now, in this branch, it only queries "current epoch" rewards, and just assigns the current epoch number to their records -- after anoma/namada#4196 is merged I would want to change this to use the epoch argument to be more precise about which epoch is being requested when rewards are indexed.

(I have a Draft commit where I've started working on that, including adding support for --backfill-from to rewards -- see #193 for context on the --backfill-from feature)

@joel-u410 @brentstone any idea when [anoma/namada#4196] will get merged in?

joel-u410 · 2025-01-13T16:32:20Z

resolved this one here: b129655

In my fork, for now, I've added this commit which fixes the /pos/reward/:address API route to restore its original behavior of returning only the current/latest reward entry.

I don't think your commit will work unless you also change the route in webserver/src/app.rs to be /pos/reward/:address/:epoch -- but that would be a breaking change to the API, and would require the caller to specify an epoch (which in most cases they probably won't want to do).

Perhaps though a 2nd endpoint, or optional epoch parameter, would be useful for (optionally) querying for past epochs. But I think the default should remain as "query latest epoch" if the user doesn't specify.

joel-u410 · 2025-01-13T17:04:41Z

@joel-u410 @brentstone any idea when [anoma/namada#4196] will get merged in?

I still need to write some unit tests for it, then I think it should be ready to merge -- pending review of course.

neocybereth · 2025-01-14T00:22:28Z

resolved this one here: b129655

In my fork, for now, I've added this commit which fixes the /pos/reward/:address API route to restore its original behavior of returning only the current/latest reward entry.

I don't think your commit will work unless you also change the route in webserver/src/app.rs to be /pos/reward/:address/:epoch -- but that would be a breaking change to the API, and would require the caller to specify an epoch (which in most cases they probably won't want to do).

Perhaps though a 2nd endpoint, or optional epoch parameter, would be useful for (optionally) querying for past epochs. But I think the default should remain as "query latest epoch" if the user doesn't specify.

Nice, I updated that here: d36ec9e to match your commit

Fraccaman · 2025-01-29T10:51:02Z

This looks very cool! I was wondering, do you think we could add a epoch query parameter to the v1/pos/reward/<address> route ?

joel-u410 · 2025-01-29T22:03:16Z

This looks very cool! I was wondering, do you think we could add a epoch query parameter to the v1/pos/reward/<address> route ?

@Fraccaman I took a shot at adding that in this commit. I haven't tested it however.

If this looks good, @neocybereth feel free to cherry-pick it into your branch here.

neocybereth · 2025-01-30T01:21:12Z

Sounds good. Let me know if that's all good @Fraccaman and I'll cherry pick it in.

neocybereth · 2025-02-04T00:25:48Z

Alright I added in 68d63cf
in my latest commit 7320744 and also fixed all merge conflicts

neocybereth · 2025-02-04T00:37:47Z

@joel-u410 I guess now that anoma/namada#4196 is merged this one needs to be updated to take advantage of it like you mentioned above?

If so, what in particular do you want done to this PR to update it accordingly?

joel-u410 · 2025-02-04T16:12:44Z

@joel-u410 I guess now that anoma/namada#4196 is merged this one needs to be updated to take advantage of it like you mentioned above?

If so, what in particular do you want done to this PR to update it accordingly?

I think this should be merged as-is, then we can follow up later to update it with the new functionality from anoma/namada#4196. That requires the RPC node to be updated in order to take advantage, so we'll probably need to wait till the next namada release for it anyway.

neocybereth · 2025-02-05T08:08:01Z

Roger that, thank you @joel-u410. @Fraccaman this is ready for a review when you next get a chance 🙏

Fraccaman · 2025-02-17T11:29:47Z

I think this needs to be rebased 1.0.0-maint

neocybereth · 2025-02-17T21:29:21Z

Alright @Fraccaman I rebased to 1.0.0-maint but it doesn't seem right to me. @joel-u410 the changes left when comparign to 1.0.0-maint after rebase seem like many more files than I've touched, were there any additionals that you're seeing here that seem out of whack as well?

Fraccaman · 2025-02-19T13:41:58Z

something is messed up in here :S

neocybereth · 2025-02-19T22:41:20Z

something is messed up in here :S

Yeaaaaa it was my rebase. I'm going to have to go through and piece the PR back together.

sug0 · 2025-02-19T23:30:31Z

do you still have the old tip of your branch in your git reflog? it would be easier to take that as a starting point

neocybereth · 2025-02-21T03:43:59Z

do you still have the old tip of your branch in your git reflog? it would be easier to take that as a starting point

omg you're a lifesaver!

neocybereth · 2025-02-21T04:08:46Z

@Fraccaman okay, we're back to clean. Everythings up-to-date against main.

Anything else I need to do before getting approval? Shall I attempt another (hopefully successful :D) rebase against 1.0.0-maint from here?

Fraccaman · 2025-02-21T11:15:19Z

@neocybereth can you take a look at. theci failures?

feat: add total reward for epoch

a3040ff

neocybereth added 2 commits December 14, 2024 11:15

fix: add migration to add epoch field to rewwrds table

523a368

fix: restore sample_env

5030f97

neocybereth added 4 commits December 17, 2024 12:08

fix: add rewards migrations

3540112

fix: add pos_rewards table

90fecd6

fix: update migrations

2146524

fix: update migration

59726d0

joel-u410 reviewed Dec 17, 2024

View reviewed changes

orm/migrations/2024-12-16-231256_add_epoch_pos_rewards_table/up.sql Outdated Show resolved Hide resolved

rewards/src/schema.rs Outdated Show resolved Hide resolved

shared/src/balance.rs Outdated Show resolved Hide resolved

fix: update accoridng to review

149ed55

ValidityOps-CLU mentioned this pull request Dec 18, 2024

Merge 1.0.0-maint into main #226

Merged

joel-u410 reviewed Dec 23, 2024

View reviewed changes

orm/src/schema.rs Show resolved Hide resolved

joel-u410 mentioned this pull request Dec 23, 2024

feat: backfill epoch rewards #235

Draft

joel-u410 reviewed Dec 23, 2024

View reviewed changes

joel-u410 reviewed Dec 24, 2024

View reviewed changes

neocybereth added 5 commits December 31, 2024 08:57

fix: formatting on schema

92e0aa7

fix: update rewards epoch func

c61a2e6

fix: orm pos_rewards fix for bigdecimal

0957a4a

fix: update sql statements

1244bab

fix: add webserver endpoint

fae5e1f

fix: update according to joels commit 02ada12

d36ec9e

fix: remove unnecessary type

bae0188

fix: add in 68d63cf

7320744

neocybereth requested a review from joel-u410 February 4, 2025 00:35

neocybereth force-pushed the feat/add-total-reward-for-epoch branch from 9b76f1d to 928d0e0 Compare February 17, 2025 20:32

neocybereth changed the base branch from main to 1.0.0-maint February 17, 2025 20:33

neocybereth changed the base branch from 1.0.0-maint to main February 17, 2025 21:48

neocybereth changed the base branch from main to 1.0.0-maint February 17, 2025 21:49

neocybereth force-pushed the feat/add-total-reward-for-epoch branch from 6d45623 to 7320744 Compare February 21, 2025 04:01

neocybereth changed the base branch from 1.0.0-maint to main February 21, 2025 04:02

fix: merge

c393c22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add total reward for epoch #204

feat: add total reward for epoch #204

neocybereth commented Dec 13, 2024

Fraccaman commented Dec 13, 2024

Fraccaman commented Dec 16, 2024

neocybereth commented Dec 16, 2024 •

edited

Loading

Fraccaman commented Dec 23, 2024

joel-u410 Dec 23, 2024

neocybereth Dec 30, 2024

joel-u410 Dec 24, 2024

neocybereth Dec 30, 2024

neocybereth commented Dec 30, 2024 •

edited

Loading

brentstone commented Jan 2, 2025 •

edited

Loading

neocybereth commented Jan 2, 2025

joel-u410 commented Jan 6, 2025 •

edited

Loading

joel-u410 commented Jan 9, 2025 •

edited

Loading

joel-u410 commented Jan 10, 2025

neocybereth commented Jan 10, 2025

neocybereth commented Jan 11, 2025

joel-u410 commented Jan 13, 2025

joel-u410 commented Jan 13, 2025

neocybereth commented Jan 14, 2025

Fraccaman commented Jan 29, 2025 •

edited

Loading

joel-u410 commented Jan 29, 2025

neocybereth commented Jan 30, 2025

neocybereth commented Feb 4, 2025 •

edited

Loading

neocybereth commented Feb 4, 2025

joel-u410 commented Feb 4, 2025 •

edited

Loading

neocybereth commented Feb 5, 2025 •

edited

Loading

Fraccaman commented Feb 17, 2025

neocybereth commented Feb 17, 2025 •

edited

Loading

Fraccaman commented Feb 19, 2025

neocybereth commented Feb 19, 2025

sug0 commented Feb 19, 2025

neocybereth commented Feb 21, 2025

neocybereth commented Feb 21, 2025

Fraccaman commented Feb 21, 2025

feat: add total reward for epoch #204

Are you sure you want to change the base?

feat: add total reward for epoch #204

Conversation

neocybereth commented Dec 13, 2024

Fraccaman commented Dec 13, 2024

Fraccaman commented Dec 16, 2024

neocybereth commented Dec 16, 2024 • edited Loading

Fraccaman commented Dec 23, 2024

joel-u410 Dec 23, 2024

Choose a reason for hiding this comment

neocybereth Dec 30, 2024

Choose a reason for hiding this comment

joel-u410 Dec 24, 2024

Choose a reason for hiding this comment

neocybereth Dec 30, 2024

Choose a reason for hiding this comment

neocybereth commented Dec 30, 2024 • edited Loading

brentstone commented Jan 2, 2025 • edited Loading

neocybereth commented Jan 2, 2025

joel-u410 commented Jan 6, 2025 • edited Loading

joel-u410 commented Jan 9, 2025 • edited Loading

joel-u410 commented Jan 10, 2025

neocybereth commented Jan 10, 2025

neocybereth commented Jan 11, 2025

joel-u410 commented Jan 13, 2025

joel-u410 commented Jan 13, 2025

neocybereth commented Jan 14, 2025

Fraccaman commented Jan 29, 2025 • edited Loading

joel-u410 commented Jan 29, 2025

neocybereth commented Jan 30, 2025

neocybereth commented Feb 4, 2025 • edited Loading

neocybereth commented Feb 4, 2025

joel-u410 commented Feb 4, 2025 • edited Loading

neocybereth commented Feb 5, 2025 • edited Loading

Fraccaman commented Feb 17, 2025

neocybereth commented Feb 17, 2025 • edited Loading

Fraccaman commented Feb 19, 2025

neocybereth commented Feb 19, 2025

sug0 commented Feb 19, 2025

neocybereth commented Feb 21, 2025

neocybereth commented Feb 21, 2025

Fraccaman commented Feb 21, 2025

neocybereth commented Dec 16, 2024 •

edited

Loading

neocybereth commented Dec 30, 2024 •

edited

Loading

brentstone commented Jan 2, 2025 •

edited

Loading

joel-u410 commented Jan 6, 2025 •

edited

Loading

joel-u410 commented Jan 9, 2025 •

edited

Loading

Fraccaman commented Jan 29, 2025 •

edited

Loading

neocybereth commented Feb 4, 2025 •

edited

Loading

joel-u410 commented Feb 4, 2025 •

edited

Loading

neocybereth commented Feb 5, 2025 •

edited

Loading

neocybereth commented Feb 17, 2025 •

edited

Loading