Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Indexer-Grpc-V2] Add HistoricalDataService. #15788

Merged
merged 1 commit into from
Jan 23, 2025
Merged

[Indexer-Grpc-V2] Add HistoricalDataService. #15788

merged 1 commit into from
Jan 23, 2025

Conversation

grao1991
Copy link
Contributor

Description

How Has This Been Tested?

Key Areas to Review

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Move Compiler
  • Other (specify)

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

@trunk-io Trunk.io
Copy link

trunk-io bot commented Jan 22, 2025 •
edited
Loading

⏱️ 53m total CI duration on this PR
Job Cumulative Duration Recent Runs
check-dynamic-deps 16m 🟩 🟩 🟩 🟩 🟩 (+2 more)
rust-cargo-deny 10m 🟩 🟩 ⬜ 🟩 🟩 (+2 more)
test-target-determinator 7m 🟩 
rust-doc-tests 6m 🟩 
execution-performance / test-target-determinator 5m 🟩 
general-lints 3m 🟩 🟩 ⬜ 🟩 🟩 (+2 more)
semgrep/ci 3m 🟩 🟩 🟩 🟩 🟩 (+2 more)
file_change_determinator 2m 🟩 🟩 🟩 🟩 🟩 (+2 more)
fetch-last-released-docker-image-tag 1m 🟩 
permission-check 19s 🟩 🟩 🟩 🟩 🟩 (+2 more)
permission-check 16s 🟩 🟩 🟩 🟩 🟩 (+2 more)
file_change_determinator 10s 🟩 
execution-performance / single-node-performance 9s 🟩 
permission-check 3s 🟩 
determine-docker-build-metadata 1s 🟩 

🚨 2 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
test-target-determinator 7m 5m +33%
execution-performance / single-node-performance 9s 17m -99%

settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io

@grao1991 grao1991 force-pushed the grao_historical_data_service branch from 1f2c07b to 3af42e9 Compare January 22, 2025 09:07
@grao1991 grao1991 force-pushed the grao_historical_data_service branch 2 times, most recently from bceab25 to 0cba850 Compare January 23, 2025 09:11
@grao1991 grao1991 force-pushed the grao_historical_data_service branch from 0cba850 to 8a3cf6f Compare January 23, 2025 19:16
@grao1991 grao1991 force-pushed the grao_historical_data_service branch from 8a3cf6f to 0578c27 Compare January 23, 2025 19:16
graphite-app[bot]
graphite-app bot reviewed Jan 23, 2025
View reviewed changes
ecosystem/indexer-grpc/indexer-grpc-data-service-v2/src/service.rs
Comment on lines +49 to +59
let request = req.into_inner();
let mut stream = live_data_service
.get_transactions(Request::new(request))
.await?
.into_inner();
let peekable = std::pin::pin!(stream.as_mut().peekable());
if let Some(Ok(_)) = peekable.peek().await {
return live_data_service
.get_transactions(Request::new(request))
.await;
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The request is consumed by into_inner() but then used again to create a new request, which will fail at runtime. To fix this, clone the request before the first consumption:

let request = req.clone().into_inner();

This ensures the request data is available for both operations.

Spotted by Graphite Reviewer

Is this helpful? React 👍 or 👎 to let us know.

@grao1991 grao1991 enabled auto-merge (squash) January 23, 2025 19:32
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions
Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 0578c27a579d46e3a7e113ce0bcb95f323a9de8b

two traffics test: inner traffic : committed: 14616.11 txn/s, latency: 2712.38 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3300 ms), latency samples: 5557380
two traffics test : committed: 99.98 txn/s, latency: 1443.27 ms, (p50: 1400 ms, p70: 1500, p90: 1600 ms, p99: 1700 ms), latency samples: 1740
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 1.492, avg: 1.456", "ConsensusProposalToOrdered: max: 0.295, avg: 0.291", "ConsensusOrderedToCommit: max: 0.411, avg: 0.400", "ConsensusProposalToCommit: max: 0.702, avg: 0.691"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.60s no progress at version 31477 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.62s no progress at version 2380284 (avg 0.62s) [limit 16].
Test Ok

@github-actions GitHub Actions
Copy link
Contributor

✅ Forge suite compat success on 9e7cf4a27cc20504cee5ffefc35bd8e7792dde36 ==> 0578c27a579d46e3a7e113ce0bcb95f323a9de8b

Compatibility test results for 9e7cf4a27cc20504cee5ffefc35bd8e7792dde36 ==> 0578c27a579d46e3a7e113ce0bcb95f323a9de8b (PR)
1. Check liveness of validators at old version: 9e7cf4a27cc20504cee5ffefc35bd8e7792dde36
compatibility::simple-validator-upgrade::liveness-check : committed: 5873.13 txn/s, submitted: 5873.47 txn/s, expired: 0.34 txn/s, latency: 2768.74 ms, (p50: 2600 ms, p70: 2800, p90: 3900 ms, p99: 5300 ms), latency samples: 374718
2. Upgrading first Validator to new version: 0578c27a579d46e3a7e113ce0bcb95f323a9de8b
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 4287.80 txn/s, latency: 7290.92 ms, (p50: 8000 ms, p70: 8600, p90: 8900 ms, p99: 9000 ms), latency samples: 91460
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 4180.60 txn/s, latency: 8157.67 ms, (p50: 9100 ms, p70: 9100, p90: 9300 ms, p99: 9500 ms), latency samples: 149160
3. Upgrading rest of first batch to new version: 0578c27a579d46e3a7e113ce0bcb95f323a9de8b
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 4309.70 txn/s, latency: 7199.78 ms, (p50: 7900 ms, p70: 8600, p90: 9000 ms, p99: 9000 ms), latency samples: 91900
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 4151.80 txn/s, latency: 8201.62 ms, (p50: 9100 ms, p70: 9100, p90: 9300 ms, p99: 9400 ms), latency samples: 145940
4. upgrading second batch to new version: 0578c27a579d46e3a7e113ce0bcb95f323a9de8b
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 1723.99 txn/s, submitted: 1724.14 txn/s, expired: 0.15 txn/s, latency: 4462.47 ms, (p50: 4800 ms, p70: 5000, p90: 6000 ms, p99: 6200 ms), latency samples: 127669
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 6887.49 txn/s, latency: 4902.76 ms, (p50: 4900 ms, p70: 5700, p90: 6200 ms, p99: 6400 ms), latency samples: 235000
5. check swarm health
Compatibility test for 9e7cf4a27cc20504cee5ffefc35bd8e7792dde36 ==> 0578c27a579d46e3a7e113ce0bcb95f323a9de8b passed
Test Ok

@grao1991 grao1991 merged commit 114204e into main Jan 23, 2025
88 checks passed
@grao1991 grao1991 deleted the grao_historical_data_service branch January 23, 2025 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@graphite-app graphite-app[bot] graphite-app[bot] left review comments

@larry-aptos larry-aptos larry-aptos approved these changes

@areshand areshand areshand approved these changes

@sitalkedia sitalkedia Awaiting requested review from sitalkedia

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@grao1991 @areshand @larry-aptos