Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not cache instances of Compiler in CompilerStack #15460

Merged
merged 2 commits into from
Sep 30, 2024

Conversation

cameel
Copy link
Member

@cameel cameel commented Sep 26, 2024

Depends on #15451. Merged.
Related to #15459.

Another follow-up to #15451, this time removing the Compiler instances stored in CompilerStack.

This is a test PR to benchmark it see if it's even worth it. But TBH storing these objects in CompilerStack seems like a weird design choice. Apparently the only thing we need from them are the generated sources. And it's not even like they're generated on demand - they're stored in compiler context and simply copied from there. For this reason I'd be in favor of merging this even it does not improve performance. Just not making it worse would be enough.

@cameel cameel added performance 🐎 has dependencies The PR depends on other PRs that must be merged first labels Sep 26, 2024
@cameel cameel requested a review from clonker September 26, 2024 21:08
@cameel cameel self-assigned this Sep 26, 2024
@cameel
Copy link
Member Author

cameel commented Sep 26, 2024

Benchmarks

Results from external.sh, averaged over 3 runs. #15451 vs this PR.

File Pipeline Time (before) Time (after) Memory (before) Memory (after)
openzeppelin legacy 10 s 10 s 517 MiB 514 MiB
openzeppelin ir 33 s 33 s 497 MiB 495 MiB
uniswap-v4 legacy 21 s 20 s 1012 MiB 1007 MiB
uniswap-v4 ir 119 s 120 s 1494 MiB 1499 MiB
eigenlayer legacy 74 s 77 s 2835 MiB 2821 MiB
eigenlayer ir 522 s 517 s 4457 MiB 4471 MiB

Looks like it does not make much difference, both in terms of memory and execution time.

@cameel
Copy link
Member Author

cameel commented Sep 26, 2024

I added a commit that removes the generatedSources and runtimeGeneratedSources fields, storing JSON version of generated sources in a LazyInit container (copied over from one #15459).

With this results seem on par with #15459 (comment).

Benchmarks after LazyInit removal

Results from external.sh, 1 run. #15451 vs this PR.

Legacy

File Time (before) Time (after) Memory (before) Memory (after)
openzeppelin 9 s 11 s 517 MiB 454 MiB
uniswap-v4 20 s 20 s 1012 MiB 902 MiB
eigenlayer 72 s 76 s 2833 MiB 2411 MiB
seaport 150 s 150 s 4477 MiB 3896 MiB
sablier-v2 141 s 141 s 5215 MiB 5020 MiB

IR

File Time (before) Time (after) Memory (before) Memory (after)
openzeppelin 31 s 35 s 497 MiB 496 MiB
uniswap-v4 117 s 127 s 1491 MiB 1498 MiB
eigenlayer 509 s 515 s 4455 MiB 4461 MiB

The running time here seemingly increased, but I suspect it's just a fluke. Will rerun to make sure. EDIT: Yeah, just a fluke.

Base automatically changed from compiler-stack-do-not-store-serialized-json to develop September 27, 2024 13:17
@cameel cameel marked this pull request as ready for review September 27, 2024 13:17
@cameel cameel removed the has dependencies The PR depends on other PRs that must be merged first label Sep 27, 2024
@cameel cameel force-pushed the compiler-stack-do-not-store-compiler-instances branch from aebd331 to 44a6f8f Compare September 27, 2024 13:19
@cameel cameel force-pushed the compiler-stack-do-not-store-compiler-instances branch from 44a6f8f to 3c5e46b Compare September 30, 2024 10:56
@cameel cameel enabled auto-merge September 30, 2024 10:56
@cameel cameel merged commit bc9342a into develop Sep 30, 2024
72 checks passed
@cameel cameel deleted the compiler-stack-do-not-store-compiler-instances branch September 30, 2024 11:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants