-
Notifications
You must be signed in to change notification settings - Fork 510
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use multi-threaded runtime instead of single-threaded ones #6743
Use multi-threaded runtime instead of single-threaded ones #6743
Conversation
- fix deadlocks caused by remaining sync primitive - increase thread count
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for contributing to Neon!
I tried this locally but found that it deadlocks.
Also, the intent was that there be a fixed number of worker threads on which a variable number of requesting thread run on.
Perhaps that wasn't clear in the issue description.
While trying to understand the deadlock, I went ahead and fixed these issues.
PR stacked atop this one: #6761
We'll review it and either close this PR or merge it in here.
Thanks for reviewing!
Ah, I see! Indeed, I was initially a bit confused about this part. 😅 The comment in #6761 about stress-testing clarified it for me.
I haven't ran into deadlocks when running it, but taking another look at the code it makes sense. :D I supposed that they were caused by the sync |
Yes |
This PR finishes the work started by https://github.com/calinanca99 in #6743 - always use default worker thread count & explain rationale behind that - fix deadlocks caused by remaining sync primitive - increase thread counts
…eondatabase#6761) This PR finishes the work started by https://github.com/calinanca99 in neondatabase#6743 - always use default worker thread count & explain rationale behind that - fix deadlocks caused by remaining sync primitive - increase thread counts
2436 tests run: 2318 passed, 0 failed, 118 skipped (full report)Code coverage (full report)
The comment gets automatically updated with the latest test results
c0847de at 2024-02-16T15:28:31.162Z :recycle: |
- re-introduce warm-up - joinset - capture reference data
Problem
Addresses #6648.
Summary of changes
add_multithreaded_walredo_requesters
that sets the number of workers based on the number of threads passed by CriterionDrop
implementation forJoinOnDrop
with ajoin_all
method. This is needed to avoid having async code in theDrop
trait implementationexecute_all
andRequest::execute
asyncResults
Result compared to
main
:Checklist before requesting a review
Checklist before merging