Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Threading race condition in _eventloop.get_asynclib() (with fix) #2116

Closed
danielrobbins opened this issue Mar 9, 2022 · 2 comments
Closed

Comments

@danielrobbins
Copy link

I have a threading Web spider that is triggering a race condition in anyio/_core/_eventloop.py's get_asynclib() function. This race can be triggered if you have a lot of threads trying to initialize their own asyncio event loop and initializing httpx all at once. It happens intermittently -- sometimes we don't trigger the race, sometimes we do. When we do, the threads catch get_asynclib() mid-initialization and certain attributes that should be present are not available. I am using httpx 0.22.0 and python3.7.

httpx-exception.txt

An ugly but functioning patch which I can confirm resolves the issue is below:

get_asynclib_thread_fix.patch.txt

In my test patch, I made the lock cover essentially the entire get_asynclib() call -- it could potentially be made tighter -- not sure.

Thanks for creating httpx -- it's very nice! :)

@danielrobbins
Copy link
Author

Sorry, I was rushing :) This was a bug in anyio. I will file with them.

@danielrobbins
Copy link
Author

Opened the same bug with anyio here: agronholm/anyio#425

geaaru pushed a commit to macaroni-os/funtoo-metatools that referenced this issue Jul 26, 2024
httpx. I also have a fix that's required for httpx 0.22.0 that I
reported upstream at encode/httpx#2116 .
This patch fixes an intermittent traceback that can happen.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant