Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New fork detection incompatible with tqdm #221

Open
bjenik opened this issue Mar 10, 2025 · 1 comment
Open

New fork detection incompatible with tqdm #221

bjenik opened this issue Mar 10, 2025 · 1 comment

Comments

@bjenik
Copy link

bjenik commented Mar 10, 2025

It appears the new fork detection is very aggressive, making tensorstore incompatible with packages like tqdm, which is fairly widely used for progress tracking in the AI/ML ecosystem and causing downstream issues. I am not certain if there was actually a problem that is now successfully detected and prevented or if we are looking at what then ends up being a false positive, but I did not experience any issues using this combination until now.

Minimal example:

import tensorstore as ts
from tqdm import tqdm 

dataset = ts.open({
    'driver': 'n5',
    'kvstore': {
        'driver': 'file',
        'path': 'tmp/dataset/',
    },
}).result()

for i in tqdm(range(10)):
    print(i)
@laramiel
Copy link
Collaborator

laramiel commented Mar 11, 2025

Yes, there were numerous freeze/lockups that have caused issues due to calling fork. However we agree that it was too aggressive, and I submitted a change to relax the fork detection quite a bit here: f2a81b6

Unfortunately I'm waiting on a pypi resource update to do another point release:
pypi/support#5719

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants