Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Workaround issues with second inference when IPEX is linked #2604

Closed

Conversation

droberts195
Copy link
Contributor

We have observed a couple of issues where IPEX is linked where the first inference call works but the second does not:

Disabling JIT profiling avoids the problems.

We have observed a couple of issues where IPEX is linked where
the first inference call works but the second does not:

- intel/intel-extension-for-pytorch#484
  happens with ELSER and PyTorch 2.1
- elastic/elasticsearch#102541
  happens with the multilingual E5 base and large models and
  PyTorch 1.13.1

Disabling JIT profiling avoids the problems.
@droberts195
Copy link
Contributor Author

Important - before we merge this we need to check whether the fix of disabling JIT profiling with IPEX makes inference slower than enabling JIT profiling without IPEX. If it does then we should instead stop linking IPEX.

@droberts195
Copy link
Contributor Author

Superseded by #2605 - we decided it's better to remove IPEX altogether.

@droberts195 droberts195 closed this Jan 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant