[BUG] GPU installation does not work

[danielburongarcia]

Describe the bug
GPU is not working

To Reproduce
I have a RTX 4080 GPU and Windows 11.
I used conda to create the environment:
conda create -n rembg python=3.12

And installed gpu support with (I need those specific versions for another:
conda install conda-forge::cuda-runtime=12.6.3 conda-forge::cudnn=9.3.0.75

A clean environment following the readme to install rembg with gpu
pip install "rembg[gpu,cli]"

when you use rembg cli or inside a python script gives this error

2025-01-13 17:50:49.6713391 [E:onnxruntime:Default, provider_bridge_ort.cc:1848 onnxruntime::TryGetProviderInfo_TensorRT] D:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1539 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\WinUser\.conda\envs\rembg\Lib\site-packages\onnxruntime\capi\onnxruntime_providers_tensorrt.dll"

*************** EP Error ***************
EP Error D:\a\_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:507 onnxruntime::python::RegisterTensorRTPluginsAsCustomOps Please install TensorRT libraries as mentioned in the GPU requirements page, make sure they're in the PATH or LD_LIBRARY_PATH, and that your GPU is supported.
 when using ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.
****************************************

Expected behavior
It should work as previous versions

OS Version:
Windows 11 and Windows 10

Rembg version:
v2.0.61

Additional context
Tested with multiple PCs with Windows 11 and Windows 10, using RTX 4080 and RTX 4070. It works as planned with CPU when I install it with
pip install "rembg[cpu,cli]"

[fransoavirtuality]

Same here...

[jitendra-koodo]

same here

[shuaijiang111111111]

同样在这里

[naveenselvam17]

Same here

[champeau87]

same here

[fransoavirtuality]

Same...

[danielgatis]

Unfortunately, at the moment I only have access to a Mac M1, so I can't test this issue on Windows with a GPU. I'd appreciate it if someone could investigate this issue further and maybe even open a PR :)

[EkSulfur]

Same here

[ShyamCrossShore]

P Error D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:507 onnxruntime::python::RegisterTensorRTPluginsAsCustomOps Please install TensorRT libraries as mentioned in the GPU requirements page, make sure they're in the PATH or LD_LIBRARY_PATH, and that your GPU is supported.
when using ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.

---

I have solved this issue by modifying a little bit of code in the library files

first make sure that your onnxruntime library is gpu version or cpu version
if its cpu version uninstall and install the gpu version by
pip install onnxruntime-gpu

step 1
install tensorrt library from
https://docs.nvidia.com/deeplearning/tensorrt/latest/installing-tensorrt/installing.html

command :
(for cuda 12.X):
pip install tensorrt-cu12
(for cuda 11.X)
pip install tensorrt-cu11

step 2:
in Lib\site-packages\rembg\sessions\base.py

change
providers=None to providers =['CUDAExecutionProvider']
self.providers=[] to self.providers =['CUDAExecutionProvider']
delete this part :
_providers = ort.get_available_providers()
if providers:
for provider in providers:
if provider in _providers:
self.providers.append(provider)
else:
self.providers.extend(_providers)
in Lib\site-packages\rembg\session_factory.py

write providers =['CUDAExecutionProvider'] in new_session function initialization

this will force the onnxruntime to use CUDA

it has worked for me on windows 10 , RTX 3060 12gb with Model birefnet-massive
DM me on linkedin https://www.linkedin.com/in/shyam-thakkar167/ if you get into any issue

[danielgatis]

@ShyamCrossShore, could you help me fix this? I don’t have a GPU. 😞

Could you try testing with:

providers = ['CUDAExecutionProvider', 'CPUExecutionProvider']

If this change allows it to run correctly on both GPU and CPU, we could open a PR to set these providers as the default. What do you think?

[ditesh]

I finally got this working.

First, problem was that several libraries were not being loaded due to LD_LIBRARY_PATH not including them. I fixed it by:

export LD_LIBRARY_PATH='/path/to/lib/python3.13/site-packages/nvidia/cublas/lib/':'/path/to/lib/python3.13/site-packages/onnxruntime/capi/':'/path/to/lib/python3.13/site-packages/nvidia/cudnn/lib/'

Secondly, I explicitly specified CUDAExecutionProvider in rembg/sessions/base.py (I think the providers are not being passed in, CMIIW)

def __init__(self, model_name: str, sess_opts: ort.SessionOptions, *args, **kwargs):
    """Initialize an instance of the BaseSession class."""
    self.model_name = model_name
    self.inner_session = ort.InferenceSession(
        str(self.__class__.download_models(*args, **kwargs)),
        sess_options=sess_opts,
        providers=["CUDAExecutionProvider"],
    )