Fix No module named 'torch'

[sakurai-youhei]

Relates #552

Issue:

eland_import_hub_model claims ModuleNotFoundError unless pytorch is installed.

Traceback (most recent call last):
  File "/usr/local/bin/eland_import_hub_model", line 34, in <module>
    import torch
ModuleNotFoundError: No module named 'torch'

Appropriate result:

eland_import_hub_model should show instruction what is missing like this.

2023-06-30 03:52:36,956 ERROR : Failed to run because module 'tqdm' is not available.

This script requires PyTorch extras to run. You can install these by running:

    /usr/local/bin/python -m pip install 'eland[pytorch]'

How to repro:

docker run -it python:3 bash -c "pip install git+https://github.com/elastic/eland.git@main && /usr/local/bin/eland_import_hub_model --help"

Console output

C:\Users\YouheiSakurai>docker run -it python:3 bash -c "pip install git+https://github.com/elastic/eland.git@main && /usr/local/bin/eland_import_hub_model --help"
Collecting git+https://github.com/elastic/eland.git@main
  Cloning https://github.com/elastic/eland.git (to revision main) to /tmp/pip-req-build-wmhhpd88
  Running command git clone --filter=blob:none --quiet https://github.com/elastic/eland.git /tmp/pip-req-build-wmhhpd88
  Resolved https://github.com/elastic/eland.git to commit bf3b092ed43a14e90fc4a1f47f3ef7bbb1b422bc
  Preparing metadata (setup.py) ... done
Collecting elasticsearch<9,>=8.3 (from eland==8.7.0)
  Downloading elasticsearch-8.8.0-py3-none-any.whl (393 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 393.8/393.8 kB 9.5 MB/s eta 0:00:00
Collecting pandas<2,>=1.5 (from eland==8.7.0)
  Downloading pandas-1.5.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.0/12.0 MB 20.5 MB/s eta 0:00:00
Collecting matplotlib>=3.6 (from eland==8.7.0)
  Downloading matplotlib-3.7.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.6/11.6 MB 20.4 MB/s eta 0:00:00
Collecting numpy<1.24,>=1.2.0 (from eland==8.7.0)
  Downloading numpy-1.23.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.1/17.1 MB 21.0 MB/s eta 0:00:00
Collecting elastic-transport<9,>=8 (from elasticsearch<9,>=8.3->eland==8.7.0)
  Downloading elastic_transport-8.4.0-py3-none-any.whl (59 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 59.5/59.5 kB 6.3 MB/s eta 0:00:00
Collecting contourpy>=1.0.1 (from matplotlib>=3.6->eland==8.7.0)
  Downloading contourpy-1.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (300 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 300.4/300.4 kB 15.9 MB/s eta 0:00:00
Collecting cycler>=0.10 (from matplotlib>=3.6->eland==8.7.0)
  Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting fonttools>=4.22.0 (from matplotlib>=3.6->eland==8.7.0)
  Downloading fonttools-4.40.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.3/4.3 MB 22.5 MB/s eta 0:00:00
Collecting kiwisolver>=1.0.1 (from matplotlib>=3.6->eland==8.7.0)
  Downloading kiwisolver-1.4.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 19.8 MB/s eta 0:00:00
Collecting packaging>=20.0 (from matplotlib>=3.6->eland==8.7.0)
  Downloading packaging-23.1-py3-none-any.whl (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 5.4 MB/s eta 0:00:00
Collecting pillow>=6.2.0 (from matplotlib>=3.6->eland==8.7.0)
  Downloading Pillow-9.5.0-cp311-cp311-manylinux_2_28_x86_64.whl (3.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 21.6 MB/s eta 0:00:00
Collecting pyparsing>=2.3.1 (from matplotlib>=3.6->eland==8.7.0)
  Downloading pyparsing-3.1.0-py3-none-any.whl (102 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 102.6/102.6 kB 11.1 MB/s eta 0:00:00
Collecting python-dateutil>=2.7 (from matplotlib>=3.6->eland==8.7.0)
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 18.5 MB/s eta 0:00:00
Collecting pytz>=2020.1 (from pandas<2,>=1.5->eland==8.7.0)
  Downloading pytz-2023.3-py2.py3-none-any.whl (502 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 502.3/502.3 kB 22.5 MB/s eta 0:00:00
Collecting urllib3<2,>=1.26.2 (from elastic-transport<9,>=8->elasticsearch<9,>=8.3->eland==8.7.0)
  Downloading urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.1/143.1 kB 15.3 MB/s eta 0:00:00
Collecting certifi (from elastic-transport<9,>=8->elasticsearch<9,>=8.3->eland==8.7.0)
  Downloading certifi-2023.5.7-py3-none-any.whl (156 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 157.0/157.0 kB 17.1 MB/s eta 0:00:00
Collecting six>=1.5 (from python-dateutil>=2.7->matplotlib>=3.6->eland==8.7.0)
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: eland
  Building wheel for eland (setup.py) ... done
  Created wheel for eland: filename=eland-8.7.0-py3-none-any.whl size=155087 sha256=ac171e6a62bbbb8fd8ea7b1c00b6347a175b0dde870a71195992b4a9f1eb02ab
  Stored in directory: /tmp/pip-ephem-wheel-cache-yrx3q1nv/wheels/e3/34/ba/4269a226b8e1fbaf7e3d1b86ae81a7c19c44c88fc0d0cbfb00
Successfully built eland
Installing collected packages: pytz, urllib3, six, pyparsing, pillow, packaging, numpy, kiwisolver, fonttools, cycler, certifi, python-dateutil, elastic-transport, contourpy, pandas, matplotlib, elasticsearch, eland
Successfully installed certifi-2023.5.7 contourpy-1.1.0 cycler-0.11.0 eland-8.7.0 elastic-transport-8.4.0 elasticsearch-8.8.0 fonttools-4.40.0 kiwisolver-1.4.4 matplotlib-3.7.1 numpy-1.23.5 packaging-23.1 pandas-1.5.3 pillow-9.5.0 pyparsing-3.1.0 python-dateutil-2.8.2 pytz-2023.3 six-1.16.0 urllib3-1.26.16
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Traceback (most recent call last):
  File "/usr/local/bin/eland_import_hub_model", line 34, in <module>
    import torch
ModuleNotFoundError: No module named 'torch'

[pquentin]

Thanks! LGTM.

[davidkyle]

LGTM

[sakurai-youhei]

Thank you! Once merged, I will probably open 2 more PRs to achieve the goal targeted in #552.

[davidkyle]

The test failure is another instance of #555 and unrelated to these changes. I will merge the PR with the failure