pip install flair fails

[ghost]

I used to have flair but when I reinstall it and install it again, I got this error. Any idea?

 Stored in directory: /private/tmp/pip-ephem-wheel-cache-xjsleknb/wheels/6e/9c/ed/4499c9865ac1002697793e0ae05ba6be33553d098f3347fb94
  Building wheel for sacremoses (setup.py) ... done
  Created wheel for sacremoses: filename=sacremoses-0.0.38-py3-none-any.whl size=884629 sha256=ead16f644b7644dee2d0663a7ea3a9a59d96796a07f394f77b9d41abb20f8ad0
  Stored in directory: /private/tmp/pip-ephem-wheel-cache-xjsleknb/wheels/03/e9/be/8b52f6e7e8c333b56f9440575b4c5eb4d96d27b5d22df5a71e
  Building wheel for tokenizers (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/user/opt/miniconda3/envs/moses/bin/python /Users/user/opt/miniconda3/envs/moses/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmphaaqpbog
       cwd: /private/tmp/pip-install-tqpn6x0f/tokenizers
  Complete output (36 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib
  creating build/lib/tokenizers
  copying tokenizers/__init__.py -> build/lib/tokenizers
  creating build/lib/tokenizers/models
  copying tokenizers/models/__init__.py -> build/lib/tokenizers/models
  creating build/lib/tokenizers/decoders
  copying tokenizers/decoders/__init__.py -> build/lib/tokenizers/decoders
  creating build/lib/tokenizers/normalizers
  copying tokenizers/normalizers/__init__.py -> build/lib/tokenizers/normalizers
  creating build/lib/tokenizers/pre_tokenizers
  copying tokenizers/pre_tokenizers/__init__.py -> build/lib/tokenizers/pre_tokenizers
  creating build/lib/tokenizers/processors
  copying tokenizers/processors/__init__.py -> build/lib/tokenizers/processors
  creating build/lib/tokenizers/trainers
  copying tokenizers/trainers/__init__.py -> build/lib/tokenizers/trainers
  creating build/lib/tokenizers/implementations
  copying tokenizers/implementations/byte_level_bpe.py -> build/lib/tokenizers/implementations
  copying tokenizers/implementations/sentencepiece_bpe.py -> build/lib/tokenizers/implementations
  copying tokenizers/implementations/base_tokenizer.py -> build/lib/tokenizers/implementations
  copying tokenizers/implementations/__init__.py -> build/lib/tokenizers/implementations
  copying tokenizers/implementations/char_level_bpe.py -> build/lib/tokenizers/implementations
  copying tokenizers/implementations/bert_wordpiece.py -> build/lib/tokenizers/implementations
  copying tokenizers/__init__.pyi -> build/lib/tokenizers
  copying tokenizers/models/__init__.pyi -> build/lib/tokenizers/models
  copying tokenizers/decoders/__init__.pyi -> build/lib/tokenizers/decoders
  copying tokenizers/normalizers/__init__.pyi -> build/lib/tokenizers/normalizers
  copying tokenizers/pre_tokenizers/__init__.pyi -> build/lib/tokenizers/pre_tokenizers
  copying tokenizers/processors/__init__.pyi -> build/lib/tokenizers/processors
  copying tokenizers/trainers/__init__.pyi -> build/lib/tokenizers/trainers
  running build_ext
  running build_rust
  error: Can not find Rust compiler
  ----------------------------------------
  ERROR: Failed building wheel for tokenizers
Successfully built mpld3 langdetect segtok tabulate sqlitedict wrapt future sacremoses
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers which use PEP 517 and cannot be installed directly

[aychang95]

Transformers, one of Flair's dependencies, has a relatively new dependency called Tokenizers. That seems to be breaking for you when you're installing flair.

I see that you're installing using a miniconda3 environment but what OS are you trying to install flair on?

[ghost]

I see, the new transformers' tokenizer is causing the error.
Mine is Mojavo version 10.14.2

[ghost]

For now, I installed flair from source and changed transformers to 2.4.1 which is the old version.

[jmacedoit]

This is also happening to me, started to happen recently

[ghost]

see my comment above if you urgently need it

[alanakbik]

You could first install the previous transformers version and then flair, maybe this fixes it for now?

pip install transformers==2.4.1
pip install flair

[SauravShroff]

@alankbik that did the trick for me. Thanks alot

[solomonjoeykao]

try using
pip3 install flair
it works for me!

[alanakbik]

Great, closing this issue now but feel free to reopen if there are more questions!

[lgov]

Hi,
I have the same problem on Mac OS X.
If the installation of flair via pip now systematically fails because of a mandatory dependency not installing completely, may I suggest you add the installation instructions of that dependency to the flair installation instructions?

The Tokenizers pip(*) page has the following instructions:

To use this method, you need to have the Rust installed:

# Install with:
curl https://sh.rustup.rs -sSf | sh -s -- -y
export PATH="$HOME/.cargo/bin:$PATH"

These instructions solved the installation issues on my machine - after I removed the enum34 package which breaks the python 3.7 enum , but that's another unrelated issue.

(*) https://pypi.org/project/tokenizers/

[alanakbik]

@lgov this looks like the error is reported: huggingface/transformers#2831 and huggingface/transformers#2980 - it looks like they will fix this soon.

[megantosh]

encountered same issue on Py 3.8 (macOS 10.12.6). Also unable to install an earlier version of Transformers. @lgov 's suggestion did it for me. Thanks!