Unable to install two extras in a project making use of different extras of a second library

[saulvargas]

Environment

• pip version: 19.2.3
• Python version: 3.6.9
• OS: macOS Mojave 10.14.6

Description

I have a setup.py that defines extra requirements which, in turn, specify different extra requirements of a third package. My issue can be reproduced when using the following toy example:

from setuptools import setup

setup(extras_require={"extras1": "dask[diagnostics]", "extras2": "dask[delayed]"})

I am trying to install both extras at once (pip install -e .[extras1,extras2]) but, when doing so, only the requirements of the first one are actually installed (i.e. it installs dask[diagnostics], but not dask[delayed]).

I have noticed that removing these two lines allows me to do my desired behaviour.

[chrahunt]

Thanks for the easy reproducer!

This is the same as or very closely related to #6239/#4391/#988.

[pradyunsg]

Yea, this is a problem with pip's extras merging logic. We'll likely fix that with #988.

[chrahunt]

I think it would be reasonable here to make a test case and mark it xfail, then we can close this and track the fix itself with #988.

[pelson]

FWIW I just checked this against master (2f3a1be) using both the default resolver and --unstable-feature=resolver, and don't see the issue being resolved.

[uranusjr]

Update from the future: The new resolver cannot solve this problem (although it’s a prerequisite) because the requirements are currently merged before it reaches the resolver, by RequirementSet.add_requirement().

The root problem is similar to #8036 (comment). I’m reopening #7571 to track this and #8036.