Why does get_requires_for_build_wheel return the runtime requirements?

[mwchase]

I'm attempting a somewhat elaborate repository layout, in which some dependencies are only available locally via constraints files, and it's failing because pip is attempting to install the runtime dependencies in an isolated environment, without reference to the constraint files.

It looks like this is down to the implementation of get_requires_for_build_wheel, which in the other PEP 517 build systems I've looked at (Poetry and Enscons) just returns an empty list. In flit, it returns the contents of tools.flit.metadata.requires, which I understand to be the runtime requirements, and I don't think I need them to build the packages. Without the ability to install something, pip fails after logging "Installing backend dependencies". (The "something" doesn't have to be remotely what's intended. My initial testing was confounded because I was specifying a local package that shared a name with an unrelated package in pypi. So, the pypi package got installed in the isolated environment, and the local package got installed in the virtualenv where pip was.)

I worked around this by disabling build isolation and explicitly installing flit in the target environment, but I'd like to better understand the intents and tradeoffs here.

[bpabel]

I agree that get_requires_for_build_wheel shouldn't be returning the requires-dist requirements.

One thing that flit does, however, is attempt to import the package to retrieve the __version__ if it can't be read from parsing the module/package file, and that does require the dependencies during build time so that the package is importable.

That being said, it's probably more correct to not include runtime dependencies as build dependencies and accept that the version discovery fallback is more likely to fail.

Truthfully, there likely needs to be another PEP for specifying package versions outside of the source python package to really solve this problem for good.

[mwchase]

So, in my specific case, could the problem be that, by the time flit is generating requirements for the isolated environment, it doesn't have visibility into the constraints files passed to pip? (At least, I think it doesn't. It might.)

To go into a little more detail, I've got a repository with multiple (currently two) flit projects. Project 2 depends on Project 1. I can install Project 1 via pip install -c /path/to/constraints.txt project_1, but pip install -c /path/to/constraints.txt project_2 fails (or rather, either fails, or succeeds in a deceptive way).

Thinking about this further, I could avoid this problem by building everything before running tests, and storing the wheels in a local index. That'd be faster and cleaner.

[bpabel]

AFAIK, flit isn't actually responsible for resolving the requirements, it just passes the requested dependency strings straight from the metadata. pip, or whatever build front-end you're using, would be responsible for actually resolving the actual version (taking into account any constraint files). It's possible pip isn't considering the constraint file for the build requirements.

[mwchase]

Since I understand why this is done now, I've concluded it would make more sense to discuss this on the pip repo, and opened the above issue over there.