Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Entries added to lockfile by pdm lock --platform linux --python 3.11 --append are missing markers when using a custom package source #3261

Open
1 task done
andersjel opened this issue Nov 7, 2024 · 0 comments
Open
1 task done

Entries added to lockfile by pdm lock --platform linux --python 3.11 --append are missing markers when using a custom package source #3261

andersjel opened this issue Nov 7, 2024 · 0 comments
Labels
🐛 bug Something isn't working

Comments

@andersjel
Copy link

  • I have searched the issue tracker and believe that this is not a duplicate.

Steps to reproduce

I am running Windows and I have this pyproject.toml in a project:

[project]
name = "bug"
version = "0.1.0"
dependencies = [
    "torch==2.1.0+cu118",
    "triton==2.1.0; platform_system=='Linux'",
]

[[tool.pdm.source]]
name = "PyTorch"
url = "https://download.pytorch.org/whl/cu118"
include_packages = ["torch"]

I then run:

pdm lock --platform=windows --python=3.11
pdm install

Everything works fine.

I then do:

pdm lock --platform=linux --python=3.11 --append
pdm install -v

And now it fails (see below for output).

Actual behavior

I get the following output from the final pdm install -v:

STATUS: Resolving packages from lockfile...
unearth.preparer: The file . is a local directory, use it directly
Synchronizing working set with resolved packages: 1 to add, 0 to update, 0 to remove

  ✖ Install triton 2.1.0 failed
pdm.termui: Error occurs adding triton:
Traceback (most recent call last):
  File "C:\Program Files\Python311\Lib\concurrent\futures\thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Anders Jellinggaard\.local\pipx\venvs\pdm\Lib\site-packages\pdm\installers\synchronizers.py", line 29, in install_candidate
    self.manager.install(can)
  File "C:\Users\Anders Jellinggaard\.local\pipx\venvs\pdm\Lib\site-packages\pdm\installers\manager.py", line 33, in install
    prepared.build(),
    ^^^^^^^^^^^^^^^^
  File "C:\Users\Anders Jellinggaard\.local\pipx\venvs\pdm\Lib\site-packages\pdm\models\candidates.py", line 397, in build
    self._obtain(allow_all=False)
  File "C:\Users\Anders Jellinggaard\.local\pipx\venvs\pdm\Lib\site-packages\pdm\models\candidates.py", line 438, in _obtain
    raise CandidateNotFound(
pdm.exceptions.CandidateNotFound: No candidate is found for `triton` that matches the environment or hashes
  Retry failed jobs(2/2)
  ✖ Install triton 2.1.0 failed
pdm.termui: Error occurs adding triton:
Traceback (most recent call last):
  File "C:\Program Files\Python311\Lib\concurrent\futures\thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Anders Jellinggaard\.local\pipx\venvs\pdm\Lib\site-packages\pdm\installers\synchronizers.py", line 29, in install_candidate
    self.manager.install(can)
  File "C:\Users\Anders Jellinggaard\.local\pipx\venvs\pdm\Lib\site-packages\pdm\installers\manager.py", line 33, in install
    prepared.build(),
    ^^^^^^^^^^^^^^^^
  File "C:\Users\Anders Jellinggaard\.local\pipx\venvs\pdm\Lib\site-packages\pdm\models\candidates.py", line 397, in build
    self._obtain(allow_all=False)
  File "C:\Users\Anders Jellinggaard\.local\pipx\venvs\pdm\Lib\site-packages\pdm\models\candidates.py", line 438, in _obtain
    raise CandidateNotFound(
pdm.exceptions.CandidateNotFound: No candidate is found for `triton` that matches the environment or hashes
  ✖ Some package operations failed.
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Anders Jellinggaard\.local\bin\pdm.exe\__main__.py", line 7, in <module>
  File "C:\Users\Anders Jellinggaard\.local\pipx\venvs\pdm\Lib\site-packages\pdm\core.py", line 378, in main
    return core.main(args or sys.argv[1:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Anders Jellinggaard\.local\pipx\venvs\pdm\Lib\site-packages\pdm\core.py", line 267, in main
    raise cast(Exception, err).with_traceback(traceback) from None
  File "C:\Users\Anders Jellinggaard\.local\pipx\venvs\pdm\Lib\site-packages\pdm\core.py", line 262, in main
    self.handle(project, options)
  File "C:\Users\Anders Jellinggaard\.local\pipx\venvs\pdm\Lib\site-packages\pdm\core.py", line 194, in handle
    command.handle(project, options)
  File "C:\Users\Anders Jellinggaard\.local\pipx\venvs\pdm\Lib\site-packages\pdm\cli\commands\install.py", line 103, in handle
    actions.do_sync(
  File "C:\Users\Anders Jellinggaard\.local\pipx\venvs\pdm\Lib\site-packages\pdm\cli\actions.py", line 301, in do_sync
    synchronizer.synchronize()
  File "C:\Users\Anders Jellinggaard\.local\pipx\venvs\pdm\Lib\site-packages\pdm\installers\synchronizers.py", line 226, in synchronize
    raise InstallationError("Some package operations failed.")
pdm.exceptions.InstallationError: Some package operations failed.

Expected behavior

pdm install should not do anything and exit successfully since the lock file is unchanged for Windows. I only appended for --platform=linux and I am running Windows.

Environment Information

PDM version:
  2.20.0.post1
Python Interpreter:
  C:\src\Scanner.Gecko\bug\.venv\Scripts\python.exe (3.11)
Project Root:
  C:/src/Scanner.Gecko/bug
Local Packages:

{
  "implementation_name": "cpython",
  "implementation_version": "3.11.9",
  "os_name": "nt",
  "platform_machine": "AMD64",
  "platform_release": "10",
  "platform_system": "Windows",
  "platform_version": "10.0.22631",
  "python_full_version": "3.11.9",
  "platform_python_implementation": "CPython",
  "python_version": "3.11",
  "sys_platform": "win32"
}

Details

Here is the content of pdm.lock after pdm lock --platform=windows --python=3.11:

# This file is @generated by PDM.
# It is not intended for manual editing.

[metadata]
groups = ["default"]
strategy = ["inherit_metadata"]
lock_version = "4.5.0"
content_hash = "sha256:758e03f8748ce837334d13838f35bd25203e3e4c943ce257eb30382612682c9f"

[[metadata.targets]]
requires_python = ">=3.11"
platform = "windows_amd64"

[[package]]
name = "filelock"
version = "3.16.1"
requires_python = ">=3.8"
summary = "A platform independent file lock."
groups = ["default"]
marker = "python_version >= \"3.11\""
files = [
    {file = "filelock-3.16.1-py3-none-any.whl", hash = "sha256:2082e5703d51fbf98ea75855d9d5527e33d8ff23099bec374a134febee6946b0"},
    {file = "filelock-3.16.1.tar.gz", hash = "sha256:c249fbfcd5db47e5e2d6d62198e565475ee65e4831e2561c8e313fa7eb961435"},
]

[[package]]
name = "fsspec"
version = "2024.10.0"
requires_python = ">=3.8"
summary = "File-system specification"
groups = ["default"]
marker = "python_version >= \"3.11\""
files = [
    {file = "fsspec-2024.10.0-py3-none-any.whl", hash = "sha256:03b9a6785766a4de40368b88906366755e2819e758b83705c88cd7cb5fe81871"},
    {file = "fsspec-2024.10.0.tar.gz", hash = "sha256:eda2d8a4116d4f2429db8550f2457da57279247dd930bb12f821b58391359493"},
]

[[package]]
name = "jinja2"
version = "3.1.4"
requires_python = ">=3.7"
summary = "A very fast and expressive template engine."
groups = ["default"]
marker = "python_version >= \"3.11\""
dependencies = [
    "MarkupSafe>=2.0",
]
files = [
    {file = "jinja2-3.1.4-py3-none-any.whl", hash = "sha256:bc5dd2abb727a5319567b7a813e6a2e7318c39f4f487cfe6c89c6f9c7d25197d"},
    {file = "jinja2-3.1.4.tar.gz", hash = "sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369"},
]

[[package]]
name = "markupsafe"
version = "3.0.2"
requires_python = ">=3.9"
summary = "Safely add untrusted strings to HTML/XML markup."
groups = ["default"]
marker = "python_version >= \"3.11\""
files = [
    {file = "MarkupSafe-3.0.2-cp311-cp311-win_amd64.whl", hash = "sha256:70a87b411535ccad5ef2f1df5136506a10775d267e197e4cf531ced10537bd6b"},
    {file = "MarkupSafe-3.0.2-cp312-cp312-win_amd64.whl", hash = "sha256:8e06879fc22a25ca47312fbe7c8264eb0b662f6db27cb2d3bbbc74b1df4b9b87"},
    {file = "MarkupSafe-3.0.2-cp313-cp313-win_amd64.whl", hash = "sha256:e6a2a455bd412959b57a172ce6328d2dd1f01cb2135efda2e4576e8a23fa3b0f"},
    {file = "MarkupSafe-3.0.2-cp313-cp313t-win_amd64.whl", hash = "sha256:e444a31f8db13eb18ada366ab3cf45fd4b31e4db1236a4448f68778c1d1a5a2f"},
    {file = "markupsafe-3.0.2.tar.gz", hash = "sha256:ee55d3edf80167e48ea11a923c7386f4669df67d7994554387f84e7d8b0a2bf0"},
]

[[package]]
name = "mpmath"
version = "1.3.0"
summary = "Python library for arbitrary-precision floating-point arithmetic"
groups = ["default"]
marker = "python_version >= \"3.11\""
files = [
    {file = "mpmath-1.3.0-py3-none-any.whl", hash = "sha256:a0b2b9fe80bbcd81a6647ff13108738cfb482d481d826cc0e02f5b35e5c88d2c"},
    {file = "mpmath-1.3.0.tar.gz", hash = "sha256:7a28eb2a9774d00c7bc92411c19a89209d5da7c4c9a9e227be8330a23a25b91f"},
]

[[package]]
name = "networkx"
version = "3.4.2"
requires_python = ">=3.10"
summary = "Python package for creating and manipulating graphs and networks"
groups = ["default"]
marker = "python_version >= \"3.11\""
files = [
    {file = "networkx-3.4.2-py3-none-any.whl", hash = "sha256:df5d4365b724cf81b8c6a7312509d0c22386097011ad1abe274afd5e9d3bbc5f"},
    {file = "networkx-3.4.2.tar.gz", hash = "sha256:307c3669428c5362aab27c8a1260aa8f47c4e91d3891f48be0141738d8d053e1"},
]

[[package]]
name = "sympy"
version = "1.13.3"
requires_python = ">=3.8"
summary = "Computer algebra system (CAS) in Python"
groups = ["default"]
marker = "python_version >= \"3.11\""
dependencies = [
    "mpmath<1.4,>=1.1.0",
]
files = [
    {file = "sympy-1.13.3-py3-none-any.whl", hash = "sha256:54612cf55a62755ee71824ce692986f23c88ffa77207b30c1368eda4a7060f73"},
    {file = "sympy-1.13.3.tar.gz", hash = "sha256:b27fd2c6530e0ab39e275fc9b683895367e51d5da91baa8d3d64db2565fec4d9"},
]

[[package]]
name = "torch"
version = "2.1.0+cu118"
requires_python = ">=3.8.0"
summary = "Tensors and Dynamic neural networks in Python with strong GPU acceleration"
groups = ["default"]
marker = "python_version >= \"3.11\""
dependencies = [
    "filelock",
    "fsspec",
    "jinja2",
    "networkx",
    "sympy",
    "typing-extensions",
]
files = [
    {file = "torch-2.1.0+cu118-cp311-cp311-win_amd64.whl", hash = "sha256:e200aba94307b7a2926f36274b92d76391f36694a1c0ca0e2c341db1fa4eca99"},
]

[[package]]
name = "typing-extensions"
version = "4.12.2"
requires_python = ">=3.8"
summary = "Backported and Experimental Type Hints for Python 3.8+"
groups = ["default"]
marker = "python_version >= \"3.11\""
files = [
    {file = "typing_extensions-4.12.2-py3-none-any.whl", hash = "sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d"},
    {file = "typing_extensions-4.12.2.tar.gz", hash = "sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8"},
]

Here is the content of pdm.lock after pdm lock --platform=linux --python=3.11 --append:

# This file is @generated by PDM.
# It is not intended for manual editing.

[metadata]
groups = ["default"]
strategy = ["inherit_metadata"]
lock_version = "4.5.0"
content_hash = "sha256:758e03f8748ce837334d13838f35bd25203e3e4c943ce257eb30382612682c9f"

[[metadata.targets]]
requires_python = ">=3.11"
platform = "windows_amd64"

[[metadata.targets]]
requires_python = ">=3.11"
platform = "manylinux_2_17_x86_64"

[[package]]
name = "filelock"
version = "3.16.1"
requires_python = ">=3.8"
summary = "A platform independent file lock."
groups = ["default"]
marker = "python_version >= \"3.11\""
files = [
    {file = "filelock-3.16.1-py3-none-any.whl", hash = "sha256:2082e5703d51fbf98ea75855d9d5527e33d8ff23099bec374a134febee6946b0"},
    {file = "filelock-3.16.1.tar.gz", hash = "sha256:c249fbfcd5db47e5e2d6d62198e565475ee65e4831e2561c8e313fa7eb961435"},
]

[[package]]
name = "fsspec"
version = "2024.10.0"
requires_python = ">=3.8"
summary = "File-system specification"
groups = ["default"]
marker = "python_version >= \"3.11\""
files = [
    {file = "fsspec-2024.10.0-py3-none-any.whl", hash = "sha256:03b9a6785766a4de40368b88906366755e2819e758b83705c88cd7cb5fe81871"},
    {file = "fsspec-2024.10.0.tar.gz", hash = "sha256:eda2d8a4116d4f2429db8550f2457da57279247dd930bb12f821b58391359493"},
]

[[package]]
name = "jinja2"
version = "3.1.4"
requires_python = ">=3.7"
summary = "A very fast and expressive template engine."
groups = ["default"]
marker = "python_version >= \"3.11\""
dependencies = [
    "MarkupSafe>=2.0",
]
files = [
    {file = "jinja2-3.1.4-py3-none-any.whl", hash = "sha256:bc5dd2abb727a5319567b7a813e6a2e7318c39f4f487cfe6c89c6f9c7d25197d"},
    {file = "jinja2-3.1.4.tar.gz", hash = "sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369"},
]

[[package]]
name = "markupsafe"
version = "3.0.2"
requires_python = ">=3.9"
summary = "Safely add untrusted strings to HTML/XML markup."
groups = ["default"]
marker = "python_version >= \"3.11\""
files = [
    {file = "MarkupSafe-3.0.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a123e330ef0853c6e822384873bef7507557d8e4a082961e1defa947aa59ba84"},
    {file = "MarkupSafe-3.0.2-cp311-cp311-win_amd64.whl", hash = "sha256:70a87b411535ccad5ef2f1df5136506a10775d267e197e4cf531ced10537bd6b"},
    {file = "MarkupSafe-3.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e17c96c14e19278594aa4841ec148115f9c7615a47382ecb6b82bd8fea3ab0c8"},
    {file = "MarkupSafe-3.0.2-cp312-cp312-win_amd64.whl", hash = "sha256:8e06879fc22a25ca47312fbe7c8264eb0b662f6db27cb2d3bbbc74b1df4b9b87"},
    {file = "MarkupSafe-3.0.2-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:15ab75ef81add55874e7ab7055e9c397312385bd9ced94920f2802310c930396"},
    {file = "MarkupSafe-3.0.2-cp313-cp313-win_amd64.whl", hash = "sha256:e6a2a455bd412959b57a172ce6328d2dd1f01cb2135efda2e4576e8a23fa3b0f"},
    {file = "MarkupSafe-3.0.2-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c0ef13eaeee5b615fb07c9a7dadb38eac06a0608b41570d8ade51c56539e509d"},
    {file = "MarkupSafe-3.0.2-cp313-cp313t-win_amd64.whl", hash = "sha256:e444a31f8db13eb18ada366ab3cf45fd4b31e4db1236a4448f68778c1d1a5a2f"},
    {file = "markupsafe-3.0.2.tar.gz", hash = "sha256:ee55d3edf80167e48ea11a923c7386f4669df67d7994554387f84e7d8b0a2bf0"},
]

[[package]]
name = "mpmath"
version = "1.3.0"
summary = "Python library for arbitrary-precision floating-point arithmetic"
groups = ["default"]
marker = "python_version >= \"3.11\""
files = [
    {file = "mpmath-1.3.0-py3-none-any.whl", hash = "sha256:a0b2b9fe80bbcd81a6647ff13108738cfb482d481d826cc0e02f5b35e5c88d2c"},
    {file = "mpmath-1.3.0.tar.gz", hash = "sha256:7a28eb2a9774d00c7bc92411c19a89209d5da7c4c9a9e227be8330a23a25b91f"},
]

[[package]]
name = "networkx"
version = "3.4.2"
requires_python = ">=3.10"
summary = "Python package for creating and manipulating graphs and networks"
groups = ["default"]
marker = "python_version >= \"3.11\""
files = [
    {file = "networkx-3.4.2-py3-none-any.whl", hash = "sha256:df5d4365b724cf81b8c6a7312509d0c22386097011ad1abe274afd5e9d3bbc5f"},
    {file = "networkx-3.4.2.tar.gz", hash = "sha256:307c3669428c5362aab27c8a1260aa8f47c4e91d3891f48be0141738d8d053e1"},
]

[[package]]
name = "sympy"
version = "1.13.3"
requires_python = ">=3.8"
summary = "Computer algebra system (CAS) in Python"
groups = ["default"]
marker = "python_version >= \"3.11\""
dependencies = [
    "mpmath<1.4,>=1.1.0",
]
files = [
    {file = "sympy-1.13.3-py3-none-any.whl", hash = "sha256:54612cf55a62755ee71824ce692986f23c88ffa77207b30c1368eda4a7060f73"},
    {file = "sympy-1.13.3.tar.gz", hash = "sha256:b27fd2c6530e0ab39e275fc9b683895367e51d5da91baa8d3d64db2565fec4d9"},
]

[[package]]
name = "torch"
version = "2.1.0+cu118"
requires_python = ">=3.8.0"
summary = "Tensors and Dynamic neural networks in Python with strong GPU acceleration"
groups = ["default"]
marker = "python_version >= \"3.11\""
dependencies = [
    "filelock",
    "fsspec",
    "jinja2",
    "networkx",
    "sympy",
    "typing-extensions",
]
files = [
    {file = "torch-2.1.0+cu118-cp311-cp311-linux_x86_64.whl", hash = "sha256:bcb17e2de6ca634d326203694d0bfb552587335e536c1917be3f28c5664b5506"},
    {file = "torch-2.1.0+cu118-cp311-cp311-win_amd64.whl", hash = "sha256:e200aba94307b7a2926f36274b92d76391f36694a1c0ca0e2c341db1fa4eca99"},
]

[[package]]
name = "triton"
version = "2.1.0"
summary = "A language and compiler for custom Deep Learning operations"
groups = ["default"]
marker = "python_version >= \"3.11\""
dependencies = [
    "filelock",
]
files = [
    {file = "triton-2.1.0-0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:919b06453f0033ea52c13eaf7833de0e57db3178d23d4e04f9fc71c4f2c32bf8"},
]

[[package]]
name = "typing-extensions"
version = "4.12.2"
requires_python = ">=3.8"
summary = "Backported and Experimental Type Hints for Python 3.8+"
groups = ["default"]
marker = "python_version >= \"3.11\""
files = [
    {file = "typing_extensions-4.12.2-py3-none-any.whl", hash = "sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d"},
    {file = "typing_extensions-4.12.2.tar.gz", hash = "sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8"},
]

Note

With this pyproject.toml I cannot reproduce the issue:

[project]
name = "bug"
version = "0.1.0"
dependencies = [
    "torch==2.1.0",
    "triton==2.1.0; platform_system=='Linux'",
]

With this pyproject.toml the entry for triton in the lockfile looks like:

[[package]]
name = "triton"
version = "2.1.0"
summary = "A language and compiler for custom Deep Learning operations"
groups = ["default"]
marker = "platform_system == \"Linux\" and python_version >= \"3.11\""
dependencies = [
    "filelock",
]
files = [
    {file = "triton-2.1.0-0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:919b06453f0033ea52c13eaf7833de0e57db3178d23d4e04f9fc71c4f2c32bf8"},
]
@andersjel andersjel added the 🐛 bug Something isn't working label Nov 7, 2024
@frostming frostming assigned frostming and unassigned frostming Nov 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
🐛 bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@andersjel @frostming