Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ninja dep #1

Open
hadim opened this issue Jan 15, 2023 · 0 comments
Open

Ninja dep #1

hadim opened this issue Jan 15, 2023 · 0 comments
Labels
question Further information is requested

Comments

@hadim
Copy link
Member

hadim commented Jan 15, 2023

Comment:

Deepspeed depends on ninja-python from pypi which is about to be packaged: conda-forge/staged-recipes#19098

That being said, only ninja is really needed and small modifications in the deepspeed source code should allow using the conda forge ninja package (https://github.com/conda-forge/ninja-feedstock).

That strategy were discussed at conda-forge/staged-recipes#21708 (comment)
@hadim hadim added the question Further information is requested label Jan 15, 2023
@hadim hadim mentioned this issue Jan 15, 2023
Merged
10 tasks
@hadim hadim mentioned this issue Feb 21, 2023
Closed
9 tasks
@weiji14 weiji14 mentioned this issue Nov 3, 2023
Merged
3 tasks
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Nov 4, 2023
@weiji14
Remove ninja as runtime dependency
b469808 
Xref conda-forge#1
weiji14 added a commit that referenced this issue Nov 4, 2023
@regro-cf-autotick-bot @weiji14
deepspeed v0.12.0 (#34)
e82c3c8 
* updated v0.12.0

* Remove ninja as runtime dependency

Xref #1

* Set DS_BUILD_CUTLASS_OPS=0

Disable buiding with CUTLASS ops, attempt to fix `  ModuleNotFoundError: No module named 'dskernels'`.

* Set DS_BUILD_RAGGED_DEVICE_OPS=0 flag

Disable buiding with ragged device ops, attempt to fix `  ModuleNotFoundError: No module named 'dskernels'`

---------

Co-authored-by: Wei Ji <[email protected]>
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Nov 4, 2023
@weiji14
Remove ninja as runtime dependency
8226b10 
Xref conda-forge#1
weiji14 added a commit that referenced this issue Nov 4, 2023
@regro-cf-autotick-bot @hadim @weiji14
deepspeed v0.12.2 (#37)
be121e7 
* updated v0.12.2

* remove hadim as maintainer

* Remove ninja as runtime dependency

Xref #1

* MNT: Re-rendered with conda-build 3.27.0, conda-smithy 3.27.1, and conda-forge-pinning 2023.11.02.10.11.59

---------

Co-authored-by: Hadrien Mary <[email protected]>
Co-authored-by: Wei Ji <[email protected]>
Co-authored-by: conda-forge-webservices[bot] <91080706+conda-forge-webservices[bot]@users.noreply.github.com>
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Dec 21, 2023
@weiji14
Remove ninja as runtime dependency and reset build number
07db270 
Xref conda-forge#1
weiji14 added a commit that referenced this issue Dec 21, 2023
@regro-cf-autotick-bot @weiji14
deepspeed v0.12.6 (#48)
d577169 
* updated v0.12.6

* Remove ninja as runtime dependency and reset build number

Xref #1

---------

Co-authored-by: Wei Ji <[email protected]>
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Jan 24, 2024
@weiji14
Remove ninja as runtime dependency
155eed6 
Xref conda-forge#1
weiji14 added a commit that referenced this issue Jan 24, 2024
@regro-cf-autotick-bot @weiji14
deepspeed v0.13.1 (#50)
4c0b607 
* updated v0.13.1

* MNT: Re-rendered with conda-build 3.28.4, conda-smithy 3.30.4, and conda-forge-pinning 2024.01.23.16.47.43

* Remove ninja as runtime dependency

Xref #1

---------

Co-authored-by: Wei Ji <[email protected]>
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Feb 12, 2024
@weiji14
Remove ninja as runtime dependency
808f629 
Xref conda-forge#1
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Feb 27, 2024
@weiji14
Remove ninja as runtime dependency
314aaf0 
Xref conda-forge#1
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Mar 12, 2024
@weiji14
Remove ninja as runtime dependency
98c8698 
Xref conda-forge#1
weiji14 added a commit that referenced this issue Mar 12, 2024
@regro-cf-autotick-bot @weiji14
deepspeed v0.13.5 (#54)
c278ace 
* updated v0.13.5

* Remove ninja as runtime dependency

Xref #1

* MNT: Re-rendered with conda-build 3.27.0, conda-smithy 3.31.1, and conda-forge-pinning 2024.03.12.02.51.02

* Configure Azure to free disk space

Xref conda-forge/cudatoolkit-feedstock#91

* MNT: Re-rendered with conda-build 24.1.2, conda-smithy 3.31.1, and conda-forge-pinning 2024.03.12.02.51.02

---------

Co-authored-by: Wei Ji <[email protected]>
Co-authored-by: conda-forge-webservices[bot] <91080706+conda-forge-webservices[bot]@users.noreply.github.com>
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Mar 12, 2024
@weiji14
Remove ninja as runtime dependency
912e9ad 
Xref conda-forge#1
weiji14 added a commit that referenced this issue Mar 12, 2024
@regro-cf-autotick-bot @weiji14
deepspeed v0.14.0 (#55)
04c6285 
* updated v0.14.0

* Remove ninja as runtime dependency

Xref #1

---------

Co-authored-by: Wei Ji <[email protected]>
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Apr 17, 2024
@weiji14
Remove ninja as runtime dependency
b6f6d13 
Xref conda-forge#1
@weiji14 weiji14 mentioned this issue Jun 12, 2024
Closed
5 tasks
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Jun 12, 2024
@weiji14
Remove ninja as runtime dependency
32f1f92 
Xref conda-forge#1
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Aug 11, 2024
@weiji14
Remove ninja as runtime dependency
63758b8 
Xref conda-forge#1
weiji14 added a commit that referenced this issue Aug 11, 2024
@regro-cf-autotick-bot @weiji14
deepspeed v0.14.4 (#63)
b0193a7 
* updated v0.14.4

* MNT: Re-rendered with conda-build 24.5.1, conda-smithy 3.36.2, and conda-forge-pinning 2024.06.21.08.07.40

* Remove ninja as runtime dependency

Xref #1

* Replace pynvml with nvidia-ml-py

Xref deepspeedai/DeepSpeed#5529.

Also added note about compatibility with pydantic 2.0.

* Reset build number to 0

* Add oneccl-devel to host dependencies

Try to fix `fatal error: oneapi/ccl.hpp: No such file or directory` on CUDA builds using suggestion at deepspeedai/DeepSpeed#5653 (comment)

* MNT: Re-rendered with conda-build 24.5.1, conda-smithy 3.38.0, and conda-forge-pinning 2024.08.11.18.23.17

---------

Co-authored-by: Wei Ji <[email protected]>
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Aug 16, 2024
@weiji14
Remove ninja as runtime dependency
dabd840 
Xref conda-forge#1
weiji14 added a commit that referenced this issue Aug 16, 2024
@regro-cf-autotick-bot @weiji14
deepspeed v0.14.5 (#64)
f447b18 
* updated v0.14.5

* Remove ninja as runtime dependency

Xref #1

* Require nvidia-ml-py only on CUDA builds

Xref deepspeedai/DeepSpeed#5840

---------

Co-authored-by: Wei Ji <[email protected]>
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Aug 23, 2024
@weiji14
Remove ninja as runtime dependency
dae2ddd 
Xref conda-forge#1
weiji14 added a commit that referenced this issue Aug 29, 2024
@regro-cf-autotick-bot @weiji14
deepspeed v0.15.0 (#65)
260a70f 
* updated v0.15.0

* MNT: Re-rendered with conda-build 24.7.1, conda-smithy 3.38.0, and conda-forge-pinning 2024.08.23.00.41.48

* Remove ninja as runtime dependency

Xref #1

* Remove note about pydantic<2.0.0 pin

Since deepspeedai/DeepSpeed#5167 is merged already.

---------

Co-authored-by: Wei Ji <[email protected]>
weiji14 added a commit that referenced this issue Sep 9, 2024
@weiji14
Re-enable OSX64 builds (#68)
8f671aa 
* Bump build number to 1

* Remove ninja as runtime dependency

Xref #1

* Sort host dependencies alphabetically

* Re-enable osx x64_64 builds

* Add setuptools to host dependencies

* MNT: Re-rendered with conda-build 24.7.1, conda-smithy 3.39.1, and conda-forge-pinning 2024.09.05.15.59.11

---------

Co-authored-by: conda-forge-webservices[bot] <91080706+conda-forge-webservices[bot]@users.noreply.github.com>
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Oct 10, 2024
@weiji14
Remove ninja as runtime dependency
d23bc1f 
Xref conda-forge#1
weiji14 added a commit that referenced this issue Oct 11, 2024
@regro-cf-autotick-bot @weiji14
deepspeed v0.15.2 (#70)
e4219ee 
* updated v0.15.2

* MNT: Re-rendered with conda-build 24.9.0, conda-smithy 3.41.1, and conda-forge-pinning 2024.10.09.19.05.11

* Sort runtime dependencies alphabetically

* Remove ninja as runtime dependency

Xref #1

---------

Co-authored-by: Wei Ji <[email protected]>
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Oct 23, 2024
@weiji14
Remove ninja as runtime dependency
0c788f8 
Xref conda-forge#1
weiji14 added a commit that referenced this issue Oct 23, 2024
@regro-cf-autotick-bot @weiji14
deepspeed v0.15.3 (#71)
4747897 
* updated v0.15.3

* MNT: Re-rendered with conda-build 24.9.0, conda-smithy 3.43.0, and conda-forge-pinning 2024.10.22.08.35.23

* Remove ninja as runtime dependency

Xref #1

---------

Co-authored-by: Wei Ji <[email protected]>
weiji14 added a commit to weiji14/deepspeed-feedstock that referenced this issue Nov 12, 2024
@weiji14
Remove ninja as runtime dependency
0feb55c 
Xref conda-forge#1
weiji14 added a commit that referenced this issue Nov 12, 2024
@weiji14
Recipe cleanup and re-enable some DS_BUILD_ ops (#74)
0f1ef59 
* Remove if-branch for CUDA 11.2

No more CUDA 11.2 builds on conda-forge since conda-forge/conda-forge-pinning-feedstock#5339

* Remove ninja as runtime dependency

Xref #1

* Bump build number to 1

* Try building with EvoFormerAttention

* Try building with CUTLASS ops

* Try building with ragged device ops

* MNT: Re-rendered with conda-build 24.9.0, conda-smithy 3.44.3, and conda-forge-pinning 2024.11.11.08.59.26
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Nov 26, 2024
@weiji14
Remove ninja as runtime dependency
eaf77b6 
Xref conda-forge#1
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Dec 6, 2024
@weiji14
Remove ninja as runtime dependency
fcf0232 
Xref conda-forge#1
weiji14 added a commit that referenced this issue Dec 6, 2024
@regro-cf-autotick-bot @weiji14
deepspeed v0.16.1 (#76)
7da4363 
* updated v0.16.1

* MNT: Re-rendered with conda-build 24.11.2, conda-smithy 3.44.9, and conda-forge-pinning 2024.12.05.23.59.20

* Remove ninja as runtime dependency

Xref #1

---------

Co-authored-by: Wei Ji <[email protected]>
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Dec 19, 2024
@weiji14
Remove ninja as runtime dependency
513f61e 
Xref conda-forge#1
weiji14 added a commit that referenced this issue Dec 19, 2024
@regro-cf-autotick-bot @weiji14
deepspeed v0.16.2 (#77)
056f307 
* updated v0.16.2

* Remove ninja as runtime dependency

Xref #1

---------

Co-authored-by: Wei Ji <[email protected]>
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Feb 16, 2025
@weiji14
Remove ninja as runtime dependency
f73e5f5 
Xref conda-forge#1
weiji14 added a commit to regro-cf-autotick-bot/deepspeed-feedstock that referenced this issue Feb 20, 2025
@weiji14
Remove ninja as runtime dependency
5ae7668 
Xref conda-forge#1
weiji14 added a commit that referenced this issue Mar 4, 2025
@regro-cf-autotick-bot @weiji14 @h-vetinari
deepspeed v0.16.4 (#84)
e7c8c22 
* updated v0.16.4

* MNT: Re-rendered with conda-build 25.1.2, conda-smithy 3.45.4, and conda-forge-pinning 2025.02.19.18.36.33

* Remove ninja as runtime dependency

Xref #1

* Rebuild for pytorch 2.6

* skip CUDA 11.8

* MNT: Re-rendered with conda-build 24.9.0, conda-smithy 3.46.1, and conda-forge-pinning 2025.03.04.20.48.20

---------

Co-authored-by: Wei Ji <[email protected]>
Co-authored-by: h-vetinari <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@hadim