Skip to content

[CUDA][CI][cusparselt] Only CUDA 11.8 ships the libcusparseLt.so.0, CUDA 12 would use PYPI libcusparselt #138547

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
nWEIdia wants to merge 6 commits into from
Closed

[CUDA][CI][cusparselt] Only CUDA 11.8 ships the libcusparseLt.so.0, CUDA 12 would use PYPI libcusparselt #138547

nWEIdia wants to merge 6 commits into from

Conversation

@nWEIdia
Copy link
Collaborator

@nWEIdia nWEIdia commented Oct 22, 2024

since nvidia-cusparselt-cu12 is available and
nvidia-cusparselt-cu11 is not available

Related: #138175

@nWEIdia nWEIdia requested a review from a team as a code owner October 22, 2024 05:03
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 22, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/138547

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 5f4dbfe with merge base 51a3d6d (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@nWEIdia nWEIdia added the ciflow/binaries_wheel Trigger binary build and upload jobs for wheel on the PR label Oct 22, 2024
@nWEIdia nWEIdia force-pushed the main-cusparselt-decouple-contd branch from 81146da to d540b64 Compare October 22, 2024 05:07
@nWEIdia nWEIdia added the release notes: cuda release notes category label Oct 22, 2024
@nWEIdia nWEIdia changed the title [CUDA][CI][cusparselt] Only CUDA 11.8 ships the libcusparseLt.so.0, CUDA 12 would use PYPI libcusparselt [Draft][CUDA][CI][cusparselt] Only CUDA 11.8 ships the libcusparseLt.so.0, CUDA 12 would use PYPI libcusparselt Oct 22, 2024
@atalman
Copy link
Contributor

atalman commented Oct 22, 2024

@nWEIdia to fix failures above you need to add code to:

https://github.com/pytorch/pytorch/pull/138547/files#diff-bf46b4a09ca67e50622bf84fefc0d11b584ffcc24ee6cc5019cf0fc7565d81a8R173

echo "Using nvidia libs from pypi."
        CUDA_RPATHS=(
            '$ORIGIN/../../nvidia/cublas/lib'
            '$ORIGIN/../../nvidia/cuda_cupti/lib'
            '$ORIGIN/../../nvidia/cuda_nvrtc/lib'
            '$ORIGIN/../../nvidia/cuda_runtime/lib'
            '$ORIGIN/../../nvidia/cudnn/lib'
            '$ORIGIN/../../nvidia/cufft/lib'
            '$ORIGIN/../../nvidia/curand/lib'
            '$ORIGIN/../../nvidia/cusolver/lib'
            '$ORIGIN/../../nvidia/cusparse/lib'
            '$ORIGIN/../../nvidia/nccl/lib'
            '$ORIGIN/../../nvidia/nvtx/lib'
        )

Rpath of the dynamically installed cusparselt library

@bdhirsh bdhirsh added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Oct 23, 2024
@nWEIdia
[CUDA][CI][cusparselt] Only CUDA 11.8 ships the libcusparseLt.so.0, C…
fe37122 
…UDA 12 would use PYPI libcusparselt
@nWEIdia
Add cusparselt
dd8cea2
@nWEIdia
Test libcusparselt installation dir
c4ea838
@nWEIdia
Add missing Requires-Dist: nvidia-cusparselt-cu12 for cu121
7ab5ea1 
Requires-Dist: nvidia-cuda-nvrtc-cu12 (==12.1.105) ; platform_system == "Linux" and platform_machine == "x86_64"
Requires-Dist: nvidia-cuda-runtime-cu12 (==12.1.105) ; platform_system == "Linux" and platform_machine == "x86_64"
Requires-Dist: nvidia-cuda-cupti-cu12 (==12.1.105) ; platform_system == "Linux" and platform_machine == "x86_64"
Requires-Dist: nvidia-cudnn-cu12 (==9.1.0.70) ; platform_system == "Linux" and platform_machine == "x86_64"
Requires-Dist: nvidia-cublas-cu12 (==12.1.3.1) ; platform_system == "Linux" and platform_machine == "x86_64"
Requires-Dist: nvidia-cufft-cu12 (==11.0.2.54) ; platform_system == "Linux" and platform_machine == "x86_64"
Requires-Dist: nvidia-curand-cu12 (==10.3.2.106) ; platform_system == "Linux" and platform_machine == "x86_64"
Requires-Dist: nvidia-cusolver-cu12 (==11.4.5.107) ; platform_system == "Linux" and platform_machine == "x86_64"
Requires-Dist: nvidia-cusparse-cu12 (==12.1.0.106) ; platform_system == "Linux" and platform_machine == "x86_64"
Requires-Dist: nvidia-nccl-cu12 (==2.21.5) ; platform_system == "Linux" and platform_machine == "x86_64"
Requires-Dist: nvidia-nvtx-cu12 (==12.1.105) ; platform_system == "Linux" and platform_machine == "x86_64"
Requires-Dist: pytorch-triton (==3.1.0+cf34004b8a) ; platform_system == "Linux" and platform_machine == "x86_64" and python_version < "3.13"
Requires-Dist: setuptools ; python_version >= "3.12"
@nWEIdia
Now fixing the less frequent case: full binary build should package
9bb4462 
libcusparselt.so.0
@nWEIdia
Rebase and modify comment about cu118 cusparselt
5f4dbfe
@nWEIdia nWEIdia force-pushed the main-cusparselt-decouple-contd branch from 7883e7a to 5f4dbfe Compare November 5, 2024 07:01
@nWEIdia nWEIdia added the ciflow/trunk Trigger trunk jobs on your pull request label Nov 5, 2024
@atalman atalman changed the title [Draft][CUDA][CI][cusparselt] Only CUDA 11.8 ships the libcusparseLt.so.0, CUDA 12 would use PYPI libcusparselt [CUDA][CI][cusparselt] Only CUDA 11.8 ships the libcusparseLt.so.0, CUDA 12 would use PYPI libcusparselt Nov 5, 2024
atalman
atalman approved these changes Nov 5, 2024
View reviewed changes
Copy link
Contributor

@atalman atalman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@atalman
Copy link
Contributor

atalman commented Nov 5, 2024

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Nov 5, 2024

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

pobin6 pushed a commit to pobin6/pytorch that referenced this pull request Dec 5, 2024
@nWEIdia
[CUDA][CI][cusparselt] Only CUDA 11.8 ships the libcusparseLt.so.0, C…
a15ff37 
…UDA 12 would use PYPI libcusparselt (pytorch#138547)

since nvidia-cusparselt-cu12 is available and
nvidia-cusparselt-cu11 is not available

Related: pytorch#138175
Pull Request resolved: pytorch#138547
Approved by: https://github.com/atalman
@nWEIdia
Copy link
Collaborator Author

nWEIdia commented Jan 9, 2025

Cross link #144477

@oraluben oraluben mentioned this pull request Jan 10, 2025
Closed
@Skylion007 Skylion007 mentioned this pull request May 29, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@atalman atalman atalman approved these changes

Assignees

No one assigned

Labels

ciflow/binaries_wheel Trigger binary build and upload jobs for wheel on the PR ciflow/trunk Trigger trunk jobs on your pull request Merged open source release notes: cuda release notes category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@nWEIdia @atalman @pytorchmergebot @bdhirsh @pytorchbot