Skip to content

[ROCm] Disable group gemm CK path when composable kernel (CK) is not enabled#167403

Closed
jagadish-amd wants to merge 2 commits intopytorch:mainfrom
jagadish-amd:disable_ck_group_gemm_onWin
Closed

[ROCm] Disable group gemm CK path when composable kernel (CK) is not enabled#167403
jagadish-amd wants to merge 2 commits intopytorch:mainfrom
jagadish-amd:disable_ck_group_gemm_onWin

Conversation

@jagadish-amd
Copy link
Contributor

@jagadish-amd jagadish-amd commented Nov 8, 2025

For ROCm builds without CK support, ensure use_fast_path is false so that the CK path is not triggered, since CK is currently not available in this configuration.

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang @naromero77amd

For ROCm windows, make sure use_fast_path is false
so that CK path is not triggered since CK is currently
not enabled on windows.

Signed-off-by: Jagadish Krishnamoorthy <jagadish.krishnamoorthy@amd.com>
@pytorch-bot
Copy link

pytorch-bot bot commented Nov 8, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/167403

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 26 Pending, 2 Unrelated Failures

As of commit 0a6d3c9 with merge base 2325c51 (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the module: rocm AMD GPU support for Pytorch label Nov 8, 2025
@jagadish-amd
Copy link
Contributor Author

@ScottTodd Can you please review, it will be great if you can confirm that the ROCm windows build is success with this PR.
cc @jeffdaily

@jagadish-amd
Copy link
Contributor Author

@pytorchbot label "topic: not user facing"

@pytorch-bot pytorch-bot bot added the topic: not user facing topic category label Nov 8, 2025
Copy link
Contributor

@ScottTodd ScottTodd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer a revert for obviously broken code over an attempt to fix forward.

Signed-off-by: Jagadish Krishnamoorthy <jagadish.krishnamoorthy@amd.com>
@jagadish-amd jagadish-amd changed the title [ROCm] Disable group gemm CK path on ROCm Windows [ROCm] Disable group gemm CK path when composable kernel (CK) is not enabled Nov 8, 2025
@jagadish-amd
Copy link
Contributor Author

Build is success with
-- USE_ROCM : ON
-- USE_ROCM_CK_GEMM : OFF
group gemm tests are passing with slow mode. (python test/test_matmul_cuda.py -v -k test_grouped_gemm)
With USE_ROCM_CK_GEMM ON, group gemm tests are passing with fast mode.

Copy link
Contributor

@ScottTodd ScottTodd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks about right to me, but please revert before attempting to fix forward. We've had broken nightly release builds for 5 days now (https://github.com/ROCm/TheRock/actions/workflows/release_windows_pytorch_wheels.yml?query=branch%3Amain) and it will be at least another 1-2 days before we'll have a chance at clear signal again even if this is reviewed+approved+merged quickly.

@jeffdaily
Copy link
Collaborator

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Nov 11, 2025
@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@jeffdaily
Copy link
Collaborator

@pytorchbot merge -f "rocm only changes, rocm build successful, lint passing"

@pytorchmergebot
Copy link
Collaborator

The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command
For more information see pytorch-bot wiki.

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

Silv3S pushed a commit to Silv3S/pytorch that referenced this pull request Nov 18, 2025
…enabled (pytorch#167403)

For ROCm builds without CK support, ensure use_fast_path is false so that the CK path is not triggered, since CK is currently not available in this configuration.

Pull Request resolved: pytorch#167403
Approved by: https://github.com/Skylion007, https://github.com/ScottTodd, https://github.com/jeffdaily
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk Trigger trunk jobs on your pull request Merged module: rocm AMD GPU support for Pytorch open source topic: not user facing topic category

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants