Skip to content

[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase#162998

Closed
xinyazhang wants to merge 8 commits intopytorch:mainfrom
ROCm:xinyazhang/fix-157094-etc
Closed

[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase#162998
xinyazhang wants to merge 8 commits intopytorch:mainfrom
ROCm:xinyazhang/fix-157094-etc

Conversation

@xinyazhang
Copy link
Collaborator

@xinyazhang xinyazhang commented Sep 15, 2025 •
edited by jithunnair-amd
Loading

@pytorch-bot
Copy link

pytorch-bot bot commented Sep 15, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/162998

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures

As of commit ebd8261 with merge base 89a6dbe (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@xinyazhang
Copy link
Collaborator Author

xinyazhang commented Sep 15, 2025

@pytorchbot label "topic: not user facing"

@pytorch-bot pytorch-bot bot added the topic: not user facing topic category label Sep 15, 2025
@jeffdaily jeffdaily changed the title Remove HIPBLASLT_ALLOW_TF32 from codebase [ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase Sep 15, 2025
@pytorch-bot pytorch-bot bot added module: rocm AMD GPU support for Pytorch module: dynamo module: inductor ciflow/inductor ciflow/rocm Trigger "default" config CI on ROCm labels Sep 15, 2025
@pytorch-bot
Copy link

pytorch-bot bot commented Sep 15, 2025

To add the ciflow label ciflow/rocm please first approve the workflows that are awaiting approval (scroll to the bottom of this page).

This helps ensure we don't trigger CI on this PR until it is actually authorized to do so. Please ping one of the reviewers if you do not have access to approve and run workflows.

@pytorch-bot
Copy link

pytorch-bot bot commented Sep 15, 2025

To add the ciflow label ciflow/inductor please first approve the workflows that are awaiting approval (scroll to the bottom of this page).

This helps ensure we don't trigger CI on this PR until it is actually authorized to do so. Please ping one of the reviewers if you do not have access to approve and run workflows.

@pytorch-bot pytorch-bot bot removed ciflow/rocm Trigger "default" config CI on ROCm ciflow/inductor labels Sep 15, 2025
@jeffdaily jeffdaily added ciflow/rocm Trigger "default" config CI on ROCm ciflow/inductor-rocm Trigger "inductor" config CI on ROCm ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 labels Sep 15, 2025
@pytorch-bot pytorch-bot bot removed ciflow/rocm Trigger "default" config CI on ROCm ciflow/inductor-rocm Trigger "inductor" config CI on ROCm ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 labels Sep 15, 2025
@jeffdaily jeffdaily added ciflow/rocm Trigger "default" config CI on ROCm ciflow/inductor-rocm Trigger "inductor" config CI on ROCm ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 labels Sep 15, 2025
@jeffdaily jeffdaily marked this pull request as ready for review September 16, 2025 01:53
@jeffdaily jeffdaily added ciflow/trunk Trigger trunk jobs on your pull request ciflow/inductor ciflow/rocm Trigger "default" config CI on ROCm ciflow/inductor-rocm Trigger "inductor" config CI on ROCm ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 ciflow/slow labels Sep 17, 2025
@jeffdaily
Copy link
Collaborator

jeffdaily commented Sep 18, 2025

I don't understand how we're getting RuntimeError: No HIP GPUs are available in https://github.com/pytorch/pytorch/actions/runs/17803811310/job/50612952691?pr=162998.

@jeffdaily
Copy link
Collaborator

jeffdaily commented Sep 18, 2025

@pytorchbot merge -f "prior failures that caused revert have been fixed, relanding"

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Sep 18, 2025

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

mansiag05 pushed a commit to mansiag05/pytorch that referenced this pull request Sep 22, 2025
@xinyazhang @jeffdaily @mansiag05
[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase (pytorch#162998)
2b8f330 
A few UT failures are caused by `HIPBLASLT_ALLOW_TF32`

Fixes pytorch#157094, pytorch#157093, pytorch#157092, pytorch#157091, pytorch#157064, pytorch#157063, pytorch#157062, pytorch#157061, pytorch#157042, pytorch#157041, pytorch#157039, pytorch#157004

Pull Request resolved: pytorch#162998
Approved by: https://github.com/jeffdaily

Co-authored-by: Jeff Daily <jeff.daily@amd.com>
mansiag05 pushed a commit to mansiag05/pytorch that referenced this pull request Sep 22, 2025
@pytorchmergebot @mansiag05
Revert "[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase (pytorch#162998
eaccab8 
)"

This reverts commit cef815d.

Reverted pytorch#162998 on behalf of https://github.com/huydhn due to Sorry for reverting this, but it seems to break a test in trunk ([comment](pytorch#162998 (comment)))
mansiag05 pushed a commit to mansiag05/pytorch that referenced this pull request Sep 22, 2025
@xinyazhang @jeffdaily @mansiag05
[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase (pytorch#162998)
6c4bb29 
A few UT failures are caused by `HIPBLASLT_ALLOW_TF32`

Fixes pytorch#157094
Fixes pytorch#157093
Fixes pytorch#157092
Fixes pytorch#157091
Fixes pytorch#157064
Fixes pytorch#157063
Fixes pytorch#157062
Fixes pytorch#157061
Fixes pytorch#157042
Fixes pytorch#157041
Fixes pytorch#157039
Fixes pytorch#157004

Pull Request resolved: pytorch#162998
Approved by: https://github.com/jeffdaily

Co-authored-by: Jeff Daily <jeff.daily@amd.com>
cleonard530 pushed a commit to cleonard530/pytorch that referenced this pull request Sep 22, 2025
@xinyazhang @jeffdaily @cleonard530
[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase (pytorch#162998)
4885ff2 
A few UT failures are caused by `HIPBLASLT_ALLOW_TF32`

Fixes pytorch#157094, pytorch#157093, pytorch#157092, pytorch#157091, pytorch#157064, pytorch#157063, pytorch#157062, pytorch#157061, pytorch#157042, pytorch#157041, pytorch#157039, pytorch#157004

Pull Request resolved: pytorch#162998
Approved by: https://github.com/jeffdaily

Co-authored-by: Jeff Daily <jeff.daily@amd.com>
cleonard530 pushed a commit to cleonard530/pytorch that referenced this pull request Sep 22, 2025
@pytorchmergebot @cleonard530
Revert "[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase (pytorch#162998
ac7d9a5 
)"

This reverts commit cef815d.

Reverted pytorch#162998 on behalf of https://github.com/huydhn due to Sorry for reverting this, but it seems to break a test in trunk ([comment](pytorch#162998 (comment)))
cleonard530 pushed a commit to cleonard530/pytorch that referenced this pull request Sep 22, 2025
@xinyazhang @jeffdaily @cleonard530
[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase (pytorch#162998)
73f3167 
A few UT failures are caused by `HIPBLASLT_ALLOW_TF32`

Fixes pytorch#157094
Fixes pytorch#157093
Fixes pytorch#157092
Fixes pytorch#157091
Fixes pytorch#157064
Fixes pytorch#157063
Fixes pytorch#157062
Fixes pytorch#157061
Fixes pytorch#157042
Fixes pytorch#157041
Fixes pytorch#157039
Fixes pytorch#157004

Pull Request resolved: pytorch#162998
Approved by: https://github.com/jeffdaily

Co-authored-by: Jeff Daily <jeff.daily@amd.com>
dsashidh pushed a commit to dsashidh/pytorch that referenced this pull request Sep 26, 2025
@xinyazhang @jeffdaily @dsashidh
[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase (pytorch#162998)
9a11716 
A few UT failures are caused by `HIPBLASLT_ALLOW_TF32`

Fixes pytorch#157094, pytorch#157093, pytorch#157092, pytorch#157091, pytorch#157064, pytorch#157063, pytorch#157062, pytorch#157061, pytorch#157042, pytorch#157041, pytorch#157039, pytorch#157004

Pull Request resolved: pytorch#162998
Approved by: https://github.com/jeffdaily

Co-authored-by: Jeff Daily <jeff.daily@amd.com>
dsashidh pushed a commit to dsashidh/pytorch that referenced this pull request Sep 26, 2025
@pytorchmergebot @dsashidh
Revert "[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase (pytorch#162998
5785c40 
)"

This reverts commit cef815d.

Reverted pytorch#162998 on behalf of https://github.com/huydhn due to Sorry for reverting this, but it seems to break a test in trunk ([comment](pytorch#162998 (comment)))
dsashidh pushed a commit to dsashidh/pytorch that referenced this pull request Sep 26, 2025
@xinyazhang @jeffdaily @dsashidh
[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase (pytorch#162998)
986d18e 
A few UT failures are caused by `HIPBLASLT_ALLOW_TF32`

Fixes pytorch#157094
Fixes pytorch#157093
Fixes pytorch#157092
Fixes pytorch#157091
Fixes pytorch#157064
Fixes pytorch#157063
Fixes pytorch#157062
Fixes pytorch#157061
Fixes pytorch#157042
Fixes pytorch#157041
Fixes pytorch#157039
Fixes pytorch#157004

Pull Request resolved: pytorch#162998
Approved by: https://github.com/jeffdaily

Co-authored-by: Jeff Daily <jeff.daily@amd.com>
@xinyazhang xinyazhang mentioned this pull request Oct 3, 2025
Merged
slojosic-amd pushed a commit to ROCm/pytorch that referenced this pull request Oct 15, 2025
@xinyazhang @jeffdaily @slojosic-amd
[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase (pytorch#162998)
e8c56af 
A few UT failures are caused by `HIPBLASLT_ALLOW_TF32`

Fixes pytorch#157094
Fixes pytorch#157093
Fixes pytorch#157092
Fixes pytorch#157091
Fixes pytorch#157064
Fixes pytorch#157063
Fixes pytorch#157062
Fixes pytorch#157061
Fixes pytorch#157042
Fixes pytorch#157041
Fixes pytorch#157039
Fixes pytorch#157004

Pull Request resolved: pytorch#162998
Approved by: https://github.com/jeffdaily

Co-authored-by: Jeff Daily <jeff.daily@amd.com>
@slojosic-amd slojosic-amd mentioned this pull request Oct 15, 2025
Merged
ScottTodd pushed a commit to ScottTodd/pytorch that referenced this pull request Oct 15, 2025
@xinyazhang @jeffdaily @ScottTodd
[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase (pytorch#162998)
3126983 
A few UT failures are caused by `HIPBLASLT_ALLOW_TF32`

Fixes pytorch#157094
Fixes pytorch#157093
Fixes pytorch#157092
Fixes pytorch#157091
Fixes pytorch#157064
Fixes pytorch#157063
Fixes pytorch#157062
Fixes pytorch#157061
Fixes pytorch#157042
Fixes pytorch#157041
Fixes pytorch#157039
Fixes pytorch#157004

Pull Request resolved: pytorch#162998
Approved by: https://github.com/jeffdaily

Co-authored-by: Jeff Daily <jeff.daily@amd.com>
ScottTodd pushed a commit to ROCm/pytorch that referenced this pull request Oct 15, 2025
@xinyazhang @jeffdaily @ScottTodd
[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase (pytorch#162998)
3211347 
A few UT failures are caused by `HIPBLASLT_ALLOW_TF32`

Fixes pytorch#157094
Fixes pytorch#157093
Fixes pytorch#157092
Fixes pytorch#157091
Fixes pytorch#157064
Fixes pytorch#157063
Fixes pytorch#157062
Fixes pytorch#157061
Fixes pytorch#157042
Fixes pytorch#157041
Fixes pytorch#157039
Fixes pytorch#157004

Pull Request resolved: pytorch#162998
Approved by: https://github.com/jeffdaily

Co-authored-by: Jeff Daily <jeff.daily@amd.com>
jithunnair-amd pushed a commit to ROCm/pytorch that referenced this pull request Oct 22, 2025
@xinyazhang @jeffdaily @jithunnair-amd
[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase (pytorch#162998)
3a62a4d 
A few UT failures are caused by `HIPBLASLT_ALLOW_TF32`

Fixes pytorch#157094
Fixes pytorch#157093
Fixes pytorch#157092
Fixes pytorch#157091
Fixes pytorch#157064
Fixes pytorch#157063
Fixes pytorch#157062
Fixes pytorch#157061
Fixes pytorch#157042
Fixes pytorch#157041
Fixes pytorch#157039
Fixes pytorch#157004

Pull Request resolved: pytorch#162998
Approved by: https://github.com/jeffdaily

Co-authored-by: Jeff Daily <jeff.daily@amd.com>
jeffdaily added a commit to ROCm/pytorch that referenced this pull request Nov 17, 2025
@xinyazhang @jeffdaily
[ROCm] Remove HIPBLASLT_ALLOW_TF32 from codebase (pytorch#162998)
dd64d0b 
A few UT failures are caused by `HIPBLASLT_ALLOW_TF32`

Fixes pytorch#157094
Fixes pytorch#157093
Fixes pytorch#157092
Fixes pytorch#157091
Fixes pytorch#157064
Fixes pytorch#157063
Fixes pytorch#157062
Fixes pytorch#157061
Fixes pytorch#157042
Fixes pytorch#157041
Fixes pytorch#157039
Fixes pytorch#157004

Pull Request resolved: pytorch#162998
Approved by: https://github.com/jeffdaily

Co-authored-by: Jeff Daily <jeff.daily@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jeffdaily jeffdaily jeffdaily approved these changes

@eqy eqy Awaiting requested review from eqy eqy is a code owner

@syed-ahmed syed-ahmed Awaiting requested review from syed-ahmed syed-ahmed is a code owner

@lezcano lezcano Awaiting requested review from lezcano lezcano is a code owner

@nikitaved nikitaved Awaiting requested review from nikitaved nikitaved is a code owner

@IvanYashchuk IvanYashchuk Awaiting requested review from IvanYashchuk IvanYashchuk is a code owner

Assignees

No one assigned

Labels

ci-no-td Do not run TD on this PR ciflow/inductor ciflow/inductor-rocm Trigger "inductor" config CI on ROCm ciflow/rocm Trigger "default" config CI on ROCm ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 ciflow/slow ciflow/trunk Trigger trunk jobs on your pull request Merged module: dynamo module: inductor module: rocm AMD GPU support for Pytorch open source Reverted topic: not user facing topic category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

DISABLED test_scaled_dot_product_attention_3D_input_dim_2D_causal_attn_mask_dropout_p_0_0_cuda (__main__.TestTransformersCUDA) DISABLED test_scaled_dot_product_attention_4D_input_dim_4D_causal_attn_mask_dropout_p_0_2_cuda (__main__.TestTransformersCUDA) DISABLED test_scaled_dot_product_attention_4D_input_dim_2D_causal_attn_mask_dropout_p_0_2_cuda (__main__.TestTransformersCUDA) DISABLED test_scaled_dot_product_attention_3D_input_dim_2D_causal_attn_mask_dropout_p_0_2_cuda (__main__.TestTransformersCUDA) DISABLED test_scaled_dot_product_attention_input_dim_4_cuda (__main__.TestNestedTensorDeviceTypeCUDA)
DISABLED test_scaled_dot_product_attention_3D_input_dim_3D_causal_attn_mask_dropout_p_0_0_cuda (__main__.TestTransformersCUDA) DISABLED test_scaled_dot_product_attention_4D_input_dim_2D_causal_attn_mask_dropout_p_0_0_cuda (__main__.TestTransformersCUDA) DISABLED test_scaled_dot_product_attention_4D_input_dim_4D_causal_attn_mask_dropout_p_0_0_cuda (__main__.TestTransformersCUDA) DISABLED test_scaled_dot_product_attention_3D_input_dim_2D_causal_attn_mask_dropout_p_0_5_cuda (__main__.TestTransformersCUDA) DISABLED test_scaled_dot_product_attention_3D_input_dim_3D_causal_attn_mask_dropout_p_0_5_cuda (__main__.TestTransformersCUDA)

5 participants

@xinyazhang @jeffdaily @pytorchmergebot @huydhn @pytorchbot