Skip to content

[ROCm] OffsetCalc Unroll Optimization#161700

Closed
amd-hhashemi wants to merge 3 commits intopytorch:mainfrom
amd-hhashemi:ofstCalcOpt
Closed

[ROCm] OffsetCalc Unroll Optimization#161700
amd-hhashemi wants to merge 3 commits intopytorch:mainfrom
amd-hhashemi:ofstCalcOpt

Conversation

@amd-hhashemi
Copy link
Contributor

@amd-hhashemi amd-hhashemi commented Aug 28, 2025 •
edited by pytorch-bot bot
Loading

Our compiler is generating inefficient code for the offsetCalc in certain situations.
The root-cause for this needs to be identified. For now specialized unrolling based on 'dims' notably helps perf.

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang @naromero77amd

@pytorch-bot
Copy link

pytorch-bot bot commented Aug 28, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/161700

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit a190954 with merge base 5babb4d (image):

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the release notes: cuda release notes category label Aug 28, 2025
@amd-hhashemi amd-hhashemi changed the title OffsetCalc Unroll Optimization ROCm] OffsetCalc Unroll Optimization Aug 28, 2025
@pytorch-bot pytorch-bot bot added the module: rocm AMD GPU support for Pytorch label Aug 28, 2025
@amd-hhashemi amd-hhashemi changed the title ROCm] OffsetCalc Unroll Optimization [ROCm] OffsetCalc Unroll Optimization Aug 28, 2025
jerrymannil
jerrymannil reviewed Aug 28, 2025
View reviewed changes
@jeffdaily jeffdaily added release notes: rocm mandatorylabel ciflow/rocm Trigger "default" config CI on ROCm ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 and removed release notes: cuda release notes category labels Aug 28, 2025
@pytorch-bot pytorch-bot bot removed ciflow/rocm Trigger "default" config CI on ROCm ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 labels Aug 30, 2025
Skylion007
Skylion007 reviewed Aug 30, 2025
View reviewed changes
aten/src/ATen/cuda/detail/OffsetCalculator.cuh
offsets[arg] = divmod.mod * strides_[0][arg];
if (dims >= 2) {
divmod = sizes_[1].divmod(divmod.div);
#pragma unroll
Copy link
Collaborator

@Skylion007 Skylion007 Aug 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to add a comment to link to the compiler issue?

Copy link
Contributor Author

@amd-hhashemi amd-hhashemi Sep 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're investigating what is exactly happening in compiler here. Will provide an issue number later when we've rootcaused.

@jeffdaily jeffdaily added ciflow/rocm Trigger "default" config CI on ROCm ciflow/inductor-rocm Trigger "inductor" config CI on ROCm ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 ciflow/periodic-rocm-mi300 Trigger "distributed" config CI on ROCm MI300/MI325 labels Sep 2, 2025
@zou3519 zou3519 added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Sep 3, 2025
jerrymannil added a commit to ROCm/pytorch that referenced this pull request Sep 3, 2025
@jerrymannil
[ROCm] OffsetCalc Unroll Optimization
9955889 
cherry-pick of pytorch#161700

Our compiler is generating inefficient code for the offsetCalc in certain situations.
The root-cause for this needs to be identified. For now specialized unrolling based on 'dims' notably helps perf.
@jerrymannil jerrymannil mentioned this pull request Sep 3, 2025
Merged
jerrymannil added a commit to ROCm/pytorch that referenced this pull request Sep 3, 2025
@jerrymannil
[ROCm] OffsetCalc Unroll Optimization (#2597)
9e206b9 
cherry-pick of pytorch#161700

Our compiler is generating inefficient code for the offsetCalc in
certain situations. The root-cause for this needs to be identified. For
now specialized unrolling based on 'dims' notably helps perf.

Fixes SWDEV-545713, SWDEV-545710
dhonnappa-amd pushed a commit to ROCm/pytorch that referenced this pull request Sep 3, 2025
@jerrymannil
[ROCm] OffsetCalc Unroll Optimization (#2597)
33d172b 
cherry-pick of pytorch#161700

Our compiler is generating inefficient code for the offsetCalc in
certain situations. The root-cause for this needs to be identified. For
now specialized unrolling based on 'dims' notably helps perf.

Fixes SWDEV-545713, SWDEV-545710
dhonnappa-amd pushed a commit to ROCm/pytorch that referenced this pull request Sep 3, 2025
@jerrymannil
[ROCm] OffsetCalc Unroll Optimization (#2597)
c7c79da 
cherry-pick of pytorch#161700

Our compiler is generating inefficient code for the offsetCalc in
certain situations. The root-cause for this needs to be identified. For
now specialized unrolling based on 'dims' notably helps perf.

Fixes SWDEV-545713, SWDEV-545710
@jeffdaily jeffdaily added the keep-going Don't stop on first failure, keep running tests until the end label Sep 8, 2025
@jerrymannil
Copy link
Collaborator

jerrymannil commented Sep 8, 2025

@pytorchbot rebase

1 similar comment
@jeffdaily
Copy link
Collaborator

jeffdaily commented Sep 8, 2025

@pytorchbot rebase

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Sep 8, 2025

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Sep 8, 2025

Successfully rebased ofstCalcOpt onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout ofstCalcOpt && git pull --rebase)

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Sep 8, 2025

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Sep 8, 2025

Successfully rebased ofstCalcOpt onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout ofstCalcOpt && git pull --rebase)

@pytorch-bot pytorch-bot bot removed the ciflow/rocm Trigger "default" config CI on ROCm label Sep 8, 2025
@pytorch-bot pytorch-bot bot removed ciflow/inductor-rocm Trigger "inductor" config CI on ROCm ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 ciflow/periodic-rocm-mi300 Trigger "distributed" config CI on ROCm MI300/MI325 labels Sep 8, 2025
@jeffdaily jeffdaily added ciflow/rocm Trigger "default" config CI on ROCm ciflow/inductor-rocm Trigger "inductor" config CI on ROCm ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 ciflow/periodic-rocm-mi300 Trigger "distributed" config CI on ROCm MI300/MI325 labels Sep 8, 2025
@jerrymannil
Copy link
Collaborator

jerrymannil commented Sep 9, 2025

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 9, 2025
@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Sep 9, 2025

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

markc-614 pushed a commit to markc-614/pytorch that referenced this pull request Sep 17, 2025
@amd-hhashemi @markc-614
[ROCm] OffsetCalc Unroll Optimization (pytorch#161700)
7e970ad 
Our compiler is generating inefficient code for the offsetCalc in certain situations.
The root-cause for this needs to be identified. For now specialized unrolling based on 'dims' notably helps perf.

Pull Request resolved: pytorch#161700
Approved by: https://github.com/jeffdaily
mansiag05 pushed a commit to mansiag05/pytorch that referenced this pull request Sep 22, 2025
@amd-hhashemi @mansiag05
[ROCm] OffsetCalc Unroll Optimization (pytorch#161700)
ea7119b 
Our compiler is generating inefficient code for the offsetCalc in certain situations.
The root-cause for this needs to be identified. For now specialized unrolling based on 'dims' notably helps perf.

Pull Request resolved: pytorch#161700
Approved by: https://github.com/jeffdaily
cleonard530 pushed a commit to cleonard530/pytorch that referenced this pull request Sep 22, 2025
@amd-hhashemi @cleonard530
[ROCm] OffsetCalc Unroll Optimization (pytorch#161700)
5d4a995 
Our compiler is generating inefficient code for the offsetCalc in certain situations.
The root-cause for this needs to be identified. For now specialized unrolling based on 'dims' notably helps perf.

Pull Request resolved: pytorch#161700
Approved by: https://github.com/jeffdaily
dsashidh pushed a commit to dsashidh/pytorch that referenced this pull request Sep 26, 2025
@amd-hhashemi @dsashidh
[ROCm] OffsetCalc Unroll Optimization (pytorch#161700)
3e79cee 
Our compiler is generating inefficient code for the offsetCalc in certain situations.
The root-cause for this needs to be identified. For now specialized unrolling based on 'dims' notably helps perf.

Pull Request resolved: pytorch#161700
Approved by: https://github.com/jeffdaily
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Skylion007 Skylion007 Skylion007 left review comments

@jerrymannil jerrymannil jerrymannil left review comments

@jeffdaily jeffdaily jeffdaily approved these changes

@eqy eqy Awaiting requested review from eqy eqy is a code owner

@syed-ahmed syed-ahmed Awaiting requested review from syed-ahmed syed-ahmed is a code owner

Assignees

No one assigned

Labels

ciflow/inductor-rocm Trigger "inductor" config CI on ROCm ciflow/periodic-rocm-mi300 Trigger "distributed" config CI on ROCm MI300/MI325 ciflow/rocm Trigger "default" config CI on ROCm ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 ciflow/trunk Trigger trunk jobs on your pull request keep-going Don't stop on first failure, keep running tests until the end Merged module: rocm AMD GPU support for Pytorch open source release notes: rocm mandatorylabel triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@amd-hhashemi @jerrymannil @jeffdaily @pytorchmergebot @Skylion007 @zou3519 @pytorchbot