Skip to content

Add Attention ops to CI#165915

Closed
jainapurva wants to merge 39 commits intomainfrom
ci_attn
Closed

Add Attention ops to CI#165915
jainapurva wants to merge 39 commits intomainfrom
ci_attn

Conversation

@jainapurva
Copy link
Contributor

@jainapurva jainapurva commented Oct 20, 2025 •
edited
Loading

This pull request introduces a new attention operator microbenchmark workflow to the CI system, enabling automated benchmarking and reporting for attention-related operations. The main changes include adding a new GitHub Actions workflow, to add attention benchmarks to the existing Pytorch operator microbenchmark dashboard

@pytorch-bot
Copy link

pytorch-bot bot commented Oct 20, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/165915

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 1 Unrelated Failure

As of commit 9473b9e with merge base a6a0379 (image):

NEW FAILURES - The following jobs have failed:

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the release notes: releng release notes category label Oct 20, 2025
@jainapurva jainapurva changed the base branch from main to attention_benchmark October 20, 2025 16:38
@jainapurva jainapurva added the ciflow/op-benchmark Trigger microbenchmark for operations. label Oct 20, 2025
@pytorch-bot pytorch-bot bot added ciflow/b200 ciflow/h100 ciflow/rocm Trigger "default" config CI on ROCm labels Oct 21, 2025
@jainapurva jainapurva added topic: performance topic category and removed release notes: releng release notes category ciflow/rocm Trigger "default" config CI on ROCm ciflow/h100 ciflow/b200 labels Oct 21, 2025
@pytorch-bot pytorch-bot bot added ciflow/b200 ciflow/h100 ciflow/rocm Trigger "default" config CI on ROCm labels Oct 21, 2025
@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Nov 11, 2025

Merge started

Your change will be merged while ignoring the following 2 checks: operator_benchmark / aarch64-opbenchmark-test / test (cpu_operator_benchmark_short, 1, 1, linux.arm64.m8g.4xlarge), operator_benchmark / x86-opbenchmark-test / test (cpu_operator_benchmark_short, 1, 1, linux.12xlarge)

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Nov 11, 2025

Merge failed

Reason: 1 jobs have failed, first few of them are: trunk / win-vs2022-cpu-py3 / test (default, 1, 4, windows.4xlarge.nonephemeral)

Details for Dev Infra team Raised by workflow job

@jainapurva
Copy link
Contributor Author

jainapurva commented Nov 12, 2025

@pytorchmergebot rebase

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Nov 12, 2025

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Nov 12, 2025

Rebase failed due to Command git -C /home/runner/work/pytorch/pytorch rebase refs/remotes/origin/viable/strict pull/165915/head returned non-zero exit code 1

Rebasing (1/28)
Auto-merging benchmarks/transformer/configs/config_basic.yaml
CONFLICT (add/add): Merge conflict in benchmarks/transformer/configs/config_basic.yaml
Auto-merging benchmarks/transformer/score_mod.py
CONFLICT (content): Merge conflict in benchmarks/transformer/score_mod.py
error: could not apply de61804393b... Add attention benchmarking
hint: Resolve all conflicts manually, mark them as resolved with
hint: "git add/rm <conflicted_files>", then run "git rebase --continue".
hint: You can instead skip this commit: run "git rebase --skip".
hint: To abort and get back to the state before "git rebase", run "git rebase --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Could not apply de61804393b... # Add attention benchmarking

Raised by https://github.com/pytorch/pytorch/actions/runs/19310282022

@jainapurva
Copy link
Contributor Author

jainapurva commented Nov 12, 2025

@pytorchmergebot merge

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Nov 12, 2025

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Nov 12, 2025

Merge failed

Reason: 1 jobs have failed, first few of them are: operator_benchmark / aarch64-opbenchmark-test / test (cpu_operator_benchmark_short, 1, 1, linux.arm64.m8g.4xlarge)

Details for Dev Infra team Raised by workflow job

@jainapurva
Copy link
Contributor Author

jainapurva commented Nov 12, 2025

@pytorchmergebot merge -i

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Nov 12, 2025

Merge started

Your change will be merged while ignoring the following 2 checks: trunk / linux-jammy-py3-clang12-executorch / test (executorch, 1, 1, linux.2xlarge, unstable), operator_benchmark / aarch64-opbenchmark-test / test (cpu_operator_benchmark_short, 1, 1, linux.arm64.m8g.4xlarge)

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Nov 12, 2025

Merge failed

Reason: 1 jobs have failed, first few of them are: operator_benchmark / x86-opbenchmark-test / test (cpu_operator_benchmark_short, 1, 1, linux.12xlarge)

Details for Dev Infra team Raised by workflow job

@jainapurva
Copy link
Contributor Author

jainapurva commented Nov 12, 2025

@pytorchmergebot merge -i

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Nov 12, 2025

Merge started

Your change will be merged while ignoring the following 3 checks: trunk / linux-jammy-py3-clang12-executorch / test (executorch, 1, 1, linux.2xlarge, unstable), operator_benchmark / aarch64-opbenchmark-test / test (cpu_operator_benchmark_short, 1, 1, linux.arm64.m8g.4xlarge), operator_benchmark / x86-opbenchmark-test / test (cpu_operator_benchmark_short, 1, 1, linux.12xlarge)

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Nov 13, 2025

The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command
For more information see pytorch-bot wiki.

@jainapurva jainapurva removed the ciflow/op-benchmark Trigger microbenchmark for operations. label Nov 13, 2025
@jainapurva
Copy link
Contributor Author

jainapurva commented Nov 13, 2025

@pytorchmergebot merge -i

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Nov 13, 2025

Merge started

Your change will be merged while ignoring the following 3 checks: trunk / linux-jammy-py3-clang12-executorch / test (executorch, 1, 1, linux.2xlarge, unstable), operator_benchmark / aarch64-opbenchmark-test / test (cpu_operator_benchmark_short, 1, 1, linux.arm64.m8g.4xlarge), operator_benchmark / x86-opbenchmark-test / test (cpu_operator_benchmark_short, 1, 1, linux.12xlarge)

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

Silv3S pushed a commit to Silv3S/pytorch that referenced this pull request Nov 18, 2025
@jainapurva @Silv3S
Add Attention ops to CI (pytorch#165915)
f7d3c74 
This pull request introduces a new attention operator microbenchmark workflow to the CI system, enabling automated benchmarking and reporting for attention-related operations. The main changes include adding a new GitHub Actions workflow, to add attention benchmarks to the existing Pytorch operator microbenchmark [dashboard](https://hud.pytorch.org/benchmark/v3/dashboard/pytorch_operator_microbenchmark?renderGroupId=main&time.start=2025-10-27T00%3A00%3A00.000Z&time.end=2025-10-29T01%3A00%3A00.000Z&filters.device=cuda&filters.arch=NVIDIA+A100-SXM4-40GB&filters.deviceName=cuda%7C%7CNVIDIA+A100-SXM4-40GB&filters.operatorName=&lcommit.commit=665df0bc7288996d638fcc3da750f8cb2addd6d0&lcommit.workflow_id=18888994873&lcommit.date=2025-10-29T00%3A00%3A00Z&lcommit.branch=refs%2Ftags%2Fciflow%2Fop-benchmark%2F165915&rcommit.commit=665df0bc7288996d638fcc3da750f8cb2addd6d0&rcommit.workflow_id=18888994873&rcommit.date=2025-10-29T00%3A00%3A00Z&rcommit.branch=refs%2Ftags%2Fciflow%2Fop-benchmark%2F165915&lbranch=refs%2Ftags%2Fciflow%2Fop-benchmark%2F165915&rbranch=refs%2Ftags%2Fciflow%2Fop-benchmark%2F165915)
Pull Request resolved: pytorch#165915
Approved by: https://github.com/jbschlosser
@github-actions github-actions bot deleted the ci_attn branch December 14, 2025 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jbschlosser jbschlosser jbschlosser approved these changes

@drisspg drisspg Awaiting requested review from drisspg

@huydhn huydhn Awaiting requested review from huydhn

Assignees

No one assigned

Labels

ciflow/trunk Trigger trunk jobs on your pull request Merged release notes: releng release notes category topic: performance topic category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@jainapurva @pytorchmergebot @drisspg @jbschlosser