Add NVFP4 two-level scaling to scaled_mm by slayton58 · Pull Request #165774 · pytorch/pytorch

slayton58 · 2025-10-17T18:06:08Z

Stack from ghstack (oldest at bottom):

-> Add NVFP4 two-level scaling to scaled_mm #165774

Summary:

Add second-level scaling dispatch to scaled_mm, tying into optional alpha passing
Add two-level tests

Test Plan:

pytest -svv -k "nvfp4_global_scale" test/test_scaled_matmul_cuda.py

Reviewers:

Subscribers:

Tasks:

Tags:
Signed-off-by: Simon Layton simonlayton@meta.com

Summary: * Add second-level scaling dispatch to scaled_mm, tying into optional `alpha` passing * Add two-level tests Test Plan: ``` pytest -svv -k "nvfp4_global_scale" test/test_scaled_matmul_cuda.py ``` Reviewers: Subscribers: Tasks: Tags: Signed-off-by: Simon Layton <simonlayton@meta.com> [ghstack-poisoned]

pytorch-bot · 2025-10-17T18:06:12Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/165774

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 3fcc25d with merge base fe80f03 ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

trunk / linux-jammy-py3-clang12-executorch / test (executorch, 1, 1, lf.linux.2xlarge) (gh) (similar failure)
export/tests/test_target_recipes.py::TestTargetRecipes::test_w2l_model

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Summary: * Add second-level scaling dispatch to scaled_mm, tying into optional `alpha` passing * Add two-level tests Test Plan: ``` pytest -svv -k "nvfp4_global_scale" test/test_scaled_matmul_cuda.py ``` Reviewers: Subscribers: Tasks: Tags: Signed-off-by: Simon Layton <simonlaytonmeta.com> ghstack-source-id: 97203e6 Pull Request resolved: #165774

Summary: * Add second-level scaling dispatch to scaled_mm, tying into optional `alpha` passing * Add two-level tests Test Plan: ``` pytest -svv -k "nvfp4_global_scale" test/test_scaled_matmul_cuda.py ``` Reviewers: Subscribers: Tasks: Tags: Signed-off-by: Simon Layton <simonlaytonmeta.com> [ghstack-poisoned]

Summary: * Add second-level scaling dispatch to scaled_mm, tying into optional `alpha` passing * Add two-level tests Test Plan: ``` pytest -svv -k "nvfp4_global_scale" test/test_scaled_matmul_cuda.py ``` Reviewers: Subscribers: Tasks: Tags: Signed-off-by: Simon Layton <simonlaytonmeta.com> ghstack-source-id: 7a02bb7 Pull Request resolved: #165774 Signed-off-by: Simon Layton <simonlayton@meta.com>

aten/src/ATen/native/cuda/Blas.cpp

test/test_scaled_matmul_cuda.py

Summary: * Add second-level scaling dispatch to scaled_mm, tying into optional `alpha` passing * Add two-level tests Test Plan: ``` pytest -svv -k "nvfp4_global_scale" test/test_scaled_matmul_cuda.py ``` Reviewers: Subscribers: Tasks: Tags: Signed-off-by: Simon Layton <simonlaytonmeta.com> [ghstack-poisoned]

Summary: * Add second-level scaling dispatch to scaled_mm, tying into optional `alpha` passing * Add two-level tests Test Plan: ``` pytest -svv -k "nvfp4_global_scale" test/test_scaled_matmul_cuda.py ``` Reviewers: Subscribers: Tasks: Tags: Signed-off-by: Simon Layton <simonlaytonmeta.com> ghstack-source-id: 0ff9d95 Pull Request resolved: #165774 Signed-off-by: Simon Layton <simonlayton@meta.com>

aten/src/ATen/native/cuda/Blas.cpp

Summary: * Add second-level scaling dispatch to scaled_mm, tying into optional `alpha` passing * Add two-level tests Test Plan: ``` pytest -svv -k "nvfp4_global_scale" test/test_scaled_matmul_cuda.py ``` Reviewers: Subscribers: Tasks: Tags: Signed-off-by: Simon Layton <simonlaytonmeta.com> [ghstack-poisoned]

Summary: * Add second-level scaling dispatch to scaled_mm, tying into optional `alpha` passing * Add two-level tests Test Plan: ``` pytest -svv -k "nvfp4_global_scale" test/test_scaled_matmul_cuda.py ``` Reviewers: Subscribers: Tasks: Tags: Signed-off-by: Simon Layton <simonlaytonmeta.com> ghstack-source-id: 7cf9738 Pull Request resolved: #165774 Signed-off-by: Simon Layton <simonlayton@meta.com>

slayton58 · 2025-10-17T22:59:44Z

@pytorchbot merge

pytorchmergebot · 2025-10-17T23:01:41Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

slayton58 · 2025-10-17T23:27:28Z

@pytorchbot rebase -b main

pytorchmergebot · 2025-10-17T23:28:12Z

Merge failed

Reason: 1 mandatory check(s) failed. The first few are:

Lint / lintrunner-noclang-all / linux-job

Dig deeper by viewing the failures on hud

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

pytorchmergebot · 2025-10-17T23:29:01Z

@pytorchbot started a rebase job onto refs/remotes/origin/main. Check the current status here

[ghstack-poisoned]

pytorchmergebot · 2025-10-17T23:29:15Z

Successfully rebased gh/slayton58/33/orig onto refs/remotes/origin/main, please pull locally before adding more changes (for example, via ghstack checkout https://github.com/pytorch/pytorch/pull/165774)

Summary: * Add second-level scaling dispatch to scaled_mm, tying into optional `alpha` passing * Add two-level tests Test Plan: ``` pytest -svv -k "nvfp4_global_scale" test/test_scaled_matmul_cuda.py ``` Reviewers: Subscribers: Tasks: Tags: Signed-off-by: Simon Layton <simonlaytonmeta.com> ghstack-source-id: 79c6033 Pull Request resolved: #165774 Signed-off-by: Simon Layton <simonlayton@meta.com>

slayton58 · 2025-10-18T12:58:09Z

@pytorchbot merge

pytorchmergebot · 2025-10-18T13:00:05Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

Summary: * Add second-level scaling dispatch to scaled_mm, tying into optional `alpha` passing * Add two-level tests Test Plan: ``` pytest -svv -k "nvfp4_global_scale" test/test_scaled_matmul_cuda.py ``` Reviewers: Subscribers: Tasks: Tags: Signed-off-by: Simon Layton <simonlayton@meta.com> Pull Request resolved: pytorch#165774 Approved by: https://github.com/drisspg

slayton58 requested review from Aidyn-A, eqy and syed-ahmed as code owners October 17, 2025 18:06

slayton58 requested review from danielvegamyhre, drisspg and ngimel October 17, 2025 18:06

slayton58 added release notes: quantization release notes category ciflow/rocm Trigger "default" config CI on ROCm ciflow/h100 ciflow/b200 labels Oct 17, 2025

Skylion007 reviewed Oct 17, 2025

View reviewed changes

aten/src/ATen/native/cuda/Blas.cpp Outdated Show resolved Hide resolved

Skylion007 reviewed Oct 17, 2025

View reviewed changes

test/test_scaled_matmul_cuda.py Show resolved Hide resolved

drisspg reviewed Oct 17, 2025

View reviewed changes

aten/src/ATen/native/cuda/Blas.cpp Outdated Show resolved Hide resolved

drisspg approved these changes Oct 17, 2025

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Oct 17, 2025

pytorchmergebot added the merging label Oct 17, 2025

pytorchmergebot removed the merging label Oct 17, 2025

Update

3fcc25d

[ghstack-poisoned]

pytorchmergebot added the merging label Oct 18, 2025

pytorchmergebot added the Merged label Oct 18, 2025

pytorchmergebot closed this in d14cbb4 Oct 18, 2025

pytorchmergebot removed the merging label Oct 18, 2025

github-actions bot deleted the gh/slayton58/33/head branch November 18, 2025 02:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add NVFP4 two-level scaling to scaled_mm#165774

Add NVFP4 two-level scaling to scaled_mm#165774
slayton58 wants to merge 5 commits intogh/slayton58/33/basefrom
gh/slayton58/33/head

slayton58 commented Oct 17, 2025 •

edited by pytorchmergebot

Loading

Uh oh!

pytorch-bot bot commented Oct 17, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

slayton58 commented Oct 17, 2025

Uh oh!

pytorchmergebot commented Oct 17, 2025

Uh oh!

slayton58 commented Oct 17, 2025

Uh oh!

pytorchmergebot commented Oct 17, 2025

Uh oh!

pytorchmergebot commented Oct 17, 2025

Uh oh!

pytorchmergebot commented Oct 17, 2025

Uh oh!

slayton58 commented Oct 18, 2025

Uh oh!

pytorchmergebot commented Oct 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

slayton58 commented Oct 17, 2025 • edited by pytorchmergebot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/165774

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

slayton58 commented Oct 17, 2025

Uh oh!

pytorchmergebot commented Oct 17, 2025

Merge started

Uh oh!

slayton58 commented Oct 17, 2025

Uh oh!

pytorchmergebot commented Oct 17, 2025

Merge failed

Uh oh!

pytorchmergebot commented Oct 17, 2025

Uh oh!

pytorchmergebot commented Oct 17, 2025

Uh oh!

slayton58 commented Oct 18, 2025

Uh oh!

pytorchmergebot commented Oct 18, 2025

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

slayton58 commented Oct 17, 2025 •

edited by pytorchmergebot

Loading

pytorch-bot bot commented Oct 17, 2025 •

edited

Loading