Skip to content

Autocast support for scaled_dot_product_attention #91066

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mikekgfb wants to merge 1 commit into from
Closed

Autocast support for scaled_dot_product_attention #91066

mikekgfb wants to merge 1 commit into from

Conversation

@mikekgfb
Copy link
Contributor

@mikekgfb mikekgfb commented Dec 17, 2022 •
edited by pytorch-bot bot
Loading

Summary: Autocast support for scaled_dot_product_attention

Test Plan: sandcastle and guthub cicd

Differential Revision: D42085525

cc @mcarilli @ptrblck @leslie-fang-intel @jgong5

@pytorch-bot
Copy link

pytorch-bot bot commented Dec 17, 2022 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/91066

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Failures

As of commit ac9af55:

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Dec 17, 2022

This pull request was exported from Phabricator. Differential Revision: D42085525

ngimel
ngimel reviewed Dec 17, 2022
View reviewed changes
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Dec 19, 2022

This pull request was exported from Phabricator. Differential Revision: D42085525

1 similar comment
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Dec 19, 2022

This pull request was exported from Phabricator. Differential Revision: D42085525

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Dec 19, 2022

This pull request was exported from Phabricator. Differential Revision: D42085525

@cpuhrsch cpuhrsch requested a review from drisspg December 19, 2022 03:48
@mikekgfb
Copy link
Contributor Author

mikekgfb commented Dec 19, 2022

@pytorchbot merge -f

@pytorch-bot
Copy link

pytorch-bot bot commented Dec 19, 2022

❌ 🤖 pytorchbot command failed:

@pytorchbot merge: error: argument -f/--force: expected one argument

usage: @pytorchbot merge [-g | -f MESSAGE | -l] [-r [{viable/strict,master}]]

Try @pytorchbot --help for more info.

@mikekgfb
Copy link
Contributor Author

mikekgfb commented Dec 19, 2022

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Dec 19, 2022
@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Dec 19, 2022

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Dec 19, 2022

Merge failed

Reason: 2 additional jobs have failed, first few of them are: trunk ,trunk / cuda11.6-py3.10-gcc7-sm86 / test (default, 3, 4, linux.g5.4xlarge.nvidia.gpu)

Details for Dev Infra team Raised by workflow job

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Dec 19, 2022

This pull request was exported from Phabricator. Differential Revision: D42085525

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Dec 19, 2022

This pull request was exported from Phabricator. Differential Revision: D42085525

mikekgfb pushed a commit to mikekgfb/pytorch that referenced this pull request Dec 19, 2022
@facebook-github-bot
Autocast support for scaled_dot_product_attention (pytorch#91066)
f044877 
Summary:
Pull Request resolved: pytorch#91066

Autocast support for scaled_dot_product_attention

Test Plan: sandcastle and guthub cicd

Differential Revision: D42085525

fbshipit-source-id: afc6746775c747a7e0829821bc5ef4a10945445f
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Dec 19, 2022

This pull request was exported from Phabricator. Differential Revision: D42085525

mikekgfb pushed a commit to mikekgfb/pytorch that referenced this pull request Dec 19, 2022
@facebook-github-bot
Autocast support for scaled_dot_product_attention (pytorch#91066)
f726889 
Summary:
Pull Request resolved: pytorch#91066

Autocast support for scaled_dot_product_attention

Test Plan: sandcastle and guthub cicd

Differential Revision: D42085525

fbshipit-source-id: 95aa9268f83b714f13e9c09ebb207aab14fc01f0
@albanD albanD removed their request for review December 19, 2022 13:00
drisspg
drisspg requested changes Dec 19, 2022
View reviewed changes
test/test_transformers.py Outdated
def test_flash_fail_fp32t(self):
device = 'cuda'
dtype = torch.float
size = (2, 2, 8, 5)
Copy link
Contributor

@drisspg drisspg Dec 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flash doesn't accept head dim size of 5 so I don't this is failing for dtype 32 which we do already test with. If you decorate with auto cast does this enable flash?

Copy link
Contributor Author

@mikekgfb mikekgfb Dec 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. good catch. I had cut and pasted another test. (Lolz, the one right above that was showing that this size does not work. :eyeroll:

Copy link
Collaborator

@ngimel ngimel Dec 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hah, I was wondering about weird sizes, but I don't know by heart what flash does and doesn't support

@mikekgfb
Copy link
Contributor Author

mikekgfb commented Dec 19, 2022

@pytorchbot merge

@pytorch-bot
Copy link

pytorch-bot bot commented Dec 19, 2022

This PR needs to be approved by an authorized maintainer before merge.

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Dec 19, 2022

This pull request was exported from Phabricator. Differential Revision: D42085525

drisspg
drisspg approved these changes Dec 19, 2022
View reviewed changes
Copy link
Contributor

@drisspg drisspg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@facebook-github-bot
Autocast support for scaled_dot_product_attention (pytorch#91066)
ac9af55 
Summary:
Pull Request resolved: pytorch#91066

Autocast support for scaled_dot_product_attention

Test Plan: sandcastle and guthub cicd

Reviewed By: drisspg

Differential Revision: D42085525

fbshipit-source-id: 58adc0daf573bdb628f040babb123e8100c7fbd3
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Dec 19, 2022

This pull request was exported from Phabricator. Differential Revision: D42085525

@mikekgfb
Copy link
Contributor Author

mikekgfb commented Dec 19, 2022

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Dec 19, 2022

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Dec 19, 2022

Merge failed

Reason: The following mandatory check(s) failed (Rule superuser):

Dig deeper by viewing the failures on hud

Details for Dev Infra team Raised by workflow job

@mikekgfb
Copy link
Contributor Author

mikekgfb commented Dec 19, 2022

@pytorchbot merge -f "ignore cancelled check"

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Dec 19, 2022

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ngimel ngimel ngimel approved these changes

@drisspg drisspg drisspg approved these changes

@cpuhrsch cpuhrsch Awaiting requested review from cpuhrsch

Assignees

No one assigned

Labels

ciflow/trunk Trigger trunk jobs on your pull request fb-exported Merged module: amp (automated mixed precision) autocast

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@mikekgfb @facebook-github-bot @pytorchmergebot @ngimel @drisspg