Introduce causal mask #90508

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Closed

mikekgfb wants to merge 1 commit into pytorch:master from mikekgfb:export-D41723137

Contributor

mikekgfb commented Dec 8, 2022 •

edited

Loading

Summary: Introduce causal mask

This PR introduces a causal mask option is_causal, since current custom kernels do not support arbitrary masks.
(Also, building these masks is expensive, and avoiding to build them is a significant opportunity since causal mask is extremely common for training Transformer models.)

Test Plan: sandcastle & github ci/cd

Differential Revision: D41723137

mikekgfb requested review from albanD and jbschlosser as code owners

December 8, 2022 23:43

pytorch-bot bot commented Dec 8, 2022 •

edited

Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/90508

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 Failures

As of commit 5560eb4:

NEW FAILURES - The following jobs have failed:

lintrunner

BROKEN TRUNK - The following jobs failed but were present on the merge base 01e7f46:

Test tools

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot added the fb-exported label

Contributor

facebook-github-bot commented Dec 8, 2022

This pull request was exported from Phabricator. Differential Revision: D41723137

1 similar comment

Contributor

facebook-github-bot commented Dec 8, 2022

This pull request was exported from Phabricator. Differential Revision: D41723137

mikekgfb force-pushed the export-D41723137 branch from dc8ff81 to fd44c03 Compare

December 8, 2022 23:52

mikekgfb force-pushed the export-D41723137 branch from fd44c03 to 035ce36 Compare

December 9, 2022 00:24

Contributor

facebook-github-bot commented Dec 9, 2022

This pull request was exported from Phabricator. Differential Revision: D41723137

mikekgfb force-pushed the export-D41723137 branch from 035ce36 to 5fa6d34 Compare

December 9, 2022 01:44

Contributor

facebook-github-bot commented Dec 9, 2022

This pull request was exported from Phabricator. Differential Revision: D41723137

mikekgfb force-pushed the export-D41723137 branch from 5fa6d34 to b5e948f Compare

December 9, 2022 02:08

Contributor

facebook-github-bot commented Dec 9, 2022

This pull request was exported from Phabricator. Differential Revision: D41723137

mikekgfb force-pushed the export-D41723137 branch from b5e948f to 9f5c9b2 Compare

December 9, 2022 02:17

Contributor

facebook-github-bot commented Dec 9, 2022

This pull request was exported from Phabricator. Differential Revision: D41723137

1 similar comment

Contributor

facebook-github-bot commented Dec 9, 2022

This pull request was exported from Phabricator. Differential Revision: D41723137

mikekgfb force-pushed the export-D41723137 branch from 9f5c9b2 to fb42950 Compare

December 9, 2022 17:04

mikekgfb force-pushed the export-D41723137 branch from fb42950 to 552160e Compare

December 9, 2022 17:17

Contributor

facebook-github-bot commented Dec 9, 2022

This pull request was exported from Phabricator. Differential Revision: D41723137

1 similar comment

Contributor

facebook-github-bot commented Dec 9, 2022

This pull request was exported from Phabricator. Differential Revision: D41723137

mikekgfb force-pushed the export-D41723137 branch from 552160e to 9c85b9b Compare

December 9, 2022 17:53

mikekgfb force-pushed the export-D41723137 branch from 4aa2325 to c29f729 Compare

December 16, 2022 00:17

Contributor

facebook-github-bot commented Dec 16, 2022

This pull request was exported from Phabricator. Differential Revision: D41723137

mikekgfb force-pushed the export-D41723137 branch from c29f729 to d529803 Compare

December 16, 2022 02:02

Contributor

facebook-github-bot commented Dec 16, 2022

This pull request was exported from Phabricator. Differential Revision: D41723137

albanD reviewed

View reviewed changes

Collaborator

albanD left a comment

Change sounds ok. Docstrings need to be updated.

torch/nn/modules/transformer.py Outdated Show resolved Hide resolved

mikekgfb requested a review from albanD

December 16, 2022 10:18

Contributor

facebook-github-bot commented Dec 16, 2022

This pull request was exported from Phabricator. Differential Revision: D41723137

mikekgfb force-pushed the export-D41723137 branch from d529803 to d7a17f1 Compare

December 16, 2022 10:20

albanD reviewed

View reviewed changes

torch/nn/functional.py Outdated Show resolved Hide resolved

torch/nn/modules/transformer.py Outdated Show resolved Hide resolved

torch/nn/modules/transformer.py Outdated Show resolved Hide resolved

Contributor

facebook-github-bot commented Dec 16, 2022

This pull request was exported from Phabricator. Differential Revision: D41723137

mikekgfb force-pushed the export-D41723137 branch from d7a17f1 to abd8ef9 Compare

December 16, 2022 10:50

mikekgfb requested a review from albanD

December 16, 2022 10:52

albanD approved these changes

View reviewed changes

Collaborator

albanD left a comment •

edited

Loading

The change itself sounds ok.
You most likely want to rebase above master to avoid trunk failures.


          Introduce causal mask (pytorch#90508)

5560eb4

Summary:
Pull Request resolved: pytorch#90508

Introduce causal mask

Test Plan: sandcastle & github ci/cd

Reviewed By: albanD, mleshen

Differential Revision: D41723137

fbshipit-source-id: 5cb222d672e8f174a1447a91414068ba87490dc3

Contributor

facebook-github-bot commented Dec 16, 2022

This pull request was exported from Phabricator. Differential Revision: D41723137

mikekgfb force-pushed the export-D41723137 branch from abd8ef9 to 5560eb4 Compare

December 16, 2022 16:03

Contributor Author

mikekgfb commented Dec 16, 2022

@pytorchbot -f merge

pytorch-bot bot commented Dec 16, 2022

❌ 🤖 pytorchbot command failed:

@pytorchbot: error: unrecognized arguments: -f

usage: @pytorchbot [-h] {merge,revert,rebase,label,drci} ...

Try @pytorchbot --help for more info.

Contributor Author

mikekgfb commented Dec 16, 2022

@pytorchbot merge -f

pytorch-bot bot commented Dec 16, 2022

❌ 🤖 pytorchbot command failed:

@pytorchbot merge: error: argument -f/--force: expected one argument

usage: @pytorchbot merge [-g | -f MESSAGE | -l] [-r [{viable/strict,master}]]

Try @pytorchbot --help for more info.

Contributor Author

mikekgfb commented Dec 16, 2022 •

edited

Loading

@pytorchbot merge -f "lint runner was broken. Using -f as suggested"

Collaborator

pytorchmergebot commented Dec 16, 2022

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot added the Merged label

pytorchmergebot closed this in

512ec18

pytorch-bot bot added the release notes: AO frontend label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fb-exported Merged release notes: AO frontend