Skip to content

[ao][sparsity] Composability of fusion and sparsity #74847

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
HDCharles wants to merge 23 commits into from
Closed

[ao][sparsity] Composability of fusion and sparsity #74847

HDCharles wants to merge 23 commits into from

Conversation

@HDCharles
Copy link
Contributor

@HDCharles HDCharles commented Mar 28, 2022 •
edited
Loading

Stack from ghstack (oldest at bottom):

Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D35240273

@HDCharles
Composability of sparsity and fusion
8555378 
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
This was referenced Mar 28, 2022
Closed
Closed
Closed
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Mar 28, 2022 •
edited
Loading

🔗 Helpful links

💊 CI failures summary and remediations

As of commit a42631a (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@HDCharles
Update on "Composability of sparsity and fusion"
c769d5f 
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
a2152de 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@HDCharles HDCharles changed the title Composability of sparsity and fusion [ao][sparsity] Composability of fusion and sparsity Mar 28, 2022
@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
e6e1af9 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
6f5473e 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
9e306b4 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
ec8e726 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
vkuzo
vkuzo reviewed Mar 29, 2022
View reviewed changes
torch/nn/intrinsic/modules/fused.py Outdated

# from torch.ao.quantization.utils import nonparam_type
from torch.nn.utils.parametrize import is_parametrized
def nonparam_type(module):
Copy link
Contributor

@vkuzo vkuzo Mar 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as previous PR, type_before_parametrizations?

Copy link
Contributor Author

@HDCharles HDCharles Mar 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
d7d8c57 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
014d7aa 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@albanD albanD removed their request for review March 29, 2022 18:09
@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
6e707aa 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@HDCharles
Copy link
Contributor Author

HDCharles commented Mar 30, 2022

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

vkuzo
vkuzo approved these changes Mar 30, 2022
View reviewed changes
torch/nn/intrinsic/modules/fused.py Outdated
import torch
from torch.nn import Conv1d, Conv2d, Conv3d, ReLU, Linear, BatchNorm1d, BatchNorm2d, BatchNorm3d

# from torch.ao.quantization.utils import type_before_parametrizations
Copy link
Contributor

@vkuzo vkuzo Mar 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove?

Copy link
Contributor Author

@HDCharles HDCharles Mar 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
02724d3 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D35240273](https://our.internmc.facebook.com/intern/diff/D35240273)

[ghstack-poisoned]
@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
5c015ae 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D35240273](https://our.internmc.facebook.com/intern/diff/D35240273)

[ghstack-poisoned]
@HDCharles
Copy link
Contributor Author

HDCharles commented Mar 30, 2022

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
fc705b8 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D35240273](https://our.internmc.facebook.com/intern/diff/D35240273)

[ghstack-poisoned]
@HDCharles
Copy link
Contributor Author

HDCharles commented Mar 31, 2022

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
3f3938c 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D35240273](https://our.internmc.facebook.com/intern/diff/D35240273)

[ghstack-poisoned]
@HDCharles
Copy link
Contributor Author

HDCharles commented Apr 4, 2022

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
b6cf179 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D35240273](https://our.internmc.facebook.com/intern/diff/D35240273)

[ghstack-poisoned]
@HDCharles
Copy link
Contributor Author

HDCharles commented Apr 4, 2022

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

1 similar comment
@HDCharles
Copy link
Contributor Author

HDCharles commented Apr 4, 2022

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
1dd0a5b 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D35240273](https://our.internmc.facebook.com/intern/diff/D35240273)

[ghstack-poisoned]
@HDCharles
Copy link
Contributor Author

HDCharles commented Apr 5, 2022

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

HDCharles added 3 commits April 6, 2022 14:39
@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
aebaa6a 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D35240273](https://our.internmc.facebook.com/intern/diff/D35240273)

[ghstack-poisoned]
@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
c23550e 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D35240273](https://our.internmc.facebook.com/intern/diff/D35240273)

[ghstack-poisoned]
@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
1947945 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D35240273](https://our.internmc.facebook.com/intern/diff/D35240273)

[ghstack-poisoned]
@HDCharles
Copy link
Contributor Author

HDCharles commented Apr 6, 2022

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
585b949 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D35240273](https://our.internmc.facebook.com/intern/diff/D35240273)

[ghstack-poisoned]
@HDCharles
Copy link
Contributor Author

HDCharles commented Apr 6, 2022

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
75caa3f 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D35240273](https://our.internmc.facebook.com/intern/diff/D35240273)

[ghstack-poisoned]
@HDCharles
Copy link
Contributor Author

HDCharles commented Apr 7, 2022

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
76cecf9 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D35240273](https://our.internmc.facebook.com/intern/diff/D35240273)

[ghstack-poisoned]
@HDCharles
Copy link
Contributor Author

HDCharles commented Apr 7, 2022

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@HDCharles
Update on "[ao][sparsity] Composability of fusion and sparsity"
a42631a 
Summary: Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan: python test/test_ao_sparsity.py TestComposability

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D35240273](https://our.internmc.facebook.com/intern/diff/D35240273)

[ghstack-poisoned]
@HDCharles
Copy link
Contributor Author

HDCharles commented Apr 7, 2022

@HDCharles has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot pushed a commit that referenced this pull request Apr 8, 2022
@HDCharles @facebook-github-bot
[ao][sparsity] Composability of fusion and sparsity (#74847)
2adb002 
Summary:
Pull Request resolved: #74847

Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.

Test Plan:
python test/test_ao_sparsity.py TestComposability

Imported from OSS

Reviewed By: vkuzo, andrewor14, jerryzh168

Differential Revision: D35240273

fbshipit-source-id: 62ed66689b285c3fa68f1e149266ab877f1cdd8e
@github-actions
Copy link
Contributor

github-actions bot commented Apr 8, 2022

Hey @HDCharles.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

@WBobby WBobby mentioned this pull request Aug 17, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vkuzo vkuzo vkuzo approved these changes

@andrewor14 andrewor14 andrewor14 approved these changes

@jerryzh168 jerryzh168 jerryzh168 approved these changes

@jbschlosser jbschlosser Awaiting requested review from jbschlosser

@dzdang dzdang Awaiting requested review from dzdang

@z-a-f z-a-f Awaiting requested review from z-a-f

@supriyar supriyar Awaiting requested review from supriyar

@terrychenism terrychenism Awaiting requested review from terrychenism

Assignees

No one assigned

Labels

cla signed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@HDCharles @facebook-github-bot @vkuzo @andrewor14 @jerryzh168