Skip to content

[quant][graphmode] FP16 quant support - Operator Fusion #40710

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
supriyar wants to merge 4 commits into from
Closed

[quant][graphmode] FP16 quant support - Operator Fusion #40710

supriyar wants to merge 4 commits into from

Conversation

@supriyar
Copy link
Contributor

@supriyar supriyar commented Jun 29, 2020 •
edited
Loading

Stack from ghstack:

Summary:

Test Plan:
python test/test_quantization.py

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D22335975

@supriyar
[quant][graphmode] FP16 quant support - Operator Fusion
f341636 
Summary:

Test Plan:
python test/test_quantization.py

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@supriyar supriyar requested a review from apaszke as a code owner June 29, 2020 17:47
This was referenced Jun 29, 2020
Closed
Closed
@facebook-github-bot facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Jun 29, 2020
@supriyar
Update on "[quant][graphmode] FP16 quant support - Operator Fusion"
43b3683 
Summary:

Test Plan:
python test/test_quantization.py

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
supriyar added a commit that referenced this pull request Jun 29, 2020
@supriyar
[quant][graphmode] FP16 quant support - Operator Fusion
5677e40 
Summary:

Test Plan:
python test/test_quantization.py

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: 3e7f69c
Pull Request resolved: #40710
@dr-ci
Copy link

dr-ci bot commented Jun 29, 2020 •
edited
Loading

💊 CI failures summary and remediations

As of commit a7ea6ba (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 8 times.

jerryzh168
jerryzh168 reviewed Jun 29, 2020
View reviewed changes
test/quantization/test_quantize_jit.py

for model in [model_traced, model_script]:
model_quantized = quantize_dynamic_jit(model, qconfig_dict, debug=False)
# TODO check model with debug=True matches quantized model result
Copy link
Contributor

@jerryzh168 jerryzh168 Jun 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can also check numerics with checkGraphModeOp

Copy link
Contributor Author

@supriyar supriyar Jun 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, right now looks like debug option set to True isn't producing identical numerics. Once I fix/figure that out I'll use checkGraphModeOp

jerryzh168
jerryzh168 reviewed Jun 29, 2020
View reviewed changes
@supriyar
Update on "[quant][graphmode] FP16 quant support - Operator Fusion"
81fb3c2 
Summary:

Test Plan:
python test/test_quantization.py

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
supriyar added a commit that referenced this pull request Jun 30, 2020
@supriyar
[quant][graphmode] FP16 quant support - Operator Fusion
bb9a36d 
Summary:

Test Plan:
python test/test_quantization.py

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: cb4d875
Pull Request resolved: #40710
jerryzh168
jerryzh168 reviewed Jun 30, 2020
View reviewed changes
torch/csrc/jit/passes/quantization/finalize.cpp
namespace torch {
namespace jit {

struct PatternReplaceInfo {
Copy link
Contributor

@jerryzh168 jerryzh168 Jun 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have this in quantization_patterns.h

Copy link
Contributor

@jerryzh168 jerryzh168 Jun 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might make sense to move the patterns there as well

Copy link
Contributor Author

@supriyar supriyar Jun 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can do this in a follow-up PR and move all patterns in this file there.

jerryzh168
jerryzh168 reviewed Jun 30, 2020
View reviewed changes
jerryzh168
jerryzh168 reviewed Jun 30, 2020
View reviewed changes
torch/csrc/jit/passes/quantization/finalize.cpp Outdated
{linear_with_quant, linear_with_quant_prepack},
{linear_fp16_with_cast,
linear_fp16_with_prepack,
{is_fp16_fp32_cast_op}}};
Copy link
Contributor

@jerryzh168 jerryzh168 Jun 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non_blocking is still not checked

Copy link
Contributor Author

@supriyar supriyar Jun 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that param doesn't have to be a specific value, so I omitted that check

Copy link
Contributor

@jerryzh168 jerryzh168 Jun 30, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did we insert the non_blocking = true code? I think it is better to check this as well, I'm not sure how non_blocking = true can be simulated with the swapped pattern

Copy link
Contributor Author

@supriyar supriyar Jun 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually non-blocking and copy are checked with the false check. The missing one is memory_format which should be None - I'll check for that.

jerryzh168
jerryzh168 reviewed Jun 30, 2020
View reviewed changes
@supriyar
Update on "[quant][graphmode] FP16 quant support - Operator Fusion"
a7ea6ba 
Summary:

Test Plan:
python test/test_quantization.py

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
supriyar added a commit that referenced this pull request Jun 30, 2020
@supriyar
[quant][graphmode] FP16 quant support - Operator Fusion
297542f 
Summary:

Test Plan:
python test/test_quantization.py

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: 2660b4e
Pull Request resolved: #40710
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Jul 1, 2020

This pull request has been merged in 26543e6.

@supriyar supriyar mentioned this pull request Jul 2, 2020
Closed
@facebook-github-bot facebook-github-bot deleted the gh/supriyar/146/head branch July 5, 2020 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jerryzh168 jerryzh168 jerryzh168 approved these changes

@apaszke apaszke Awaiting requested review from apaszke

@raghuramank100 raghuramank100 Awaiting requested review from raghuramank100

Assignees

No one assigned

Labels

Merged oncall: jit Add this issue/PR to JIT oncall triage queue

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@supriyar @facebook-github-bot @jerryzh168 @mruberry