add channels last with mixed data type support for GroupNorm backward #89485

CaoE · 2022-11-22T05:44:08Z

Stack from ghstack (oldest at bottom):

-> add channels last with mixed data type support for GroupNorm backward #89485

Motivation

Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last.
Same as add mixed data type support for GroupNorm backward on CPU #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward.

Testing

Single socket (28cores):

Contiguous:

shape	forward / s	forward / s	backward / s	backward / s
	fp32	mixed fp32 bf16	fp32	mixed fp32 bf16
[10, 128, 20, 20]	3.20E-05	3.60E-05	8.31E-05	8.13E-05
[10, 128, 50, 50]	0.000126	0.000115	0.000356	0.000257

Channels Last:

shape	forward / s	forward / s	backward / s	backward / s
	fp32	mixed fp32 bf16	fp32	mixed fp32 bf16
[10, 128, 20, 20]	4.11E-05	4.12E-05	9.74E-05	9.66E-05
[10, 128, 50, 50]	0.000179	0.000178	0.000393	0.000317

Single core:

Contiguous:

shape	forward / s	forward / s	backward / s	backward / s
	fp32	mixed fp32 bf16	fp32	mixed fp32 bf16
[10, 128, 20, 20]	2.47E-04	2.53E-04	5.92E-04	4.50E-04
[10, 128, 50, 50]	0.001559	0.001384	0.004343	0.002436

Channels Last:

shape	forward / s	forward / s	backward / s	backward / s
	fp32	mixed fp32 bf16	fp32	mixed fp32 bf16
[10, 128, 20, 20]	2.27E-04	3.24E-04	0.0006224	0.000459
[10, 128, 50, 50]	0.00167	0.001278	0.0041858	0.003027

cc @ezyang @gchanan @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @VitalyFedyunin

pytorch-bot · 2022-11-22T05:44:11Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/89485

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit a8eb674:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 8749cdc Pull Request resolved: #89485

[ghstack-poisoned]

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 [ghstack-poisoned]

ghstack-source-id: 4549af4 Pull Request resolved: #89485

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 [ghstack-poisoned]

ghstack-source-id: 065abc1 Pull Request resolved: #89485

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

ghstack-source-id: bda3595 Pull Request resolved: #89485

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

ghstack-source-id: 9bb35db Pull Request resolved: #89485

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

CaoE · 2022-12-23T01:00:59Z

@pytorchbot merge

pytorchmergebot · 2022-12-23T01:03:00Z

Merge failed

Reason: Not merging any PRs at the moment because there is a merge blocking https://github.com/pytorch/pytorch/labels/ci:%20sev issue open at:
#91332

Details for Dev Infra team

Raised by workflow job

ghstack-source-id: e117d1e Pull Request resolved: #89485

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

ghstack-source-id: 9551495 Pull Request resolved: #89485

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

CaoE · 2022-12-29T05:21:45Z

@pytorchbot merge

pytorchmergebot · 2022-12-29T05:23:32Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

CaoE · 2022-12-29T07:46:13Z

@malfet Is there any adjustments that can be made in later PRs. Thanks for your any suggestions.

jgong5 · 2022-12-29T12:18:37Z

@malfet Is there any adjustments that can be made in later PRs. Thanks for your any suggestions.

@malfet In particular, may I know what code duplicate you were concerned about in this PR. We can address this in the follow-up PR. Thanks.

malfet · 2023-01-19T22:50:07Z

This PR regressed training step for https://github.com/lucidrains/DALLE2-pytorch, see #92166

CaoE · 2023-01-20T03:08:28Z

@malfet Sorry for the incontinence. Can we move the .grads[0].contiguous and input.contiguous from https://github.com/pytorch/pytorch/pull/89485/files#diff-e4c2f99a2404e98c3586e07425da73008f36b1bada790648a7297af141d37f8cL1171 to the native_group_norm_backward function in aten/src/ATen/native/group_norm.cpp and add memory format check for X and dY . In this way, we can neither break the channel last on CPU nor affect CUDA.

malfet · 2023-01-20T03:10:48Z

@malfet Sorry for the incontinence. Can we move the .grads[0].contiguous and input.contiguous from https://github.com/pytorch/pytorch/pull/89485/files#diff-e4c2f99a2404e98c3586e07425da73008f36b1bada790648a7297af141d37f8cL1171 to the native_group_norm_backward function in aten/src/ATen/native/group_norm.cpp and add memory format check for X and dY . In this way, we can neither break the channel last on CPU nor affect CUDA.

That's what I'm doing: forcing the continuous on the CUDA side, but also adding the unit test so it would not regress again

CaoE · 2023-01-20T03:14:49Z

@malfet Thank you very much ! I will submit a PR for the issue.

CaoE · 2023-01-20T04:57:18Z

@malfet I submitted a PR #92668 to fix it but I have no CUDA machine to verify if the issue is fixed. Could you please help to check it ?

Fixes regression introduced by #89485 Adds test to prevent those regressions from happening in the future In process, discovered that GroupNormBackwards on CPU does not produce the same results if input and gradient memory_format is different

malfet · 2023-01-20T06:00:12Z

@CaoE This sound very similar to the approach I've taken in #92671 and it fixes the problem for CUDA, but surprisingly I had to skip part of the test on CPU, as I think current implementation silently produces wrong results, but probably requires some further refactoring...

Fixes regression introduced by #89485 Adds test to prevent those regressions from happening in the future In process, discovered that GroupNormBackwards on CPU does not produce the same results if input and gradient memory_format is different Fixes #92166 Pull Request resolved: #92671 Approved by: https://github.com/ngimel, https://github.com/xuzhao9

CaoE requested review from albanD and soulitzer as code owners November 22, 2022 05:44

CaoE mentioned this pull request Nov 22, 2022

add mixed data type support for LayerNorm and GroupNorm #88662

Closed

pytorch-bot bot added the release notes: nn release notes category label Nov 22, 2022

CaoE mentioned this pull request Nov 22, 2022

add mixed data type support for GroupNorm backward on CPU #88663

Closed

github-actions bot added the module: cpu CPU specific problem (e.g., perf, algorithm) label Nov 22, 2022

CaoE marked this pull request as draft November 22, 2022 05:45

pytorchbot added the open source label Nov 22, 2022

CaoE requested review from jgong5 and mingfeima November 22, 2022 05:55

CaoE added a commit that referenced this pull request Nov 22, 2022

add channels last with mixed data type support for GroupNorm backward

8e46193

ghstack-source-id: 8749cdc Pull Request resolved: #89485

CaoE added 3 commits November 22, 2022 12:42

add channels last with mixed data type support for GroupNorm backward

7d92503

[ghstack-poisoned]

CaoE added a commit that referenced this pull request Nov 23, 2022

add channels last with mixed data type support for GroupNorm backward

7580cf1

ghstack-source-id: 4549af4 Pull Request resolved: #89485

jgong5 approved these changes Nov 23, 2022

View reviewed changes

CaoE added 2 commits November 23, 2022 08:45

CaoE added the intel This tag is for PR from Intel label Nov 30, 2022

CaoE added a commit that referenced this pull request Dec 5, 2022

add channels last with mixed data type support for GroupNorm backward

5e6347d

ghstack-source-id: 065abc1 Pull Request resolved: #89485

CaoE added a commit that referenced this pull request Dec 9, 2022

add channels last with mixed data type support for GroupNorm backward

3fd6cad

ghstack-source-id: bda3595 Pull Request resolved: #89485

CaoE added the ciflow/trunk Trigger trunk jobs on your pull request label Dec 13, 2022

CaoE added a commit that referenced this pull request Dec 13, 2022

add channels last with mixed data type support for GroupNorm backward

2fe54ee

ghstack-source-id: 9bb35db Pull Request resolved: #89485

CaoE added the intel priority matters to intel architecture from performance wise label Dec 13, 2022

CaoE added 2 commits December 21, 2022 09:35

CaoE added a commit that referenced this pull request Dec 26, 2022

add channels last with mixed data type support for GroupNorm backward

0439d51

ghstack-source-id: e117d1e Pull Request resolved: #89485

CaoE added a commit that referenced this pull request Dec 28, 2022

add channels last with mixed data type support for GroupNorm backward

60d74c5

ghstack-source-id: 9551495 Pull Request resolved: #89485

pytorchmergebot added the Merged label Dec 29, 2022

pytorchmergebot closed this in 5030929 Dec 29, 2022

malfet added the module: bc-breaking Related to a BC-breaking change label Jan 19, 2023

pytorch-bot bot added the topic: bc breaking topic category label Jan 19, 2023

malfet mentioned this pull request Jan 20, 2023

Fix GroupNorm backward prop on CUDA #92671

Closed

CaoE mentioned this pull request Jan 28, 2023

Align input memory format and grad memory format for GroupNorm backward #92668

Closed

facebook-github-bot deleted the gh/CaoE/7/head branch June 8, 2023 14:30

add channels last with mixed data type support for GroupNorm backward #89485

add channels last with mixed data type support for GroupNorm backward #89485

Uh oh!

Conversation

CaoE commented Nov 22, 2022 • edited by pytorch-bot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Testing

Uh oh!

pytorch-bot bot commented Nov 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/89485

✅ No Failures

Uh oh!

CaoE commented Dec 23, 2022

Uh oh!

pytorchmergebot commented Dec 23, 2022

Merge failed

Uh oh!

CaoE commented Dec 29, 2022

Uh oh!

pytorchmergebot commented Dec 29, 2022

Merge started

Uh oh!

CaoE commented Dec 29, 2022

Uh oh!

jgong5 commented Dec 29, 2022

Uh oh!

malfet commented Jan 19, 2023

Uh oh!

CaoE commented Jan 20, 2023

Uh oh!

malfet commented Jan 20, 2023

Uh oh!

CaoE commented Jan 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CaoE commented Jan 20, 2023

Uh oh!

malfet commented Jan 20, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

CaoE commented Nov 22, 2022 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Nov 22, 2022 •

edited

Loading

CaoE commented Jan 20, 2023 •

edited

Loading