Skip to content

NPU Adaption for FLUX#9751

Merged
sayakpaul merged 15 commits intohuggingface:mainfrom
leisuzz:main
Nov 1, 2024
Merged

NPU Adaption for FLUX#9751
sayakpaul merged 15 commits intohuggingface:mainfrom
leisuzz:main

Conversation

@leisuzz
Copy link
Contributor

@leisuzz leisuzz commented Oct 23, 2024 •
edited
Loading

What does this PR do?

Implement Flash Attention for NPU machine
Add cleaning memory function for NPU machine

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

yiyixuxu
yiyixuxu reviewed Oct 23, 2024
View reviewed changes
@leisuzz
Copy link
Contributor Author

leisuzz commented Oct 24, 2024

@yiyixuxu @sayakpaul Please let me know if there is any improvement for this PR, and thanks for your help!

yiyixuxu
yiyixuxu approved these changes Oct 25, 2024
View reviewed changes
Copy link
Collaborator

@yiyixuxu yiyixuxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@yiyixuxu
Copy link
Collaborator

yiyixuxu commented Oct 25, 2024

can you run make style and make fix-copies to the CI would pass?

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Oct 25, 2024

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@leisuzz
Copy link
Contributor Author

leisuzz commented Oct 26, 2024

@yiyixuxu I've fixed the code quality issue, please approve the workflow. Thanks

@leisuzz
Copy link
Contributor Author

leisuzz commented Oct 28, 2024

@yiyixuxu Sorry about my mistake, could you please approve the workflow? Thanks

@yiyixuxu
Copy link
Collaborator

yiyixuxu commented Oct 28, 2024

hi @leisuzz
if you installed our dev environment, you can run make style and make fix-copeis from the terminal that will automatically adjust the style for you so the CI will pass

see here https://huggingface.co/docs/diffusers/en/conceptual/contribution#how-to-open-a-pr

@leisuzz
Copy link
Contributor Author

leisuzz commented Oct 29, 2024

@yiyixuxu Thanks for the information!

@leisuzz
Copy link
Contributor Author

leisuzz commented Oct 31, 2024

Hi @yiyixuxu ,
Sorry for the trouble, I think there is something wrong with the ruff in my env as there is no code quality in my end. But I think I've fix it manually. Please approve the workflow. Thanks for your help

@leisuzz
Copy link
Contributor Author

leisuzz commented Oct 31, 2024

@yiyixuxu
Could you please redo the approval of the workflow? Thanks for your help and sorry for all the troubles!

@leisuzz
Copy link
Contributor Author

leisuzz commented Oct 31, 2024

Hi @yiyixuxu ,
There was a copyright / consistency issue with "def fuse_qkv_projections" as it has to match models.unets.unet_2d_condition.UNet2DConditionModel.fuse_qkv_projections. Does that mean I have to change both of these at the same time? For now, please accept this workflows as fuse_qkv_projections can be added later in another PR.

@leisuzz
Copy link
Contributor Author

leisuzz commented Nov 1, 2024

@yiyixuxu @sayakpaul Could you please approve the workflow? Thanks for your help!

@sayakpaul
Copy link
Member

sayakpaul commented Nov 1, 2024

Failing tests are unrelated.

@sayakpaul sayakpaul closed this Nov 1, 2024
@sayakpaul sayakpaul reopened this Nov 1, 2024
@sayakpaul sayakpaul merged commit 9dcac83 into huggingface:main Nov 1, 2024
a-r-r-o-w pushed a commit that referenced this pull request Nov 1, 2024
@leisuzz @a-r-r-o-w
NPU Adaption for FLUX (#9751)
68a90d8 
* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

---------

Co-authored-by: 蒋硕 <jiangshuo9@h-partners.com>
sayakpaul pushed a commit that referenced this pull request Dec 23, 2024
@leisuzz @sayakpaul
NPU Adaption for FLUX (#9751)
ad754e6 
* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

* NPU implementation for FLUX

---------

Co-authored-by: 蒋硕 <jiangshuo9@h-partners.com>
@a-r-r-o-w
Copy link
Contributor

a-r-r-o-w commented Feb 23, 2025

@yiyixuxu I'm not completely sure why we merged this PR with the following changes:

image

I believe we should default to just the normal SDPA processors otherwise there is inconsistencies introduced. Users can already set custom attention processors, no?

@a-r-r-o-w a-r-r-o-w mentioned this pull request Feb 23, 2025
Merged
@yiyixuxu
Copy link
Collaborator

yiyixuxu commented Feb 24, 2025 •
edited
Loading

@a-r-r-o-w
yeah we should revert this change - it was my fault, I realized we should not do that here https://github.com/huggingface/diffusers/pull/10465/files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiyixuxu yiyixuxu yiyixuxu approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@leisuzz @yiyixuxu @HuggingFaceDocBuilderDev @sayakpaul @a-r-r-o-w