Skip to content

Hunyuan VAE tiling fixes and transformer docs#10295

Merged
a-r-r-o-w merged 3 commits intomainfrom
hunyuan-fixes
Dec 19, 2024
Merged

Hunyuan VAE tiling fixes and transformer docs#10295
a-r-r-o-w merged 3 commits intomainfrom
hunyuan-fixes

Conversation

@a-r-r-o-w
Copy link
Contributor

@a-r-r-o-w a-r-r-o-w commented Dec 19, 2024

While training loras, found some things that slipped by when we merged HunyuanVideo

  • Enables gradient checkpointing in transformer
  • Fixes the encode implementation in VAE
  • Lowers the default frame tile size. A tile size of 64 requires ~24 GB memory, while a tile size of 16 requires only ~8 GB at 768x512 resolution
  • Adds docstring for transformer

@a-r-r-o-w a-r-r-o-w requested a review from yiyixuxu December 19, 2024 01:57
@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Dec 19, 2024

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

yiyixuxu
yiyixuxu approved these changes Dec 19, 2024
View reviewed changes
Copy link
Collaborator

@yiyixuxu yiyixuxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@a-r-r-o-w
Copy link
Contributor Author

a-r-r-o-w commented Dec 19, 2024

The test now also matches the VAE in model test with VAE in transformer test for consistency

@yiyixuxu
Copy link
Collaborator

yiyixuxu commented Dec 19, 2024

feel free to merge!

@a-r-r-o-w a-r-r-o-w merged commit f781b8c into main Dec 19, 2024
14 of 15 checks passed
@a-r-r-o-w a-r-r-o-w deleted the hunyuan-fixes branch December 19, 2024 04:58
Foundsheep pushed a commit to Foundsheep/diffusers that referenced this pull request Dec 23, 2024
@a-r-r-o-w @Foundsheep
Hunyuan VAE tiling fixes and transformer docs (huggingface#10295)
f8c1e4a 
* update

* udpate

* fix test
sayakpaul pushed a commit that referenced this pull request Dec 23, 2024
@a-r-r-o-w @sayakpaul
Hunyuan VAE tiling fixes and transformer docs (#10295)
6d7896e 
* update

* udpate

* fix test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiyixuxu yiyixuxu yiyixuxu approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@a-r-r-o-w @HuggingFaceDocBuilderDev @yiyixuxu