fix min-snr implementation#8466
Conversation
sayakpaul
left a comment
sayakpaul
left a comment
There was a problem hiding this comment.
Thanks for the fix!
Do you want to apply it to the other scripts too?
|
@ethansmith2000 a gentle ping :) |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
|
@ethansmith2000 |
|
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
|
Will merge after the CI is green. |
* fix min-snr implementation https://github.com/kohya-ss/sd-scripts/blob/main/library/custom_train_functions.py#L66 * Update train_dreambooth.py fix variable name mse_loss_weights * fix divisor * make style --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: YiYi Xu <yixu310@gmail.com>
4 participants
What does this PR do?
Fixes the implementation of min-snr training for v-prediction models
Based on implementation seen here.
https://github.com/kohya-ss/sd-scripts/blob/main/library/custom_train_functions.py#L66
These two graphs show the weight scheme based on timestep from the kohya implementation.
the current implementation in diffusers takes the eps weight and increases everything by + 1, it appears that the +1 should be relocated to the denominator