[Advanced LoRA v1.5] fix: gradient unscaling problem#7018
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
@linoytsaban a gentle ping here. |
|
@linoytsaban sorry about the repeated pings here. |
|
sorry missed this! |
|
Would you be able to test it as well? I don;t have the training commands for v1.5 to test. |
|
Apologies for the delay, tested with ⬇️ and it works good! we can ship :) Thanks a lot! |
|
@linoytsaban thanks but we need to check if it can successfully resumed as well. Refer to #6514 (comment) for more details. |
|
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
|
Friendly ping here @linoytsaban |
|
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
|
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
|
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
fix: gradient unscaling problem Co-authored-by: Linoy Tsaban <57615435+linoytsaban@users.noreply.github.com>
What does this PR do?
Part of #6552.
@linoytsaban I couldn't find instructions to run this script in the README. Would appreciate some help in verifying it.