[Bugfix] Fix moe weight losing all extra attrs after process_weights_after_loading.#16854
Conversation
Signed-off-by: charlifu <charlifu@amd.com>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
charlifu
changed the title
process_weights_after_loading.process_weights_after_loading.
Signed-off-by: charlifu <charlifu@amd.com>
charlifu
requested review from
mgoin,
robertgshaw2-redhat and
tlrmchlsmth
as code owners
|
add a small fix for not calling fp8 skinny gemm incorrectly. |
mgoin
added
bug
…_after_loading`. (vllm-project#16854) Signed-off-by: charlifu <charlifu@amd.com>
…_after_loading`. (vllm-project#16854) Signed-off-by: charlifu <charlifu@amd.com>
…_after_loading`. (vllm-project#16854) Signed-off-by: charlifu <charlifu@amd.com> Signed-off-by: Mu Huai <tianbowen.tbw@antgroup.com>
…_after_loading`. (vllm-project#16854) Signed-off-by: charlifu <charlifu@amd.com> Signed-off-by: Yuqi Zhang <yuqizhang@google.com>
2 participants
For #16842
Only set the
Parameter.datafield to retain the all extra attrs of weight.