Skip to content

Add LoRA support for DeepSeek models (V2, V3, R1-0528) #23971

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
vllm-bot merged 6 commits into from
Aug 30, 2025
Merged

Add LoRA support for DeepSeek models (V2, V3, R1-0528) #23971

vllm-bot merged 6 commits into from
Aug 30, 2025

Conversation

@sadeghja1070
Copy link
Contributor

@sadeghja1070 sadeghja1070 commented Aug 29, 2025 •
edited by github-actions bot
Loading

  • Add SupportsLoRA interface to DeepseekV2ForCausalLM
  • Enable LoRA fine-tuning for DeepSeek-V2, DeepSeek-V3, and DeepSeek-R1-0528
  • MoE detection already handled by existing is_moe_model() function
  • Maintains compatibility with existing FusedMoE warning system

This change follows the same pattern as Qwen MoE LoRA support (PR #20932).

🤖 Generated with Claude Code

Purpose

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@github-actions
Copy link

github-actions bot commented Aug 29, 2025

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

@mergify mergify bot added the deepseek Related to DeepSeek models label Aug 29, 2025
gemini-code-assist[bot]
gemini-code-assist bot reviewed Aug 29, 2025
View reviewed changes
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds LoRA support for DeepSeek models by inheriting from the SupportsLoRA interface. While this is a good first step, it's incomplete. For LoRA to work on token embeddings and the language model head, the embedding_modules and embedding_padding_modules attributes must be defined on the DeepseekV2ForCausalLM class. Without these, LoRA for embeddings will not be enabled. I've added a suggestion to include these missing attributes.

@sadeghja1070 sadeghja1070 force-pushed the feature/deepseek-lora-support branch from 8ab9751 to 9fbef38 Compare August 29, 2025 22:49
@sadeghja1070 @claude
Add LoRA support for DeepSeek models (V2, V3, R1-0528)
faf5309 
- Add SupportsLoRA interface to DeepseekV2ForCausalLM
- Enable LoRA fine-tuning for DeepSeek-V2, DeepSeek-V3, and DeepSeek-R1-0528
- Add embedding_modules and embedding_padding_modules for complete LoRA support
- MoE detection already handled by existing is_moe_model() function
- Maintains compatibility with existing FusedMoE warning system

This change follows the same pattern as Qwen MoE LoRA support (PR vllm-project#20932).
Addresses code review feedback by including proper embedding layer configuration.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: sadeghja1070 <sadegh.ja1070@gmail.com>
@sadeghja1070 sadeghja1070 force-pushed the feature/deepseek-lora-support branch from 9fbef38 to faf5309 Compare August 29, 2025 23:00
jeejeelee
jeejeelee approved these changes Aug 30, 2025
View reviewed changes
Copy link
Collaborator

@jeejeelee jeejeelee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you , we also need to update the model info in the doc

@github-project-automation github-project-automation bot moved this from Backlog to In progress in DeepSeek V3/R1 Aug 30, 2025
@sadeghja1070
Copy link
Contributor Author

sadeghja1070 commented Aug 30, 2025

Thank you , we also need to update the model info in the doc

Would I need to do this or would you take care of it?

@jeejeelee
Copy link
Collaborator

jeejeelee commented Aug 30, 2025

Thank you , we also need to update the model info in the doc

Would I need to do this or would you take care of it?

If it's convenient for you, please handle it, thank you

sadeghja1070 and others added 2 commits August 29, 2025 20:20
@sadeghja1070 @claude
Update documentation: Add LoRA support for DeepSeek models
d96260a 
Update supported_models.md to reflect LoRA support for:
- DeepseekForCausalLM (DeepSeek)
- DeepseekV2ForCausalLM (DeepSeek-V2)
- DeepseekV3ForCausalLM (DeepSeek-V3, includes R1 variants)

Also added DeepSeek-R1 as an example model for DeepSeek-V3 architecture.

This documentation update corresponds to the LoRA implementation
added in the previous commit.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: sadeghja1070 <sadegh.ja1070@gmail.com>
@sadeghja1070 @claude
Remove embedding_modules from DeepSeek LoRA support
92d5316 
Per reviewer feedback, remove embedding_modules and embedding_padding_modules
as they are rarely used for LoRA fine-tuning. This simplifies the implementation
while maintaining the core LoRA functionality for DeepSeek models.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: sadeghja1070 <sadegh.ja1070@gmail.com>
@sadeghja1070 sadeghja1070 requested a review from hmellor as a code owner August 30, 2025 03:21
@mergify mergify bot added the documentation Improvements or additions to documentation label Aug 30, 2025
@sadeghja1070
Copy link
Contributor Author

sadeghja1070 commented Aug 30, 2025

Thank you , we also need to update the model info in the doc

Would I need to do this or would you take care of it?

If it's convenient for you, please handle it, thank you

both requests are done now

@jeejeelee
Fix mapping
4f74c46 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
jeejeelee
jeejeelee reviewed Aug 30, 2025
View reviewed changes
vllm/model_executor/models/deepseek.py
"qkv_proj": ["q_proj", "k_proj", "v_proj"],
"gate_up_proj": ["gate_proj", "up_proj"],
}

Copy link
Collaborator

@jeejeelee jeejeelee Aug 30, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, for decoder-only models to support LoRA, you need to:

  • Inherit from SupportLoRA
  • Impl packed_modules_mapping

You can take a look at those models that support LoRA.

@jeejeelee jeejeelee added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 30, 2025
@DarkLight1337 DarkLight1337 enabled auto-merge (squash) August 30, 2025 06:29
@vllm-bot vllm-bot merged commit 379ea28 into vllm-project:main Aug 30, 2025
37 of 42 checks passed
@github-project-automation github-project-automation bot moved this from In progress to Done in DeepSeek V3/R1 Aug 30, 2025
eicherseiji pushed a commit to eicherseiji/vllm that referenced this pull request Sep 9, 2025
@sadeghja1070 @claude
@jeejeelee @DarkLight1337 @eicherseiji
Add LoRA support for DeepSeek models (V2, V3, R1-0528) (vllm-project#…
17679fa 
…23971)

Signed-off-by: sadeghja1070 <sadegh.ja1070@gmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
FeiDaLI pushed a commit to FeiDaLI/vllm that referenced this pull request Sep 25, 2025
@sadeghja1070 @claude
@jeejeelee @DarkLight1337 @FeiDaLI
Add LoRA support for DeepSeek models (V2, V3, R1-0528) (vllm-project#…
3248593 
…23971)

Signed-off-by: sadeghja1070 <sadegh.ja1070@gmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jeejeelee jeejeelee jeejeelee approved these changes

@hmellor hmellor Awaiting requested review from hmellor

+1 more reviewer

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

deepseek Related to DeepSeek models documentation Improvements or additions to documentation ready ONLY add when PR is ready to merge/full CI is needed

Projects

DeepSeek V3/R1
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@sadeghja1070 @jeejeelee @vllm-bot @DarkLight1337