[Bugfix] Fix incorrect import of CacheConfig #24631
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
DarkLight1337
added
the
ready
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request correctly fixes an incorrect import of CacheConfig in vllm/attention/layers/cross_attention.py. The change replaces the import from the transformers library with the correct one from vllm.config. The implementation is clean and directly addresses the bug described. I have no further comments.
heheda12345
approved these changes
Sep 11, 2025
Collaborator
|
Sorry. Same problem happened again. #23459 |
Member
Author
|
Once we upgrade vLLM's transformers version, mypy should be able to detect the missing import |
hmellor
reviewed
Sep 11, 2025
| from vllm.attention.layer import Attention | ||
| from vllm.attention.selector import get_attn_backend | ||
| from vllm.config import VllmConfig | ||
| from vllm.config import CacheConfig, VllmConfig |
Member
There was a problem hiding this comment.
Small nit but canonical imports are preferred
Suggested change
| from vllm.config import CacheConfig, VllmConfig | |
| from vllm.config import VllmConfig | |
| from vllm.config.cache import CacheConfig |
Member
Author
There was a problem hiding this comment.
Let's address this separately
Member
|
sorry, and thank you! |
skyloevil
pushed a commit
to skyloevil/vllm
that referenced
this pull request
Sep 13, 2025
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
dsxsteven
pushed a commit
to dsxsteven/vllm_splitPR
that referenced
this pull request
Sep 15, 2025
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
FeiDaLI
pushed a commit
to FeiDaLI/vllm
that referenced
this pull request
Sep 25, 2025
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
xuebwang-amd
pushed a commit
to xuebwang-amd/vllm
that referenced
this pull request
Oct 10, 2025
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
xuebwang-amd
pushed a commit
to xuebwang-amd/vllm
that referenced
this pull request
Oct 24, 2025
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
#21088 incorrectly imports the deprecated
CacheConfigfrom transformers library instead of vLLM, which breaks Whisper model (and any models that use Whisper vLLM impl as encoder) when using latest transformers version.FIX https://buildkite.com/vllm/ci/builds/30279/steps/canvas?sid=019936ef-4094-4ebf-85c3-047098b7a6ee
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.