Skip to content

Nathan/qwen vl alt #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
dcw02 wants to merge 219 commits into
base: qwen_vl_eagle3
Choose a base branch
from
Open

Nathan/qwen vl alt #13

dcw02 wants to merge 219 commits into from

Conversation

@dcw02
Copy link

@dcw02 dcw02 commented Sep 2, 2025

No description provided.

CatherineSue and others added 30 commits August 19, 2025 16:44
@CatherineSue
[router] Implement gRPC SGLangSchedulerClient (sgl-project#9364)
7638f5e
@slin1237 @CatherineSue
[router] add tokenizer chat template support (sgl-project#9370)
5fbad30 
Co-authored-by: Chang Su <chang.s.su@oracle.com>
@key4ng
[router] Implement OpenAI Responses API specification (sgl-project#9367)
5ae5eca
@fzyzcjy
Fix mini lb timeout issue (sgl-project#9369)
fe43e88
@ispobock
Fix triton backend eagle illegal memory access (sgl-project#9344)
e0ce171
@key4ng
Fix gpt-oss response api streaming issue (sgl-project#9368)
f515449
@iforgetmyname @ping1jing2
@anon189Ty @Makcum888e @ssshinigami
[feature] Rework Ascend NPU graph support (sgl-project#9350)
3680d6f 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
Co-authored-by: yezhifeng (D) <y00897525@china.huawei.com>
Co-authored-by: anon189Ty <Stari_Falcon@outlook.com>
Co-authored-by: Maksim <makcum888e@mail.ru>
Co-authored-by: ssshinigami <44640852+ssshinigami@users.noreply.github.com>
@merrymercy
[minor] Sync style changes (sgl-project#9376)
f20b6a3
@wisclmy0611
[readme] Add SGLang x AMD SF meetup information (sgl-project#9380)
f1b0eda
@ShangmingCai
[CI] Fix disaggregation failure tolerance CI (sgl-project#9378)
d8ed60f 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
@merrymercy
[Docs] Update contribution guide (sgl-project#9383)
1ec9769
@iforgetmyname
Revert "[feature] Rework Ascend NPU graph support" (sgl-project#9385)
de2dd73
@oraluben
Reduce overhead for fa by not calling heavy CUDA property check (sgl-…
c9bf387 
…project#7375)
@fzyzcjy
Add PDL support for quant kernel and rope kernel (sgl-project#9106)
42c8704
@hnyls2002 @gemini-code-assist
Fix the --allow-auto-truncate argument in tokenizer manager. (sgl-p…
08ebdf7 
…roject#9391)

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@BBuf
Refactor allreduce add rmsnorm pattern (sgl-project#9278)
f96413c
@trevor-m
[2/2] Fuse routed scaling factor into select_experts (sgl-project#8690)
a91e90d
@thecodingwizard
Fix FlashInfer GPU <-> CPU sync (sgl-project#9409)
24eaebe
@lifuhuang
Support pinning adapter via server args. (sgl-project#9249)
b0980af
@lifuhuang @JustinTong0323
Fix incorrect logic in chat template handling. (sgl-project#9336)
d4bce29 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
@nvcastet
Support DP attention with GPT-OSS (sgl-project#9359)
c10b8e6
@jiapingW
Fixed the issue where eagle3 TPOT was not as good as without eagle3. (s…
e99729c 
…gl-project#9404)
@JustinTong0323
fix: InternS1 don't recognize image, updates image token for InternVL…
84719b5 
… processor (sgl-project#9381)

Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
@mickqian
misc: parse bench_serving result as markdown table (sgl-project#9377)
ef3004d
@nathanrchn
Add support for Qwen3-seq-cls (sgl-project#9357)
8f5b991
@strgrb
Support trtllm_allreduce_fusion in flashinfer for cuda<12.8 (sgl-proj…
88fbc31 
…ect#9339)

Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
@key4ng @zhyncs
[router] Add IGW (Inference Gateway) Feature Flag (sgl-project#9371)
3828db4 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
@CatherineSue
[router] add tokenizer integration test with real mini tokenizer (sgl…
e652310 
…-project#9413)
@CatherineSue
[router] add glm and step3 reasoning parser (sgl-project#9415)
5cfbb4c
@Qiaolin-Yu
Fix max_seq_len_k in trtllm_mha attention backend (sgl-project#9416)
af1973b
slin1237 and others added 30 commits August 28, 2025 12:07
@slin1237
[router] grpc router bootstraps (sgl-project#9759)
5343058
@hubertlu-tw
[AMD] Support Hierarchical Caching on AMD GPUs (sgl-project#8236)
711390a
@zixuanzhang226
feat: add tuned fused moe config for GLM-4.5-Air-FP8 tp = 4 on B200 (s…
dc20c22 
…gl-project#9770)
@chenxu140 @iforgetmyname
[Feature] Support NPUGraph for DeepSeek on Ascend NPU (sgl-project#9355)
74dd424 
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
@DellCurry
feat(draft_model): support draft_model for RemoteModelLoader (sgl-pro…
a38c149 
…ject#6407)

Signed-off-by: wangyu <wangyu.steph@bytedance.com>
@DellCurry
fix: fix MLA for ShardedModelLoader/RemoteModelLoader (sgl-project#6287)
9f81d74 
Signed-off-by: wangyu <wangyu.steph@bytedance.com>
@mingfeima
Optimize prefill performance on cpu backend (sgl-project#8750)
5ad296b
@xiezhq-hermann
[HiCache] change the default policy to write through (sgl-project#9772)
001f519
@hzh0425
bugfix(hicache): Move exists check before key suffixing (sgl-project#…
38cd5fb 
…9749)
@hlu1
Skip some tests on Blackwell (sgl-project#9777)
a7d825f 
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
@hnyls2002 @gemini-code-assist
Raise error when topk>1 and page>1 for paged attention backends. (s…
a23c302 
…gl-project#9784)

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@sogalin
ROCm 7.0 update (sgl-project#9757)
4b7034d
@JustinTong0323
Merge branch 'main' into qwen_vl_eagle3
15b78d3
@timmy-feng @thecodingwizard
Support overlap scheduling for speculative decoding
ed11d66 
Co-authored-by: Nathan Wang <nathan.r.wang@gmail.com>
@thecodingwizard
remove unused free_cache_loc_cpu from GenerationResult
bb33ce3
@thecodingwizard
Confirm that output_ids check is needed
98be63c
@thecodingwizard
Avoid duplicating eagle_utils.py
e9d062e
@thecodingwizard
Separate bench_speculative.py changes
4cdbfa7
@thecodingwizard
Fix seq_lens sync
63d7aec
@thecodingwizard
Clean up
c825d67
@thecodingwizard
fix typo in import path
55ac8bc
@thecodingwizard
fix return logprob logic
9d7c25e
@timmy-feng @thecodingwizard
remove mrope position sync
124a4cc
@timmy-feng @thecodingwizard
remove get_next_input_positions
df3d079
@thecodingwizard
run formatter
69dabf1
@thecodingwizard
correctly override extend_seq_lens for draft extend after decode
452ce2f
@thecodingwizard
fix compute spec mrope positions and overlap
ef646a5
@thecodingwizard
Add debug memory pool to TokenToKVPoolAllocator + misc debugging
8678b5a
@thecodingwizard
Fix freeing incorrect out cache loc bug
07657f9
@thecodingwizard
remove debugging comments
00f0dd9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.