Repository navigation
Enable INT4 QuickReduce for MiniMax M3.1 MI355X vLLM / 为 MiniMax M3.1 MI355X vLLM 启用 INT4 QuickReduce - #4485
Open
Fangzhou-Ai wants to merge 1 commit into
Open
Fangzhou-Ai wants to merge 1 commit into
Fangzhou-Ai wants to merge 1 commit into
Conversation
为 MI355X vLLM MiniMax-M3.1-Flash AgentX 启用 INT4 QuickReduce，其他服务和工作负载设置保持不变。 Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
Fangzhou-Ai
added
the
full-sweep-fail-fast
Contributor
|
Thanks for the contribution!
中文感谢你的贡献！
|
Fangzhou-Ai
added
full-sweep-enabled
Contributor
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40165146154 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40165146154 |
Fangzhou-Ai
marked this pull request as ready for review
October 25, 2026 05:21
Contributor
|
Claude finished @Fangzhou-Ai's task in 25s —— View job Review
No blocking issues found. The recipe change adds one env var, There is one 🟡 warning, posted inline: the changelog INT4 reduction loses precision, so check the GSM8K eval from the sweep against the baseline before merging. |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - minimaxm31flash-fp4-mi355x-vllm-agentic-dspark | ||
| description: | ||
| - "Enable VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 for MiniMax-M3.1-Flash MI355X vLLM AgentX; keep the pinned image and all other serving and workload settings unchanged." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX |
Contributor
There was a problem hiding this comment.
🟡 WARNING: pr-link is still the pull/XXX placeholder.
Why it matters: This changelog entry is the record behind this config's result change. With a placeholder, the dashboard and changelog history link to a page that doesn't exist.
Fix:
Suggested change
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX | |
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4485 |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Enable
VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4for MiniMax-M3.1-Flash TP4 AgentX on MI355X. The pinned image, serving arguments, workloads and per-layer sparse-index recomputation remain unchanged. The only other file change is the required performance-changelog entry.The completed one-hour c32 INT4 run improves the observed result over the original pinned-image run:
The full INT4 GSM8K evaluation scored 1272/1319 (96.44%) strict match, versus 1266/1319 (95.98%) for the frequency-1 SGLang reference.
The c4 and c8 jobs failed before vLLM startup because prior-job GPU memory was not reclaimed within the 15-minute host-setup limit. The full sweep has not passed.
Validation details
VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4.AI model disclosure
PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entriesinferencex-e2e/perf-changelog.yamlentry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs):full-sweep-fail-fast(recommended),full-sweep-enabled, ornon-canary-full-sweep-enabled. Optional modifiersall-evals,evals-only, andagentx-fastrequire a primary label; the last two block reuse while applied.OWNER/MEMBER/COLLABORATOR) has commented/use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.中文
为 MI355X 上的 MiniMax-M3.1-Flash TP4 AgentX 启用
VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4。固定镜像、服务参数、工作负载和每层稀疏索引重算设置保持不变；另一个文件仅追加仓库要求的性能变更日志。已完成的一小时 INT4 c32 测试 相比原始固定镜像测试，输出吞吐量从 1209.52 提升到 1221.25 tok/s（+0.97%），P90 interactivity 从 97.33 提升到 101.54 tok/s/user（+4.32%）。共计 4307 条有效测量请求，丢弃 354 条预热记录，请求错误为零，测量时长为 3615.38 秒。
完整 INT4 GSM8K 评测 strict match 为 1272/1319（96.44%）；使用 frequency 1 的 SGLang 对比基准 为 1266/1319（95.98%）。
c4 和 c8 在 vLLM 启动前失败，原因是前一任务占用的 GPU 显存未在主机准备阶段的 15 分钟期限内释放；完整 sweep 尚未通过。归档启动日志确认服务参数一致，均为 K3、FP8 draft KV 和 synthetic acceptance 2.88，角色环境仅新增 INT4 设置。工作流任务名称中的 acceptance 标签不同，但实际 synthetic 服务配置均为 2.88。
本 PR 未包含本地探索中的 CP/K7/BF16-draft、attention、FlyDSL reducer 或 prefill tile 改动。
AI 模型使用说明：由 OpenAI GPT-6（通过 Codex）协助准备配方、变更日志、检查生成配置、分析 SA 测试及评测产物，并编写 PR 描述。运行环境未提供可核验的精确部署模型名称/版本；没有委派其他模型参与本 PR。