Skip to content

Conversation

@vllmellm
Copy link
Contributor

@vllmellm vllmellm commented Jul 15, 2025

Essential Elements of an Effective PR Description Checklist

  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

As vLLM expands ROCm/AMD support (e.g., for MI300X, MI350X, CDNA4, and Radeon series), issues mentioning these often need quick triage. This workflow automatically adds the "rocm" label (existing in repo) to matching issues, improving community support. It complements the PR labeler (file-path based) by focusing on issues via keywords.

This PR introduces a GitHub Action that automatically labels issues based on configurable keywords, substrings, and regex patterns found in issue titles and bodies. The action helps maintain consistent labeling across the repository and reduces manual triage effort.

Features

Multi-Pattern Matching Support

  • Keywords: Whole word matching with word boundaries (e.g., "rocm" matches "ROCm" but not "rocmtest")
  • Substrings: Partial matching anywhere in text (e.g., "hip-" matches "hip-python", "hip-dev")
  • Regex Patterns: Complex pattern matching (e.g., \bmi\d{3}[a-z]*\b for AMD GPU names like "mi300x")

Flexible Search Locations

Each pattern can be configured to search in:

  • title: Only in issue title
  • body: Only in issue body
  • both: In both title and body (default)

Detailed Logging

  • Shows exactly which terms matched and where
  • Provides line numbers and context for matches
  • Displays match counts and reasoning for label decisions

How It Works

Trigger Events

The action runs on:

  • issues.opened - New issues
  • issues.edited - Modified issues
  • issues.reopened - Reopened issues

Concurrency Control

Uses issue-specific concurrency groups to prevent race conditions when multiple events occur simultaneously on the same issue.

Processing Flow

  1. Parse Configuration: Reads label configuration with keywords, substrings, and regex patterns
  2. Text Analysis: Searches issue title and body based on each pattern's searchIn setting
  3. Pattern Matching: Applies appropriate matching strategy (keyword/substring/regex)
  4. Label Application: Adds labels only if they don't already exist
  5. Detailed Logging: Reports all matches with context and line numbers

📝 Configuration Structure

labelConfig:
  label-name:
    keywords:           # Whole word matching
      - term: "exact word"
        searchIn: "both"  # "title", "body", or "both"
    
    substrings:         # Partial matching
      - term: "partial"
        searchIn: "title"
    
    regexPatterns:      # Complex patterns
      - pattern: "\\bmi\\d{3}[a-z]*\\b"
        description: "AMD GPU names"
        flags: "gi"
        searchIn: "both"

Extending for Your Use Case

Adding New Labels

  1. Add a new entry to labelConfig:
const labelConfig = {
  "your-label": {
    keywords: [
      { term: "your keyword", searchIn: "both" }
    ],
    substrings: [
      { term: "partial-match", searchIn: "title" }
    ]
  }
};

Example Output

When processing an issue, the action provides detailed logs:

Term: "rocm" (substring search) found 2 time(s) in body:
  1. Line 15 in body: "ROCm" [substring]
     Context: Installing ROCm 5.7 on Ubuntu 22.04
  2. Line 23 in body: "rocm-dev" [substring]  
     Context: sudo apt install rocm-dev rocm-libs

Final decision: ADD LABEL
Reason: Found 2 total matches (0 in title, 2 in body)

Current Configuration

The action is currently configured to auto-label issues with rocm based on ROCm-related terms, AMD GPU references, and HIP-related keywords. This serves as a comprehensive example of all matching types in action.


This action provides a robust foundation for automated issue labeling that can be easily extended and customized for any repository's specific needs.

Test Plan

This GH action workflow is now active on https://github.com/vllm-project/vllm/issues , create a github issue and check the github actions output.

Test Result

Created 30 non-ROCm related issue by extracting actual issues from vllm-project/vllm from past 7 days. All didn't trigger ROCm label


  1. #23655: [Usage]: RuntimeError: Unexpected error from cudaGetDeviceCo... [usage]
  2. #23645: [Bug]: Qwen2.5-VL GPTQ W4A16 quantized with llm-compressor b... [bug]
  3. #23641: [Feature]: Frequency and Cost Aware Eviction Policy for Pref... [feature request]
  4. #23638: [Feature]: InternVL3_5 Support... [feature request]
  5. #23632: [Feature]: AttributeError: Model GptOssForCausalLM does not ... [bug]
  6. #23631: [Bug]: Qwen3-30B-A3B-GPTQ-Int4 infer failure!... [bug]
  7. #23627: [Usage]: How to start deepseek with pd disaggregation throug... [usage]
  8. #23626: [Feature]: Support for Multi-Vision Tower Models (e.g., Vall... [feature request, multi-modality]
  9. #23621: [Feature]: Flashinfer per-request... [feature request]
  10. #23615: [Bug]: vllm运行Qwen3 reranker准确性问题... [bug]
  11. #23612: [Feature]: Allow oot custom compiler extension via CompilerI... [feature request]
  12. #23611: [Usage]: Which dataset do you recommend using for the ngram ... [usage]
  13. #23610: [Feature]: Add LoRA support for gpt-oss model... [feature request]
  14. #23609: [Usage]: Unable to see more than 20% improvement on b200 for... [usage]
  15. #23607: [Usage]: How to reason about a modified large language model... [usage]
  16. #23603: [Feature]: Log prompt for gpt-oss... [feature request]
  17. #23600: [Bug]: (standard_in) 1: syntax error {"detail":"Not Found"}(... [bug]
  18. #23594: [CI]: Host images used by multimodal tests locally... [good first issue, ci/build]
  19. #23593: [CI]: Declarative regression tests for API parameters... [ci/build]
  20. #23590: [CI]: Audit use of fixtures across tests to minimize server ... [ci/build]
  21. #23589: [CI]: Investigate use of CPU backend for subset of tests... [ci/build]
  22. #23588: [CI]: Reduce docker build time with caching... [ci/build]
  23. #23587: [Bug]: NIXL Crashes if P/D Protocol is off... [bug]
  24. #23582: [Bug]: vLLM server timeout due to multiprocessing communicat... [bug]
  25. #23581: [Installation]:... [installation]
  26. #23580: [Installation]:... [installation]
  27. #23579: [Installation]:... [installation]
  28. #23578: [Installation]:... [installation]
  29. #23577: [Bug]: default_weight_loader receives unexpected `weight_nam... [bug]
  30. #23575: [Feature]: Add usage in TranscriptionResponse... [feature request]

Extracted all rocm related issues from past 7 days to test True Positives


Found 4 issues to process:
  1. #22590: [Bug]: ROCm build falls back to default arch despite ARG_PYT... [bug, rocm]
  2. #22458: [Feature]: Please add support for Amd Instinct MI50... [feature request, rocm]
  3. #22245: [Bug]: VLLM_ROCM_USE_AITER=1 hit device_gemm with the specif... [bug, rocm]
  4. #14964: [Feature] [ROCm]: AITER Kernel Integration... [feature request, rocm]

Creating issues in vllmellm/vllm...
Creating issue: [Bug]: ROCm build falls back to default arch despite ARG_PYT...
  ✅ Created: #48 - https://github.com/vllmellm/vllm/issues/48
Creating issue: [Feature]: Please add support for Amd Instinct MI50...
  ✅ Created: #49 - https://github.com/vllmellm/vllm/issues/49
Creating issue: [Bug]: VLLM_ROCM_USE_AITER=1 hit device_gemm with the specif...
  ✅ Created: #50 - https://github.com/vllmellm/vllm/issues/50
Creating issue: [Feature] [ROCm]: AITER Kernel Integration...
  ✅ Created: #51 - https://github.com/vllmellm/vllm/issues/51

✅ Successfully created 4 issues!

🔍 Validating GitHub Action labeling...
  Waiting for GitHub Action to process issue #48...
    [10s] Current labels: ['rocm']

✅ Issue #48: [Bug]: ROCm build falls back to default arch despite ARG_PYT...
   Result: Correctly labeled
   Current labels: ['rocm']
   ROCm matches found: 32
   Detected ROCm content:
     • rocm (substring) in title: 3 matches
       (configured to search in: title)
       - Line 1: 'ROCm'
       - Line 1: 'ROCM'
       - Line 1: 'rocm'
     • rccl (keyword) in body: 2 matches
       (configured to search in: body)
       - Line 158: 'rccl'
       - Line 158: 'rccl'
     • hip- (substring) in body: 6 matches
       (configured to search in: both)
     • gfx (substring) in body: 11 matches
       (configured to search in: both)
     • torch_hip (substring) in body: 2 matches
       (configured to search in: body)
       - Line 65: 'TORCH_HIP'
       - Line 219: 'TORCH_HIP'
     • _hip (substring) in body: 2 matches
       (configured to search in: both)
       - Line 65: '_HIP'
       - Line 219: '_HIP'
     • hip_ (substring) in body: 6 matches
       (configured to search in: both)
   URL: https://github.com/vllmellm/vllm/issues/48
  Waiting for GitHub Action to process issue #49...
    [10s] Current labels: ['rocm']

✅ Issue #49: [Feature]: Please add support for Amd Instinct MI50...
   Result: Correctly labeled
   Current labels: ['rocm']
   ROCm matches found: 1
   Detected ROCm content:
     • amd (substring) in title: 1 matches
       (configured to search in: title)
       - Line 1: 'Amd'
   URL: https://github.com/vllmellm/vllm/issues/49
  Waiting for GitHub Action to process issue #50...
    [10s] Current labels: ['rocm']

✅ Issue #50: [Bug]: VLLM_ROCM_USE_AITER=1 hit device_gemm with the specif...
   Result: Correctly labeled
   Current labels: ['rocm']
   ROCm matches found: 13
   Detected ROCm content:
     • VLLM_ROCM_ (substring) in title: 1 matches
       (configured to search in: both)
       - Line 1: 'VLLM_ROCM_'
     • rocm (substring) in title: 1 matches
       (configured to search in: title)
       - Line 1: 'ROCM'
     • ROCm System Management Interface (keyword) in body: 1 matches
       (configured to search in: body)
       - Line 106: 'ROCm System Management Interface'
     • VLLM_ROCM_ (substring) in body: 1 matches
       (configured to search in: both)
       - Line 185: 'VLLM_ROCM_'
     • gfx (substring) in body: 7 matches
       (configured to search in: both)
     • _hip (substring) in body: 1 matches
       (configured to search in: both)
       - Line 217: '_hip'
     • AMD GPU names (mi + 3 digits + optional letters) (regex) in body: 1 matches
       (configured to search in: both)
       - Line 34: 'MI300X'
         Description: AMD GPU names (mi + 3 digits + optional letters)
   URL: https://github.com/vllmellm/vllm/issues/50
  Waiting for GitHub Action to process issue #51...
    [10s] Current labels: ['rocm']

✅ Issue #51: [Feature] [ROCm]: AITER Kernel Integration...
   Result: Correctly labeled
   Current labels: ['rocm']
   ROCm matches found: 1
   Detected ROCm content:
     • rocm (substring) in title: 1 matches
       (configured to search in: title)
       - Line 1: 'ROCm'
   URL: https://github.com/vllmellm/vllm/issues/51

============================================================
🔍 ROCm LABELING VALIDATION REPORT
============================================================
Total Issues Analyzed: 4
Correct Predictions: 4
Accuracy: 100.0%

Detailed Results:
  ✅ Correct Positive (should have + has label): 4
  ✅ Correct Negative (should not have + no label): 0
  ❌ False Negative (should have + missing label): 0
  ⚠️  False Positive (should not have + has label): 0

Visit your repository: https://github.com/vllmellm/vllm/issues

(Optional) Documentation Update

vllmellm added 2 commits July 15, 2025 11:00
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
@gemini-code-assist
Copy link
Contributor

Note

Gemini is unable to generate a summary for this pull request due to the file types involved not being currently supported.

@mergify mergify bot added ci/build rocm Related to AMD ROCm labels Jul 15, 2025
@github-actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

vllmellm added 2 commits July 15, 2025 13:40
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Comment on lines 87 to 92
if (escapedExclusions.length > 0) {
const exclusionPattern = escapedExclusions.join('|');
return new RegExp(`\\b(?!.*(?:${exclusionPattern}))(${keywordPattern})\\b`, 'i');
}

return new RegExp(`\\b(${keywordPattern})\\b`, 'i');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (escapedExclusions.length > 0) {
const exclusionPattern = escapedExclusions.join('|');
return new RegExp(`\\b(?!.*(?:${exclusionPattern}))(${keywordPattern})\\b`, 'i');
}
return new RegExp(`\\b(${keywordPattern})\\b`, 'i');
if (escapedExclusions.length === 0) {
return new RegExp(`\\b(${keywordPattern})\\b`, 'i');
}
const exclusionPattern = escapedExclusions.join('|');
return new RegExp(`\\b(?!.*(?:${exclusionPattern}))(${keywordPattern})\\b`, 'i');

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handle the simple case first

Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
vllmellm and others added 5 commits July 15, 2025 14:04
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
[CI] [ROCm] update rocm issue auto label
@hongxiayang hongxiayang self-requested a review July 30, 2025 15:41
Comment on lines 43 to 61
"amd radeon rx 9070",
"amd radeon rx 9070 xt",
"amd radeon rx 9070 gre",
"amd radeon ai pro r9700",
"amd radeon rx 9060 xt",
"amd radeon rx 7900 xtx",
"amd radeon rx 7900 xt",
"amd radeon rx 7900 gre",
"amd radeon rx 7800 xt",

// GPU Models - Pro Series
"amd radeon pro w7900",
"amd radeon pro w7900 dual slot",
"amd radeon pro w7800",
"amd radeon pro w7800 48gb",
"amd radeon pro w7700",
"amd radeon pro v710",
"amd radeon pro v620",
"amd radeon pro w6800",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you simplify these instances? (1) use "radeon" only, as "amd" is already listed. (2) Also, is there a need to list all of these instances, since you are using regression expression?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These has been simplify. Given that the product name like w7900 w7800 etc are mainly unique to AMD GPU product, so the product name are still retained.

Comment on lines 64 to 71
"amd instinct mi325x",
"amd instinct mi300x",
"amd instinct mi300a",
"amd instinct mi250x",
"amd instinct mi250",
"amd instinct mi210",
"amd instinct mi350x",
"amd instinct mi355x",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, simplify instead of list all the mixxx instances

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Theses are shorten instead of using regex as in the reviewer would like to keep the feature easier to extend by not asking developer to write regex syntax when adding new keywords or sub-string.

Comment on lines 74 to 89
"cdna3",
"cdna2",
"cdna4",
"cdna",
"gfx942",
"gfx90a",
"gfx1100",
"gfx1101",
"gfx1201",
"gfx1200",
"gfx1030",
"gfx950",
"gfx908",
"rdna4",
"rdna3",
"rdna2",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here to simplify

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Added a new matching feature: substring match. So we have simplify these to just do a substring match
cdna gfx and rdna.

Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
@DarkLight1337 DarkLight1337 enabled auto-merge (squash) August 26, 2025 14:41
@DarkLight1337 DarkLight1337 disabled auto-merge August 26, 2025 14:41
@DarkLight1337 DarkLight1337 enabled auto-merge (squash) August 26, 2025 14:46
@DarkLight1337 DarkLight1337 disabled auto-merge August 26, 2025 14:46
@DarkLight1337 DarkLight1337 added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 26, 2025
@DarkLight1337 DarkLight1337 enabled auto-merge (squash) August 26, 2025 14:51
@DarkLight1337 DarkLight1337 merged commit 44ac25e into vllm-project:main Aug 26, 2025
21 of 23 checks passed
tc-mb pushed a commit to tc-mb/vllm that referenced this pull request Aug 27, 2025
…llm-project#20988)

Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Signed-off-by: tc-mb <caitianchi@modelbest.cn>
epwalsh pushed a commit to epwalsh/vllm that referenced this pull request Aug 28, 2025
…llm-project#20988)

Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
xiao-llm pushed a commit to xiao-llm/vllm that referenced this pull request Aug 28, 2025
…llm-project#20988)

Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Signed-off-by: Xiao Yu <xiao.yu@amd.com>
zhewenl pushed a commit to zhewenl/vllm that referenced this pull request Aug 28, 2025
…llm-project#20988)

Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
zhewenl pushed a commit to zhewenl/vllm that referenced this pull request Sep 3, 2025
…llm-project#20988)

Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
FeiDaLI pushed a commit to FeiDaLI/vllm that referenced this pull request Sep 25, 2025
…llm-project#20988)

Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/build ready ONLY add when PR is ready to merge/full CI is needed rocm Related to AMD ROCm

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants