-
-
Notifications
You must be signed in to change notification settings - Fork 12.1k
[ROCm][CI/Build] Sync ROCm dockerfiles with the ROCm fork #24279
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request syncs the ROCm Dockerfiles with the ROCm fork. The changes include updating the base image to ROCm 6.4.1, updating various dependency versions, and refining the build process for components like Triton. These changes generally improve the build's stability and robustness. I have identified one issue where a new Docker build stage is defined but not utilized in the final image construction, which should be addressed to improve efficiency and clarity.
…ct#24279) Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
…ct#24279) Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
…ct#24279) Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
…ct#24279) Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
…ct#24279) Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
| RUN if [ -d triton/python/triton_kernels ]; then pip install build && cd triton/python/triton_kernels \ | ||
| && python3 -m build --wheel && cp dist/*.whl /app/install; fi |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't expect compatibility with pytorch-triton-rocm==3.5.0?
Seems like triton_kernels / https://github.com/ROCm/triton/tree/57c693b627fe058878ade4163a0a8df95d9fefa1/python/triton_kernels is not shipped with it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, does not work with upstream triton from https://download.pytorch.org/whl/rocm6.4:
(base) root@felix-marty-job-torch-vllm-1-rmfqs:~# vllm serve /models/openai_gpt-oss-20b --tensor-parallel-size 1 --enforce-e
ager
Traceback (most recent call last):
File "/root/miniforge3/bin/vllm", line 33, in <module>
sys.exit(load_entry_point('vllm', 'console_scripts', 'vllm')())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/bin/vllm", line 25, in importlib_load_entry_point
return next(matches).load()
^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/lib/python3.12/importlib/metadata/__init__.py", line 205, in load
module = import_module(match.group('module'))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 999, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/shared_volume/repos/vllm/vllm/entrypoints/cli/__init__.py", line 4, in <module>
from vllm.entrypoints.cli.benchmark.serve import BenchmarkServingSubcommand
File "/shared_volume/repos/vllm/vllm/entrypoints/cli/benchmark/serve.py", line 5, in <module>
from vllm.benchmarks.serve import add_cli_args, main
File "/shared_volume/repos/vllm/vllm/benchmarks/serve.py", line 41, in <module>
from vllm.benchmarks.datasets import SampleRequest, add_dataset_parser, get_samples
File "/shared_volume/repos/vllm/vllm/benchmarks/datasets.py", line 39, in <module>
from vllm.lora.utils import get_adapter_absolute_path
File "/shared_volume/repos/vllm/vllm/lora/utils.py", line 22, in <module>
from vllm.lora.layers import (
File "/shared_volume/repos/vllm/vllm/lora/layers/__init__.py", line 14, in <module>
from vllm.lora.layers.fused_moe import FusedMoEWithLoRA
File "/shared_volume/repos/vllm/vllm/lora/layers/fused_moe.py", line 17, in <module>
from vllm.model_executor.layers.fused_moe import FusedMoE
File "/shared_volume/repos/vllm/vllm/model_executor/layers/fused_moe/__init__.py", line 7, in <module>
from vllm.model_executor.layers.fused_moe.config import FusedMoEConfig
File "/shared_volume/repos/vllm/vllm/model_executor/layers/fused_moe/config.py", line 26, in <module>
from triton_kernels.matmul_ogs import PrecisionConfig
File "/shared_volume/repos/triton/python/triton_kernels/triton_kernels/matmul_ogs.py", line 11, in <module>
from .matmul_ogs_details._matmul_ogs import _compute_writeback_idx
File "/shared_volume/repos/triton/python/triton_kernels/triton_kernels/matmul_ogs_details/_matmul_ogs.py", line 7, in <module>
from triton_kernels.numerics_details.flexpoint import float_to_flex, load_scale
File "/shared_volume/repos/triton/python/triton_kernels/triton_kernels/numerics_details/flexpoint.py", line 55, in <module>
@tl.constexpr_function
^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'triton.language' has no attribute 'constexpr_function'
(base) root@felix-marty-job-torch-vllm-1-rmfqs:~# pip list | grep triton
conch-triton-kernels 1.2.1
pytorch-triton-rocm 3.5.0
Bringing dockerfiles in sync with the ROCm fork, to match what is used to build rocm/vllm-dev:base; rocm/vllm-dev:nightly and rocm/vllm images