This example runs Kimi-K2-Instruct at native precision with:

• 4 nodes with 8x H100 GPUs each (32 H100s total)
• Tensor parallel size: 16, Pipeline parallel size: 2
• RDMA networking for high-performance inter-node communication
• Ray for distributed orchestration
• vLLM nightly build for Kimi-K2-Instruct pipeline parallelism support

Checklist

• Example is documented with comments throughout, in a Literate Programming style.
• Example does not require third-party dependencies to be installed locally
• Example follows the style guide
• Example pins its dependencies
  • Example pins container images to a stable tag, not a dynamic tag like latest
  • Example specifies a python_version for the base image, if it is used
  • Example pins all dependencies to at least minor version, ~=x.y.z or ==x.y
  • Example dependencies with version < 1 are pinned to patch version, ==0.y.z

(Modal's internal guide page for this repo is Multi-node examples guidance.)