Search results

[Bug]: AutoModelForCausalLM.

bugSomething isn't workingSomething isn't working

Status: Open.

#8942 In NVIDIA/TensorRT-LLM;

· lyc1995452 opened on Nov 5, 2025

Support for Qwen2-VL 2B/4B? or Qwen2.5-VL

new modelRequest to add a new modelRequest to add a new model

questionFurther information is requestedFurther information is requested

Status: Open.

#8927 In NVIDIA/TensorRT-LLM;

· lyc1995452 opened on Nov 5, 2025

[Feature]: AutoDeploy: torch 2.9 upgrade

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

Status: Open.

Task

#8924 In NVIDIA/TensorRT-LLM;

· lucaslie opened on Nov 4, 2025

[Feature]: Add Symetric mem all reduce strategy

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Scale-out<NV>Multi-GPU and distributed inference scaling issues, tensor/pipeline/data parallelism<NV>Multi-GPU and distributed inference scaling issues, tensor/pipeline/data parallelism

Status: Open.

#8921 In NVIDIA/TensorRT-LLM;

· MrGeva opened on Nov 4, 2025

What is the default KV cache dtype with <code>trtllm-serve</code>, and can I lower it for better VRAM efficiency?

KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inference

questionFurther information is requestedFurther information is requested

Status: Open.

#8843 In NVIDIA/TensorRT-LLM;

· NEWbie0709 opened on Oct 31, 2025

it run good on rtx 5060 ti 16 gb

questionFurther information is requestedFurther information is requested

waiting for feedback

Status: Open.

#8821 In NVIDIA/TensorRT-LLM;

· kmnnmk212-source opened on Oct 31, 2025

wheel for tensorrt-llm

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

InstallationSetting up and building TRTLLM: compilation, pip install, dependencies, env config, CMake.Setting up and building TRTLLM: compilation, pip install, dependencies, env config, CMake.

waiting for feedback

Status: Open.

#8819 In NVIDIA/TensorRT-LLM;

· kmnnmk212-source opened on Oct 30, 2025

[Feature]: AutoDeploy: investigate fp8 gemms that do not have pad to 16 requirement

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.

Status: Open.

Task

#8814 In NVIDIA/TensorRT-LLM;

· lucaslie opened on Oct 30, 2025

[Bug]: Output includes tokens from other prompts when host kv cache enabled

bugSomething isn't workingSomething isn't working

KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inference

Status: Open.

#8813 In NVIDIA/TensorRT-LLM;

· pathorn opened on Oct 30, 2025

[Feature]: AutoDeploy: padding for fp8 linear kernel when size % 16 != 0

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.

Status: Open.

Task

#8811 In NVIDIA/TensorRT-LLM;

· lucaslie opened on Oct 30, 2025

[Bug]: EXAONE-4.0-32B with VSWA causes OOM on A100-40G x 4

bugSomething isn't workingSomething isn't working

Investigating

KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inference

MemoryMemory utilization in TRTLLM: leak/OOM handling, footprint optimization, memory profiling.Memory utilization in TRTLLM: leak/OOM handling, footprint optimization, memory profiling.

Pytorch<NV>Pytorch backend related issues<NV>Pytorch backend related issues

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

#8802 In NVIDIA/TensorRT-LLM;

· lkm2835 opened on Oct 30, 2025

[Bug]: Can both <code>enable_inductor</code> and <code>enable_piecewise_cuda_graph</code> be set to <code>true</code>?

bugSomething isn't workingSomething isn't working

Pytorch<NV>Pytorch backend related issues<NV>Pytorch backend related issues

Status: Open.

#8784 In NVIDIA/TensorRT-LLM;

· morganliang-llm opened on Oct 30, 2025