Skip to content

[Feature Request] Support Using litellm.Router Instance #110

@Ocyss

Description

@Ocyss

Is your feature request related to a problem? Please describe.

ADK currently supports using LiteLLM as a model provider. I noticed that while there is a llm_client parameter available, it does not seem to support passing a custom client instance. Moreover, the LiteLLM SDK features a Router that provides powerful load balancing capabilities, which could be particularly useful for managing multiple model deployments.

I prefer programmatic model management over environment variables, especially when working with multiple models that have different base URLs and API keys.

Describe the solution you'd like

I propose either:

  1. Allowing ADK to globally accept a configured litellm.Router instance, or
  2. Enabling litellm.Router to be passed directly via the llm_client parameter

Describe alternatives you've considered

  1. LiteLLM Proxy - adds unnecessary external dependency

Metadata

Metadata

Labels

core[Component] This issue is related to the core interface and implementationmodels[Component] Issues related to model supportneeds review[Status] The PR/issue is awaiting review from the maintainer

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions