-
Notifications
You must be signed in to change notification settings - Fork 2.9k
Open
Labels
core[Component] This issue is related to the core interface and implementation[Component] This issue is related to the core interface and implementationmodels[Component] Issues related to model support[Component] Issues related to model supportneeds review[Status] The PR/issue is awaiting review from the maintainer[Status] The PR/issue is awaiting review from the maintainer
Milestone
Description
Is your feature request related to a problem? Please describe.
ADK currently supports using LiteLLM as a model provider. I noticed that while there is a llm_client parameter available, it does not seem to support passing a custom client instance. Moreover, the LiteLLM SDK features a Router that provides powerful load balancing capabilities, which could be particularly useful for managing multiple model deployments.
I prefer programmatic model management over environment variables, especially when working with multiple models that have different base URLs and API keys.
Describe the solution you'd like
I propose either:
- Allowing ADK to globally accept a configured
litellm.Routerinstance, or - Enabling
litellm.Routerto be passed directly via thellm_clientparameter
Describe alternatives you've considered
- LiteLLM Proxy - adds unnecessary external dependency
Reactions are currently unavailable
Metadata
Metadata
Labels
core[Component] This issue is related to the core interface and implementation[Component] This issue is related to the core interface and implementationmodels[Component] Issues related to model support[Component] Issues related to model supportneeds review[Status] The PR/issue is awaiting review from the maintainer[Status] The PR/issue is awaiting review from the maintainer