Add CheaperInference provider - #7010
Open
aiapienthusiast wants to merge 4 commits into
Open
aiapienthusiast wants to merge 4 commits into
aiapienthusiast wants to merge 4 commits into
Conversation
CheaperInference is an OpenAI-compatible gateway that serves each request from one of several providers for the requested model, ranked by discount, speed or a balance of both, at or below the model maker's list price. Adds the provider entry plus 16 relayed models, all using base_model against existing lab metadata with provider-side cost, cache rates, long-context tiers and reasoning_options only. Rates come from GET /v1/models (pricing_version sha256:d7fdd65d…, checked 2026-09-13T10:00:38Z). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Action items
|
The gateway documents that the reasoning field is forwarded to the serving provider but names no on/off control of its own, so only the effort levels inherited from the lab entries are authored. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Action items
|
Tested on the live chat completions endpoint: reasoning.enabled = false removes reasoning tokens and the reasoning field on kimi-k3, qwen-3-8-max, deepseek-v4.1-flash and deepseek-v4-flash-0731, and true restores them, so those four carry toggle with the wire path as a leading comment. On claude-sonnet-5 neither reasoning.enabled nor thinking had any observable effect on this path, so it stays effort-only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Registers a sync module for the CheaperInference provider so the hourly workflow keeps its rates current instead of leaving them to drift. The gateway's catalog endpoint is authoritative for pricing: exact per-million input, output, cache-read and cache-write rates, an above_threshold band with its own token threshold for models priced in two bands, and the limits this host actually serves. It exposes no reasoning controls, so the module sets skipCreates and refuses to sync a reasoning model whose reasoning_options were not hand-authored; image and video routes are skipped because their per-unit media pricing cannot be expressed by the cost schema. Reading the catalog needs a key, so the workflow gains a CHEAPERINFERENCE_API_KEY secret. The endpoint accepts a key restricted to read-only scope, which cannot spend from the account behind it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Author
|
Addressed all three.
|
Contributor
|
No actionable findings. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cheaper Inference is an OpenAI-compatible gateway that serves each request from one of several providers for the requested model, ranked by discount, speed or a balance of both (a
rankingfield;balanceby default), at or below the model maker's list price.Adds
providers/cheaperinference/withprovider.toml,logo.svgand 16 models, eachbase_model-ed against the existing lab entry and carrying onlycost,reasoning_optionsandlimitdeltas. No new files undermodels/.gpt-6-astraopenai/gpt-6-astragpt-5.6-solopenai/gpt-5.6-solgpt-5.6-terraopenai/gpt-5.6-terragpt-5.6-lunaopenai/gpt-5.6-lunagpt-5-miniopenai/gpt-5-minigpt-oss-120bopenai/gpt-oss-120bclaude-opus-5anthropic/claude-opus-5claude-sonnet-5anthropic/claude-sonnet-5gemini-3.7-flashgoogle/gemini-3.7-flashgrok-4.5xai/grok-4.5kimi-k3moonshotai/kimi-k3qwen-3-8-maxalibaba/qwen3.8-maxglm-5.3zhipuai/glm-5.3glm-5.3-flashzhipuai/glm-5.3-flashdeepseek-v4.1-flashdeepseek/deepseek-v4.1-flashdeepseek-v4-flash-0731deepseek/deepseek-v4-flash-0731Rates are from
GET /v1/models(read 2026-09-13T10:00:38Z,pricing_versionsha256:d7fdd65d…), including cache rates and the above-272k band the gateway prices separately for four OpenAI models. The gateway reprices often (88 changes across 73 models in the 12 hours before this PR), so these are a timestamped snapshot. A sync module is included; it needs aCHEAPERINFERENCE_API_KEYsecret in the workflow (a read-only key works).reasoning_optionsfollow the lab entries;toggleis authored only wherereasoning.enabledwas verified on the live endpoint.limitoverrides only where the gateway's declared limits differ from the lab entry (six models).bun validatepasses;bun run testshows the same failures as a cleandevcheckout.