Skip to content

Add CheaperInference provider - #7010

Open
aiapienthusiast wants to merge 4 commits into
anomalyco:devfrom
aiapienthusiast:cheaperinference-provider
Open

aiapienthusiast wants to merge 4 commits into
anomalyco:devfrom
aiapienthusiast:cheaperinference-provider

Conversation

@aiapienthusiast

@aiapienthusiast aiapienthusiast commented Sep 13, 2026

Copy link
Copy Markdown

Cheaper Inference is an OpenAI-compatible gateway that serves each request from one of several providers for the requested model, ranked by discount, speed or a balance of both (a ranking field; balance by default), at or below the model maker's list price.

Adds providers/cheaperinference/ with provider.toml, logo.svg and 16 models, each base_model-ed against the existing lab entry and carrying only cost, reasoning_options and limit deltas. No new files under models/.

Model base_model input output cache read above 272k in/out
gpt-6-astra openai/gpt-6-astra 7 35 0.7 14 / 52.5
gpt-5.6-sol openai/gpt-5.6-sol 1 5 0.1 2 / 7.5
gpt-5.6-terra openai/gpt-5.6-terra 0.8 4.8 0.08 1.6 / 7.2
gpt-5.6-luna openai/gpt-5.6-luna 0.08 0.48 0.008 0.16 / 0.72
gpt-5-mini openai/gpt-5-mini 0.201421 1.611374 0.020142
gpt-oss-120b openai/gpt-oss-120b 0.04 0.2 0.0085
claude-opus-5 anthropic/claude-opus-5 3.5 17.5 0.35
claude-sonnet-5 anthropic/claude-sonnet-5 1.4 7 0.14
gemini-3.7-flash google/gemini-3.7-flash 0.525 2.625 0.0525
grok-4.5 xai/grok-4.5 1.4 4.2 0.17
kimi-k3 moonshotai/kimi-k3 2.1 10.5 0.239969
qwen-3-8-max alibaba/qwen3.8-max 1.75 5.25 0.2125
glm-5.3 zhipuai/glm-5.3 0.77 2.42 0.119
glm-5.3-flash zhipuai/glm-5.3-flash 0.105 0.35 0.01275
deepseek-v4.1-flash deepseek/deepseek-v4.1-flash 0.120853 0.483412 0.002417
deepseek-v4-flash-0731 deepseek/deepseek-v4-flash-0731 0.032227 0.064454 0.006445

Rates are from GET /v1/models (read 2026-09-13T10:00:38Z, pricing_version sha256:d7fdd65d…), including cache rates and the above-272k band the gateway prices separately for four OpenAI models. The gateway reprices often (88 changes across 73 models in the 12 hours before this PR), so these are a timestamped snapshot. A sync module is included; it needs a CHEAPERINFERENCE_API_KEY secret in the workflow (a read-only key works).

reasoning_options follow the lab entries; toggle is authored only where reasoning.enabled was verified on the live endpoint. limit overrides only where the gateway's declared limits differ from the lab entry (six models).

bun validate passes; bun run test shows the same failures as a clean dev checkout.

CheaperInference is an OpenAI-compatible gateway that serves each request
from one of several providers for the requested model, ranked by discount,
speed or a balance of both, at or below the model maker's list price. Adds the
provider entry plus 16 relayed models, all using base_model against existing
lab metadata with provider-side cost, cache rates, long-context tiers and
reasoning_options only.

Rates come from GET /v1/models (pricing_version sha256:d7fdd65d…, checked
2026-09-13T10:00:38Z).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] providers/cheaperinference/models/claude-sonnet-5.toml:98 - Check: Every toggle must have a leading top-of-file wire comment with the exact request path. Why: This multi-model OpenAI-compatible gateway adds toggle on five models (claude-sonnet-5, deepseek-v4-flash-0731, deepseek-v4.1-flash, kimi-k3, qwen-3-8-max) with only pricing comments above the first key. Callers and the catalog cannot tell which field turns reasoning on/off on this host (e.g. thinking.type, enable_thinking, reasoning.enabled). Action: Add a leading # Toggle: … (and matching effort wire path if applicable) on each of those five files documenting this host’s real control, or drop toggle if the gateway does not expose a separate on/off control.

The gateway documents that the reasoning field is forwarded to the serving
provider but names no on/off control of its own, so only the effort levels
inherited from the lab entries are authored.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [possible mistake] providers/cheaperinference/models/claude-sonnet-5.toml:5 - Check: Relay reasoning_options must match lab + same-surface peers for controls this host actually forwards. Why: Patch 2 removed toggle on claude-sonnet-5, both DeepSeek Flash models, kimi-k3, and qwen-3-8-max while stating the gateway forwards reasoning to the upstream provider. Lab/OpenRouter baselines for those models use a separate on/off control plus graded effort (no none in effort). Effort-only leaves no documented off path and understates capability if thinking / enable_thinking / equivalent fields pass through. Action: Either restore toggle (with a leading wire-path comment) on those five models to match lab/peers, or cite host docs/tests showing on/off fields are stripped or rejected and keep effort-only.
  • [medium] [possible mistake] providers/cheaperinference/models/deepseek-v4-flash-0731.toml:17 - Check: Provider [limit] overrides must be real host deltas from the catalog API, not copy-paste caps. Why: Lab deepseek-v4-flash-0731 output is 384_000, but this file sets output = 65536 while sibling deepseek-v4.1-flash only bumps context and still inherits 384_000 output. That split is inconsistent for one gateway and matches common 64k defaults more than the lab Flash ceiling. Action: Confirm both DeepSeek limits from GET /v1/models (or equivalent) and align output/context with what the host actually serves; drop non-delta overrides.
  • [low] [possible mistake] providers/cheaperinference/provider.toml:1 - Check: Rich provider catalog APIs should get a sync module when they can authoritatively drive model/pricing data. Why: The PR already keys rates off versioned GET /v1/models (pricing_version), so hand-maintained costs/limits will drift. Action: Add a sync module per sync.md, or note in the PR why this catalog stays hand-authored.

aiapienthusiast and others added 2 commits September 13, 2026 17:19
Tested on the live chat completions endpoint: reasoning.enabled = false
removes reasoning tokens and the reasoning field on kimi-k3, qwen-3-8-max,
deepseek-v4.1-flash and deepseek-v4-flash-0731, and true restores them, so
those four carry toggle with the wire path as a leading comment. On
claude-sonnet-5 neither reasoning.enabled nor thinking had any observable
effect on this path, so it stays effort-only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Registers a sync module for the CheaperInference provider so the hourly
workflow keeps its rates current instead of leaving them to drift.

The gateway's catalog endpoint is authoritative for pricing: exact per-million
input, output, cache-read and cache-write rates, an above_threshold band with
its own token threshold for models priced in two bands, and the limits this
host actually serves. It exposes no reasoning controls, so the module sets
skipCreates and refuses to sync a reasoning model whose reasoning_options were
not hand-authored; image and video routes are skipped because their per-unit
media pricing cannot be expressed by the cost schema.

Reading the catalog needs a key, so the workflow gains a
CHEAPERINFERENCE_API_KEY secret. The endpoint accepts a key restricted to
read-only scope, which cannot spend from the account behind it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@aiapienthusiast

Copy link
Copy Markdown
Author

Addressed all three.

  1. toggle: tested on the live /v1/chat/completions. reasoning: {enabled: false} removes reasoning tokens and the reasoning field on kimi-k3, qwen-3-8-max, deepseek-v4.1-flash and deepseek-v4-flash-0731, and enabled: true restores them (kimi-k3: 26 reasoning tokens → 0 → 26). Those four now carry toggle with # Toggle: reasoning.enabled = true|false as a leading comment. On claude-sonnet-5 neither reasoning.enabled nor thinking had any observable effect on this path, so it stays effort-only.
  2. DeepSeek limits: both come from GET /v1/models. deepseek-v4-flash-0731 returns context_length: 1048576, max_output_tokens: 65536; deepseek-v4.1-flash returns context_length: 1048576, max_output_tokens: 384000. Hence 0731 overrides both and 4.1-flash only context.
  3. Sync module: added to this PR (packages/core/src/sync/providers/cheaperinference.ts with tests, registration in sync/index.ts, and a CHEAPERINFERENCE_API_KEY line in the workflow). It only reads GET /v1/models, so a read-only key works; without the secret the matrix job for this provider fails the same way other keyed providers do. Replaying the live catalog through bun models:sync cheaperinference --dry-run gives 0 created, 0 updated, 0 removed, 16 unchanged against the files in this PR.

@github-actions

Copy link
Copy Markdown
Contributor

No actionable findings.

@github-actions github-actions Bot added the reviewer: ready Automated review found no actionable items label Sep 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

reviewer: ready Automated review found no actionable items

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant