--- description: Run machine learning models, powered by serverless GPUs, on Cloudflare's global network. title: Cloudflare Workers AI image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Cloudflare Workers AI Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Run machine learning models, powered by serverless GPUs, on Cloudflare's global network. Available on Free and Paid plans Workers AI allows you to run AI models in a serverless way, without having to worry about scaling, maintaining, or paying for unused infrastructure. You can invoke models running on GPUs on Cloudflare's network from your own code — from [Workers](https://developers.cloudflare.com/workers/), [Pages](https://developers.cloudflare.com/pages/), or anywhere via [the Cloudflare API](https://developers.cloudflare.com/api/resources/ai/methods/run/). Workers AI gives you access to: * **50+ [open-source models](https://developers.cloudflare.com/workers-ai/models/)**, available as a part of our model catalog * Serverless, **pay-for-what-you-use** [pricing model](https://developers.cloudflare.com/workers-ai/platform/pricing/) * All as part of a **fully-featured developer platform**, including [AI Gateway](https://developers.cloudflare.com/ai-gateway/), [Vectorize](https://developers.cloudflare.com/vectorize/), [Workers](https://developers.cloudflare.com/workers/) and more... [Get started](https://developers.cloudflare.com/workers-ai/get-started)[Watch a Workers AI demo](https://youtu.be/cK%5FleoJsBWY?si=4u6BIy%5FuBOZf9Ve8) Custom requirements If you have custom requirements like private custom models or higher limits, complete the [Custom Requirements Form ↗](https://forms.gle/axnnpGDb6xrmR31T6). Cloudflare will contact you with next steps. Workers AI is now Generally Available To report bugs or give feedback, go to the [#workers-ai Discord channel ↗](https://discord.cloudflare.com). If you are having issues with Wrangler, report issues in the [Wrangler GitHub repository ↗](https://github.com/cloudflare/workers-sdk/issues/new/choose). --- ## Features [Models](https://developers.cloudflare.com/workers-ai/models/) Workers AI comes with a curated set of popular open-source models that enable you to do tasks such as image classification, text generation, object detection and more. Browse models --- ## Related products [AI Gateway](https://developers.cloudflare.com/ai-gateway/) Observe and control your AI applications with caching, rate limiting, request retries, model fallback, and more. [Vectorize](https://developers.cloudflare.com/vectorize/) Build full-stack AI applications with Vectorize, Cloudflare’s vector database. Adding Vectorize enables you to perform tasks such as semantic search, recommendations, anomaly detection or can be used to provide context and memory to an LLM. [Workers](https://developers.cloudflare.com/workers/) Build serverless applications and deploy instantly across the globe for exceptional performance, reliability, and scale. [Pages](https://developers.cloudflare.com/pages/) Create full-stack applications that are instantly deployed to the Cloudflare global network. [R2](https://developers.cloudflare.com/r2/) Store large amounts of unstructured data without the costly egress bandwidth fees associated with typical cloud storage services. [D1](https://developers.cloudflare.com/d1/) Create new serverless SQL databases to query from your Workers and Pages projects. [Durable Objects](https://developers.cloudflare.com/durable-objects/) A globally distributed coordination API with strongly consistent storage. [KV](https://developers.cloudflare.com/kv/) Create a global, low-latency, key-value data storage. --- ## More resources ### [Get started](https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/) Build and deploy your first Workers AI application. ### [Plans](https://developers.cloudflare.com/workers-ai/platform/pricing/) Learn about Free and Paid plans. ### [Limits](https://developers.cloudflare.com/workers-ai/platform/limits/) Learn about Workers AI limits. ### [Use cases](https://developers.cloudflare.com/use-cases/ai/) Learn how you can build and deploy ambitious AI applications to Cloudflare's global network. ### [Storage options](https://developers.cloudflare.com/workers/platform/storage-options/) Learn which storage option is best for your project. ### [Developer Discord](https://discord.cloudflare.com) Connect with the Workers community on Discord to ask questions, share what you are building, and discuss the platform with other developers. ### [@CloudflareDev](https://x.com/cloudflaredev) Follow @CloudflareDev on Twitter to learn about product announcements, and what is new in Cloudflare Workers. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/#page","headline":"Overview · Cloudflare Workers AI docs","description":"Run machine learning models, powered by serverless GPUs, on Cloudflare's global network.","url":"https://developers.cloudflare.com/workers-ai/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: Set up your first Workers AI project using the dashboard, CLI, or REST API. title: Getting started image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Getting started Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/get-started/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) There are several options to build your Workers AI projects on Cloudflare. To get started, choose your preferred method: * [Workers Bindings](https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/) * [REST API](https://developers.cloudflare.com/workers-ai/get-started/rest-api/) * [Dashboard](https://developers.cloudflare.com/workers-ai/get-started/dashboard/) Note These examples are geared towards creating new Workers AI projects. For help adding Workers AI to an existing Worker, refer to [Workers Bindings](https://developers.cloudflare.com/workers-ai/configuration/bindings/). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/get-started/#page","headline":"Getting started · Cloudflare Workers AI docs","description":"Set up your first Workers AI project using the dashboard, CLI, or REST API.","url":"https://developers.cloudflare.com/workers-ai/get-started/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Create and deploy a Workers AI application using the Cloudflare dashboard. title: Dashboard image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Dashboard Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/get-started/dashboard/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Follow this guide to create a Workers AI application using the Cloudflare dashboard. ## Prerequisites Sign up for a [Cloudflare account ↗](https://dash.cloudflare.com/sign-up/workers-and-pages) if you have not already. ## Setup To create a Workers AI application: 1. In the Cloudflare dashboard, go to the **Workers & Pages** page. [Go to **Workers & Pages** ↗](https://dash.cloudflare.com/?to=/:account/workers-and-pages) 2. Select **Create application**. 3. Under **Select a template**, select **LLM Chat App**. 4. Select **Deploy**. 5. Name your Worker, then select **Create and deploy**. 6. Preview your Worker at its provided [workers.dev](https://developers.cloudflare.com/workers/configuration/routing/workers-dev/) subdomain. ## Development ### Dashboard Editing in the dashboard is helpful for simpler use cases. Once you have created your Worker script, you can edit and deploy your Worker using the Cloudflare dashboard: 1. In the Cloudflare dashboard, go to the **Workers & Pages** page. [Go to **Workers & Pages** ↗](https://dash.cloudflare.com/?to=/:account/workers-and-pages) 2. Select your application. 3. Select **Edit Code**. ![Edit code directly within the Cloudflare dashboard](https://developers.cloudflare.com/cdn-cgi/image/onerror=redirect,width=566,height=386,format=webp/_astro/workers-edit-code.CKxxvQSe.png) ### Wrangler CLI To develop more advanced applications or [implement tests](https://developers.cloudflare.com/workers/testing/), start working in the Wrangler CLI. 1. Install [npm ↗](https://docs.npmjs.com/getting-started). 2. Install [Node.js ↗](https://nodejs.org/en/). Node.js version manager Use a Node version manager like [Volta ↗](https://volta.sh/) or [nvm ↗](https://github.com/nvm-sh/nvm) to avoid permission issues and change Node.js versions. [Wrangler](https://developers.cloudflare.com/workers/wrangler/install-and-update/), discussed later in this guide, requires a Node version of `16.17.0` or later. 1. Run the following command, replacing the value of `[]` which the location you want to put your Worker Script. npmyarnpnpm ``` npm create cloudflare@latest -- [] --type=pre-existing ``` ``` yarn create cloudflare [] --type=pre-existing ``` ``` pnpm create cloudflare@latest [] --type=pre-existing ``` After you run this command - and work through the prompts - your local changes will not automatically sync with dashboard. So, once you download your script, continue using the CLI. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/get-started/dashboard/#page","headline":"Get started - Dashboard · Cloudflare Workers AI docs","description":"Create and deploy a Workers AI application using the Cloudflare dashboard.","url":"https://developers.cloudflare.com/workers-ai/get-started/dashboard/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Use the Cloudflare Workers AI REST API to deploy a large language model (LLM). title: REST API image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # REST API Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/get-started/rest-api/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) This guide will instruct you through setting up and deploying your first Workers AI project. You will use the Workers AI REST API to experiment with a large language model (LLM). ## Prerequisites Sign up for a [Cloudflare account ↗](https://dash.cloudflare.com/sign-up/workers-and-pages) if you have not already. ## 1\. Get API token and Account ID You need your API token and Account ID to use the REST API. To get these values: 1. In the Cloudflare dashboard, go to the **Workers AI** page. [Go to **Workers AI** ↗](https://dash.cloudflare.com/?to=/:account/ai/workers-ai) 2. Select **Use REST API**. 3. Get your API token: 1. Select **Create a Workers AI API Token**. 2. Review the prefilled information. 3. Select **Create API Token**. 4. Select **Copy API Token**. 5. Save that value for future use. This token will be visible [on your profile](https://developers.cloudflare.com/api/get-started/create-token/). 4. For **Get Account ID**, copy the value for **Account ID**. Save that value for future use. Note If you choose to [create an API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) instead of using the template, that token will need permissions for both `Workers AI - Read` and `Workers AI - Edit`. ## 2\. Run a model via API After creating your API token, authenticate and make requests to the API using your API token in the request. You will use the [Execute AI model](https://developers.cloudflare.com/api/resources/ai/methods/run/) endpoint to run the [@cf/meta/llama-3.1-8b-instruct](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct/) model: ```bash curl https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/meta/llama-3.1-8b-instruct \ -H 'Authorization: Bearer {API_TOKEN}' \ -d '{ "prompt": "Where did the phrase Hello World come from" }' ``` Replace the values for `{ACCOUNT_ID}` and `{API_TOKEN}`. The API response will look like the following: ```json { "result": { "response": "Hello, World first appeared in 1974 at Bell Labs when Brian Kernighan included it in the C programming language example. It became widely used as a basic test program due to simplicity and clarity. It represents an inviting greeting from a program to the world." }, "success": true, "errors": [], "messages": [] } ``` This example execution uses the `@cf/meta/llama-3.1-8b-instruct` model, but you can use any of the models in the [Workers AI models catalog](https://developers.cloudflare.com/workers-ai/models/). If using another model, you will need to replace `{model}` with your desired model name. By completing this guide, you have created a Cloudflare account (if you did not have one already) and an API token that grants Workers AI read permissions to your account. You executed the [@cf/meta/llama-3.1-8b-instruct](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct/) model using a cURL command from the terminal and received an answer to your prompt in a JSON response. ## Related resources * [Models](https://developers.cloudflare.com/workers-ai/models/) \- Browse the Workers AI models catalog. * [AI SDK](https://developers.cloudflare.com/workers-ai/configuration/ai-sdk) \- Learn how to integrate with an AI model. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/get-started/rest-api/#page","headline":"Get started - REST API · Cloudflare Workers AI docs","description":"Use the Cloudflare Workers AI REST API to deploy a large language model (LLM).","url":"https://developers.cloudflare.com/workers-ai/get-started/rest-api/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Deploy your first Cloudflare Workers AI project using the CLI. title: Workers Bindings image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Workers Bindings Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) This guide will instruct you through setting up and deploying your first Workers AI project. You will use [Workers](https://developers.cloudflare.com/workers/), a Workers AI binding, and a large language model (LLM) to deploy your first AI-powered application on the Cloudflare global network. 1. Sign up for a [Cloudflare account ↗](https://dash.cloudflare.com/sign-up/workers-and-pages). 2. Install [Node.js ↗](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm). Node.js version manager Use a Node version manager like [Volta ↗](https://volta.sh/) or [nvm ↗](https://github.com/nvm-sh/nvm) to avoid permission issues and change Node.js versions. [Wrangler](https://developers.cloudflare.com/workers/wrangler/install-and-update/), discussed later in this guide, requires a Node version of `16.17.0` or later. ## 1\. Create a Worker project You will create a new Worker project using the `create-cloudflare` CLI (C3). [C3 ↗](https://github.com/cloudflare/workers-sdk/tree/main/packages/create-cloudflare) is a command-line tool designed to help you set up and deploy new applications to Cloudflare. Create a new project named `hello-ai` by running: npmyarnpnpm ``` npm create cloudflare@latest -- hello-ai ``` ``` yarn create cloudflare hello-ai ``` ``` pnpm create cloudflare@latest hello-ai ``` Running `npm create cloudflare@latest` will prompt you to install the [create-cloudflare package ↗](https://www.npmjs.com/package/create-cloudflare), and lead you through setup. C3 will also install [Wrangler](https://developers.cloudflare.com/workers/wrangler/), the Cloudflare Developer Platform CLI. For setup, select the following options: * For _What would you like to start with?_, choose `Hello World example`. * For _Which template would you like to use?_, choose `Worker only`. * For _Which language do you want to use?_, choose `TypeScript`. * For _Do you want to use git for version control?_, choose `Yes`. * For _Do you want to deploy your application?_, choose `No` (we will be making some changes before deploying). This will create a new `hello-ai` directory. Your new `hello-ai` directory will include: * A `"Hello World"` [Worker](https://developers.cloudflare.com/workers/get-started/guide/#3-write-code) at `src/index.ts`. * A [wrangler.jsonc](https://developers.cloudflare.com/workers/wrangler/configuration/) configuration file. Go to your application directory: ```sh cd hello-ai ``` ## 2\. Connect your Worker to Workers AI You must create an AI binding for your Worker to connect to Workers AI. [Bindings](https://developers.cloudflare.com/workers/runtime-apis/bindings/) allow your Workers to interact with resources, like Workers AI, on the Cloudflare Developer Platform. To bind Workers AI to your Worker, add the following to the end of your Wrangler file: ```jsonc { "ai": { "binding": "AI" } } ``` ```toml [ai] binding = "AI" ``` Your binding is [available in your Worker code](https://developers.cloudflare.com/workers/reference/migrate-to-module-workers/#bindings-in-es-modules-format) on [env.AI](https://developers.cloudflare.com/workers/runtime-apis/handlers/fetch/). You can also bind Workers AI to a Pages Function. For more information, refer to [Functions Bindings](https://developers.cloudflare.com/pages/functions/bindings/#workers-ai). ## 3\. Run an inference task in your Worker You are now ready to run an inference task in your Worker. In this case, you will use an LLM, [llama-3.1-8b-instruct](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct/), to answer a question. Update the `index.ts` file in your `hello-ai` application directory with the following code: ```js export default { async fetch(request, env) { const response = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", { prompt: "What is the origin of the phrase Hello, World", }); return new Response(JSON.stringify(response)); }, }; ``` ```ts export interface Env { // If you set another name in the Wrangler config file as the value for 'binding', // replace "AI" with the variable name you defined. AI: Ai; } export default { async fetch(request, env): Promise { const response = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", { prompt: "What is the origin of the phrase Hello, World", }); return new Response(JSON.stringify(response)); }, } satisfies ExportedHandler; ``` Up to this point, you have created an AI binding for your Worker and configured your Worker to be able to execute the Llama 3.1 model. You can now test your project locally before you deploy globally. ## 4\. Develop locally with Wrangler While in your project directory, test Workers AI locally by running [wrangler dev](https://developers.cloudflare.com/workers/wrangler/commands/general/#dev): ```sh npx wrangler dev ``` Workers AI local development usage charges Using Workers AI always accesses your Cloudflare account in order to run AI models and will incur usage charges even in local development. You will be prompted to log in after you run `wrangler dev`. When you run `npx wrangler dev`, Wrangler will give you a URL (most likely `localhost:8787`) to review your Worker. After you go to the URL Wrangler provides, a message will render that resembles the following example: ```json { "response": "Ah, a most excellent question, my dear human friend! *adjusts glasses*\n\nThe origin of the phrase \"Hello, World\" is a fascinating tale that spans several decades and multiple disciplines. It all began in the early days of computer programming, when a young man named Brian Kernighan was tasked with writing a simple program to demonstrate the basics of a new programming language called C.\nKernighan, a renowned computer scientist and author, was working at Bell Labs in the late 1970s when he created the program. He wanted to showcase the language's simplicity and versatility, so he wrote a basic \"Hello, World!\" program that printed the familiar greeting to the console.\nThe program was included in Kernighan and Ritchie's influential book \"The C Programming Language,\" published in 1978. The book became a standard reference for C programmers, and the \"Hello, World!\" program became a sort of \"Hello, World!\" for the programming community.\nOver time, the phrase \"Hello, World!\" became a shorthand for any simple program that demonstrated the basics" } ``` ## 5\. Deploy your AI Worker Before deploying your AI Worker globally, log in with your Cloudflare account by running: ```sh npx wrangler login ``` You will be directed to a web page asking you to log in to the Cloudflare dashboard. After you have logged in, you will be asked if Wrangler can make changes to your Cloudflare account. Scroll down and select **Allow** to continue. Finally, deploy your Worker to make your project accessible on the Internet. To deploy your Worker, run: ```sh npx wrangler deploy ``` ```sh https://hello-ai..workers.dev ``` Your Worker will be deployed to your custom [workers.dev](https://developers.cloudflare.com/workers/configuration/routing/workers-dev/) subdomain. You can now visit the URL to run your AI Worker. By finishing this tutorial, you have created a Worker, connected it to Workers AI through an AI binding, and ran an inference task from the Llama 3 model. ## Related resources * [Cloudflare Developers community on Discord ↗](https://discord.cloudflare.com) \- Submit feature requests, report bugs, and share your feedback directly with the Cloudflare team by joining the Cloudflare Discord server. * [Models](https://developers.cloudflare.com/workers-ai/models/) \- Browse the Workers AI models catalog. * [AI SDK](https://developers.cloudflare.com/workers-ai/configuration/ai-sdk) \- Learn how to integrate with an AI model. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/#page","headline":"Get started - Workers and Wrangler · Cloudflare Workers AI docs","description":"Deploy your first Cloudflare Workers AI project using the CLI.","url":"https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Browse the catalog of machine learning models available on Workers AI. title: Workers AI Models image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Workers AI Models Last updated Aug 12, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/models/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Looking for more models, including external providers? Check out the [unified AI model catalog](https://developers.cloudflare.com/ai/models/). Task TypesCapabilitiesAuthorsNewest first We found 81 modelsClear filters No models found Try a different search term, or broaden your search by removing filters. [![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg)aura-1DeepgramText-to-SpeechAura is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output.Cloudflare-hostedBatchPartnerReal-time](https://developers.cloudflare.com/workers-ai/models/aura-1/) [![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg)aura-2-enDeepgramText-to-SpeechAura-2 is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output.Cloudflare-hostedBatchPartnerReal-time](https://developers.cloudflare.com/workers-ai/models/aura-2-en/) [![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg)aura-2-esDeepgramText-to-SpeechAura-2 is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output.Cloudflare-hostedBatchPartnerReal-time](https://developers.cloudflare.com/workers-ai/models/aura-2-es/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)bart-large-cnnBetaMetaSummarizationBART is a transformer encoder-encoder (seq2seq) model with a bidirectional (BERT-like) encoder and an autoregressive (GPT-like) decoder. You can use this model for text summarization.Cloudflare-hostedDeprecated](https://developers.cloudflare.com/workers-ai/models/bart-large-cnn/) [![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg)bge-base-en-v1.5BAAIText EmbeddingsBAAI general embedding (Base) model that transforms any given text into a 768-dimensional vectorCloudflare-hostedBatch](https://developers.cloudflare.com/workers-ai/models/bge-base-en-v1.5/) [![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg)bge-large-en-v1.5BAAIText EmbeddingsBAAI general embedding (Large) model that transforms any given text into a 1024-dimensional vectorCloudflare-hostedBatch](https://developers.cloudflare.com/workers-ai/models/bge-large-en-v1.5/) [![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg)bge-m3BAAIText EmbeddingsMulti-Functionality, Multi-Linguality, and Multi-Granularity embeddings model.Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/bge-m3/) [![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg)bge-reranker-baseBAAIText ClassificationDifferent from embedding model, reranker uses question and document as input and directly output similarity instead of embedding. You can get a relevance score by inputting query and passage to the reranker. And the score can be mapped to a float value in \[0,1\] by sigmoid function. Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/bge-reranker-base/) [![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg)bge-small-en-v1.5BAAIText EmbeddingsBAAI general embedding (Small) model that transforms any given text into a 384-dimensional vectorCloudflare-hostedBatch](https://developers.cloudflare.com/workers-ai/models/bge-small-en-v1.5/) [![DeepSeek logo](https://developers.cloudflare.com/_astro/deepseek.CkzAgvN6.svg)deepseek-r1-distill-qwen-32bDeepSeekText GenerationDeepSeek-R1-Distill-Qwen-32B is a model distilled from DeepSeek-R1 based on Qwen2.5\. It outperforms OpenAI-o1-mini across various benchmarks, achieving new state-of-the-art results for dense models.Cloudflare-hostedReasoning](https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)detr-resnet-50BetaMetaObject DetectionDEtection TRansformer (DETR) model trained end-to-end on COCO 2017 object detection (118k annotated images).Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/detr-resnet-50/) [![HuggingFace logo](https://developers.cloudflare.com/_astro/huggingface.DMS-v5TA.svg)distilbert-sst-2-int8HuggingFaceText ClassificationDistilled BERT model that was finetuned on SST-2 for sentiment classificationCloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/distilbert-sst-2-int8/) [ldreamshaper-8-lcmlykonText-to-ImageStable Diffusion model that has been fine-tuned to be better at photorealism without sacrificing range.Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/dreamshaper-8-lcm/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)embeddinggemma-300mGoogleText EmbeddingsEmbeddingGemma is a 300M parameter, state-of-the-art for its size, open embedding model from Google, built from Gemma 3 (with T5Gemma initialization) and the same research and technology used to create Gemini models. EmbeddingGemma produces vector representations of text, making it well-suited for search and retrieval tasks, including classification, clustering, and semantic similarity search. This model was trained with data in 100+ spoken languages.Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/embeddinggemma-300m/) [![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg)fluxDeepgramAutomatic Speech RecognitionFlux is the first conversational speech recognition model built specifically for voice agents.Cloudflare-hostedPartnerReal-time](https://developers.cloudflare.com/workers-ai/models/flux/) [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)flux-1-schnellBlack Forest LabsText-to-ImageFLUX.1 \[schnell\] is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions. Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/flux-1-schnell/) [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)flux-2-devBlack Forest LabsText-to-ImageFLUX.2 \[dev\] is an image model from Black Forest Labs where you can generate highly realistic and detailed images, with multi-reference support.Cloudflare-hostedPartner](https://developers.cloudflare.com/workers-ai/models/flux-2-dev/) [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)flux-2-klein-4bBlack Forest LabsText-to-ImageFLUX.2 \[klein\] is an ultra-fast, distilled image model. It unifies image generation and editing in a single model, delivering state-of-the-art quality enabling interactive workflows, real-time previews, and latency-critical applications.Cloudflare-hostedPartner](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-4b/) [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)flux-2-klein-9bBlack Forest LabsText-to-ImageFLUX.2 \[klein\] 9B is an ultra-fast, distilled image model with enhanced quality. It unifies image generation and editing in a single model, delivering state-of-the-art quality enabling interactive workflows, real-time previews, and latency-critical applications.Cloudflare-hostedPartner](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-9b/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemma-2b-it-loraBetaGoogleText GenerationThis is a Gemma-2B base model that Cloudflare dedicates for inference with LoRA adapters. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models.Cloudflare-hostedLoRA](https://developers.cloudflare.com/workers-ai/models/gemma-2b-it-lora/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemma-3-12b-itGoogleText GenerationGemma 3 models are well-suited for a variety of text generation and image understanding tasks, including question answering, summarization, and reasoning. Gemma 3 models are multimodal, handling text and image input and generating text output, with a large, 128K context window, multilingual support in over 140 languages, and is available in more sizes than previous versions.Cloudflare-hostedLoRADeprecated](https://developers.cloudflare.com/workers-ai/models/gemma-3-12b-it/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemma-4-26b-a4b-itGoogleText GenerationGemma 4 is Google's most intelligent family of open models, built from Gemini 3 research to maximize intelligence-per-parameter.Cloudflare-hostedFunction callingReasoningVision](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemma-7b-itBetaGoogleText GenerationGemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. They are text-to-text, decoder-only large language models, available in English, with open weights, pre-trained variants, and instruction-tuned variants.Cloudflare-hostedLoRADeprecated](https://developers.cloudflare.com/workers-ai/models/gemma-7b-it/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemma-7b-it-loraBetaGoogleText Generation This is a Gemma-7B base model that Cloudflare dedicates for inference with LoRA adapters. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models.Cloudflare-hostedLoRA](https://developers.cloudflare.com/workers-ai/models/gemma-7b-it-lora/) [agemma-sea-lion-v4-27b-itaisingaporeText GenerationSEA-LION stands for Southeast Asian Languages In One Network, which is a collection of Large Language Models (LLMs) which have been pretrained and instruct-tuned for the Southeast Asia (SEA) region.Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/) [Pinned![Zhipu AI logo](https://developers.cloudflare.com/_astro/zai-org.Dj2vcayE.svg)glm-4.7-flashZhipu AIText GenerationGLM-4.7-Flash is a fast and efficient multilingual text generation model with a 131,072 token context window. Optimized for dialogue, instruction-following, and multi-turn tool calling across 100+ languages.Cloudflare-hostedFunction callingReasoning](https://developers.cloudflare.com/workers-ai/models/glm-4.7-flash/) [![Zhipu AI logo](https://developers.cloudflare.com/_astro/zai-org.Dj2vcayE.svg)glm-5.2Zhipu AIText GenerationZ.ai's flagship agentic coding modelCloudflare-hostedFunction callingReasoning](https://developers.cloudflare.com/workers-ai/models/glm-5.2/) [Pinned![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-oss-120bOpenAIText GenerationOpenAI's open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases – gpt-oss-120b is for production, general purpose, high reasoning use-cases.Cloudflare-hostedFunction callingReasoning](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-oss-20bOpenAIText GenerationOpenAI's open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases – gpt-oss-20b is for lower latency, and local or specialized use-cases.Cloudflare-hostedFunction callingReasoning](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/) [![IBM logo](https://developers.cloudflare.com/_astro/ibm.CYdLt4EI.svg)granite-4.0-h-microIBMText GenerationGranite 4.0 instruct models deliver strong performance across benchmarks, achieving industry-leading results in key agentic tasks like instruction following and function calling. These efficiencies make the models well-suited for a wide range of use cases like retrieval-augmented generation (RAG), multi-agent workflows, and edge deployments.Cloudflare-hostedFunction calling](https://developers.cloudflare.com/workers-ai/models/granite-4.0-h-micro/) [nhermes-2-pro-mistral-7bBetanousresearchText GenerationHermes 2 Pro on Mistral 7B is the new flagship 7B Hermes! Hermes 2 Pro is an upgraded, retrained version of Nous Hermes 2, consisting of an updated and cleaned version of the OpenHermes 2.5 Dataset, as well as a newly introduced Function Calling and JSON Mode dataset developed in-house.Cloudflare-hostedFunction callingDeprecated](https://developers.cloudflare.com/workers-ai/models/hermes-2-pro-mistral-7b/) [aindictrans2-en-indic-1Bai4bharatTranslationIndicTrans2 is the first open-source transformer-based multilingual NMT model that supports high-quality translations across all the 22 scheduled Indic languagesCloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/indictrans2-en-indic-1B/) [![Moonshot AI logo](https://developers.cloudflare.com/_astro/moonshotai.DjWMkXUS.svg)kimi-k2.5Moonshot AIText GenerationKimi K2.5 is a frontier-scale open-source model with a 256k context window, multi-turn tool calling, vision inputs, and structured outputs for agentic workloads.Cloudflare-hostedFunction callingDeprecatedReasoningVision](https://developers.cloudflare.com/workers-ai/models/kimi-k2.5/) [![Moonshot AI logo](https://developers.cloudflare.com/_astro/moonshotai.DjWMkXUS.svg)kimi-k2.6Moonshot AIText GenerationKimi K2.6 is a frontier-scale open-source 1T parameter model with a 262.1k context window, multi-turn tool calling, vision inputs, and structured outputs for agentic workloads.Cloudflare-hostedFunction callingReasoningVision](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/) [Pinned![Moonshot AI logo](https://developers.cloudflare.com/_astro/moonshotai.DjWMkXUS.svg)kimi-k2.7-codeMoonshot AIText GenerationKimi K2.7 is a frontier-scale open-source 1T parameter model with a 262.1k context window, multi-turn tool calling, vision inputs, and structured outputs for agentic workloads.Cloudflare-hostedFunction callingReasoningVision](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-2-7b-chat-fp16MetaText GenerationFull precision (fp16) generative text model with 7 billion parameters from MetaCloudflare-hostedDeprecated](https://developers.cloudflare.com/workers-ai/models/llama-2-7b-chat-fp16/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-2-7b-chat-hf-loraBetaMetaText GenerationThis is a Llama2 base model that Cloudflare dedicated for inference with LoRA adapters. Llama 2 is a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. This is the repository for the 7B fine-tuned model, optimized for dialogue use cases and converted for the Hugging Face Transformers format. Cloudflare-hostedLoRA](https://developers.cloudflare.com/workers-ai/models/llama-2-7b-chat-hf-lora/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-2-7b-chat-int8MetaText GenerationQuantized (int8) generative text model with 7 billion parameters from MetaCloudflare-hostedDeprecated](https://developers.cloudflare.com/workers-ai/models/llama-2-7b-chat-int8/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3-8b-instructMetaText GenerationGeneration over generation, Meta Llama 3 demonstrates state-of-the-art performance on a wide range of industry benchmarks and offers new capabilities, including improved reasoning.Cloudflare-hostedDeprecated](https://developers.cloudflare.com/workers-ai/models/llama-3-8b-instruct/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3-8b-instruct-awqMetaText GenerationQuantized (int4) generative text model with 8 billion parameters from Meta.Cloudflare-hostedDeprecated](https://developers.cloudflare.com/workers-ai/models/llama-3-8b-instruct-awq/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.1-70b-instructMetaText GenerationThe Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction tuned generative models. The Llama 3.1 instruction tuned text only models are optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks.Cloudflare-hostedDeprecated](https://developers.cloudflare.com/workers-ai/models/llama-3.1-70b-instruct/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.1-8b-instructMetaText GenerationThe Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction tuned generative models. The Llama 3.1 instruction tuned text only models are optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks.Cloudflare-hostedDeprecated](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.1-8b-instruct-awqMetaText GenerationQuantized (int4) generative text model with 8 billion parameters from Meta. Cloudflare-hostedDeprecated](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-awq/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.1-8b-instruct-fastMetaText Generation\[Fast version\] The Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction tuned generative models. The Llama 3.1 instruction tuned text only models are optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks.Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fast/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.1-8b-instruct-fp8MetaText GenerationLlama 3.1 8B quantized to FP8 precisionCloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fp8/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.2-11b-vision-instructMetaText Generation The Llama 3.2-Vision instruction-tuned models are optimized for visual recognition, image reasoning, captioning, and answering general questions about an image.Cloudflare-hostedLoRAVision](https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.2-1b-instructMetaText GenerationThe Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks.Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/llama-3.2-1b-instruct/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.2-3b-instructMetaText GenerationThe Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks.Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/llama-3.2-3b-instruct/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.3-70b-instruct-fp8-fastMetaText GenerationLlama 3.3 70B quantized to fp8 precision, optimized to be faster.Cloudflare-hostedBatchFunction calling](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/) [Pinned![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-4-scout-17b-16e-instructMetaText GenerationMeta's Llama 4 Scout is a 17 billion parameter model with 16 experts that is natively multimodal. These models leverage a mixture-of-experts architecture to offer industry-leading performance in text and image understanding.Cloudflare-hostedBatchFunction callingVision](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-guard-3-8bMetaText GenerationLlama Guard 3 is a Llama-3.1-8B pretrained model, fine-tuned for content safety classification. Similar to previous versions, it can be used to classify content in both LLM inputs (prompt classification) and in LLM responses (response classification). It acts as an LLM – it generates text in its output that indicates whether a given prompt or response is safe or unsafe, and if unsafe, it also lists the content categories violated.Cloudflare-hostedLoRA](https://developers.cloudflare.com/workers-ai/models/llama-guard-3-8b/) [lllava-1.5-7b-hfBetallava-hfImage-to-TextLLaVA is an open-source chatbot trained by fine-tuning LLaMA/Vicuna on GPT-generated multimodal instruction-following data. It is an auto-regressive language model, based on the transformer architecture.Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/llava-1.5-7b-hf/) [![Leonardo logo](https://developers.cloudflare.com/_astro/leonardo.JZysY-g3.svg)lucid-originLeonardoText-to-ImageLucid Origin from Leonardo.AI is their most adaptable and prompt-responsive model to date. Whether you're generating images with sharp graphic design, stunning full-HD renders, or highly specific creative direction, it adheres closely to your prompts, renders text with accuracy, and supports a wide array of visual styles and aesthetics – from stylized concept art to crisp product mockups. Cloudflare-hostedPartner](https://developers.cloudflare.com/workers-ai/models/lucid-origin/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)m2m100-1.2bMetaTranslationMultilingual encoder-decoder (seq-to-seq) model trained for Many-to-Many multilingual translationCloudflare-hostedBatch](https://developers.cloudflare.com/workers-ai/models/m2m100-1.2b/) [![MyShell logo](https://developers.cloudflare.com/_astro/myshell.6ROagMV2.svg)melottsMyShellText-to-SpeechMeloTTS is a high-quality multi-lingual text-to-speech library by MyShell.ai.Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/melotts/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)meta-llama-3-8b-instructMetaText GenerationGeneration over generation, Meta Llama 3 demonstrates state-of-the-art performance on a wide range of industry benchmarks and offers new capabilities, including improved reasoning. Cloudflare-hostedDeprecated](https://developers.cloudflare.com/workers-ai/models/meta-llama-3-8b-instruct/) [![MistralAI logo](https://developers.cloudflare.com/_astro/mistralai.Bn9UMUMu.svg)mistral-7b-instruct-v0.1MistralAIText GenerationInstruct fine-tuned version of the Mistral-7b generative text model with 7 billion parametersCloudflare-hostedLoRADeprecated](https://developers.cloudflare.com/workers-ai/models/mistral-7b-instruct-v0.1/) [![MistralAI logo](https://developers.cloudflare.com/_astro/mistralai.Bn9UMUMu.svg)mistral-7b-instruct-v0.2BetaMistralAIText GenerationThe Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.2\. Mistral-7B-v0.2 has the following changes compared to Mistral-7B-v0.1: 32k context window (vs 8k context in v0.1), rope-theta = 1e6, and no Sliding-Window Attention.Cloudflare-hostedLoRADeprecated](https://developers.cloudflare.com/workers-ai/models/mistral-7b-instruct-v0.2/) [![MistralAI logo](https://developers.cloudflare.com/_astro/mistralai.Bn9UMUMu.svg)mistral-7b-instruct-v0.2-loraBetaMistralAIText GenerationThe Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.2.Cloudflare-hostedLoRA](https://developers.cloudflare.com/workers-ai/models/mistral-7b-instruct-v0.2-lora/) [![MistralAI logo](https://developers.cloudflare.com/_astro/mistralai.Bn9UMUMu.svg)mistral-small-3.1-24b-instructMistralAIText GenerationBuilding upon Mistral Small 3 (2501), Mistral Small 3.1 (2503) adds state-of-the-art vision understanding and enhances long context capabilities up to 128k tokens without compromising text performance. With 24 billion parameters, this model achieves top-tier capabilities in both text and vision tasks.Cloudflare-hostedFunction calling](https://developers.cloudflare.com/workers-ai/models/mistral-small-3.1-24b-instruct/) [mmoondream3.1-9B-A2BmoondreamImage-to-TextMoondream 3 is a fast, efficient 9B mixture-of-experts vision language model (2B active parameters) that delivers frontier-level visual reasoning for tasks like object detection, pointing, OCR, and structured output.Cloudflare-hostedVision](https://developers.cloudflare.com/workers-ai/models/moondream3.1-9B-A2B/) [![NVIDIA logo](https://developers.cloudflare.com/_astro/nvidia.DI1bb8hH.svg)nemotron-3-120b-a12bNVIDIAText GenerationNVIDIA Nemotron 3 Super is a hybrid MoE model with leading accuracy for multi-agent applications and specialized agentic AI systems.Cloudflare-hostedFunction callingReasoning](https://developers.cloudflare.com/workers-ai/models/nemotron-3-120b-a12b/) [![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg)nova-3DeepgramAutomatic Speech RecognitionTranscribe audio using Deepgram’s speech-to-text modelCloudflare-hostedBatchPartnerReal-time](https://developers.cloudflare.com/workers-ai/models/nova-3/) [![Microsoft logo](https://developers.cloudflare.com/_astro/microsoft.LujcDJ--.svg)phi-2BetaMicrosoftText GenerationPhi-2 is a Transformer-based model with a next-word prediction objective, trained on 1.4T tokens from multiple passes on a mixture of Synthetic and Web datasets for NLP and coding.Cloudflare-hostedDeprecated](https://developers.cloudflare.com/workers-ai/models/phi-2/) [![Leonardo logo](https://developers.cloudflare.com/_astro/leonardo.JZysY-g3.svg)phoenix-1.0LeonardoText-to-ImagePhoenix 1.0 is a model by Leonardo.Ai that generates images with exceptional prompt adherence and coherent text.Cloudflare-hostedPartner](https://developers.cloudflare.com/workers-ai/models/phoenix-1.0/) [pplamo-embedding-1bpfnetText EmbeddingsPLaMo-Embedding-1B is a Japanese text embedding model developed by Preferred Networks, Inc. It can convert Japanese text input into numerical vectors and can be used for a wide range of applications, including information retrieval, text classification, and clustering.Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/plamo-embedding-1b/) [![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg)qwen2.5-coder-32b-instructQwenText GenerationQwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:Cloudflare-hostedLoRA](https://developers.cloudflare.com/workers-ai/models/qwen2.5-coder-32b-instruct/) [![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg)qwen3-30b-a3b-fp8QwenText GenerationQwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support.Cloudflare-hostedBatchFunction callingReasoning](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/) [![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg)qwen3-embedding-0.6bQwenText EmbeddingsThe Qwen3 Embedding model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks. Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/qwen3-embedding-0.6b/) [![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg)qwq-32bQwenText GenerationQwQ is the reasoning model of the Qwen series. Compared with conventional instruction-tuned models, QwQ, which is capable of thinking and reasoning, can achieve significantly enhanced performance in downstream tasks, especially hard problems. QwQ-32B is the medium-sized reasoning model, which is capable of achieving competitive performance against state-of-the-art reasoning models, e.g., DeepSeek-R1, o1-mini.Cloudflare-hostedLoRAReasoning](https://developers.cloudflare.com/workers-ai/models/qwq-32b/) [![Microsoft logo](https://developers.cloudflare.com/_astro/microsoft.LujcDJ--.svg)resnet-50MicrosoftImage Classification50 layers deep image classification CNN trained on more than 1M images from ImageNetCloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/resnet-50/) [![Pipecat logo](https://developers.cloudflare.com/_astro/pipecat.B-PNBdef.svg)smart-turn-v2PipecatVoice Activity DetectionAn open source, community-driven, native audio turn detection model in 2nd versionCloudflare-hostedBatchReal-time](https://developers.cloudflare.com/workers-ai/models/smart-turn-v2/) [![Defog logo](https://developers.cloudflare.com/_astro/defog.C0vfV4et.svg)sqlcoder-7b-2BetaDefogText GenerationThis model is intended to be used by non-technical users to understand data inside their SQL databases. Cloudflare-hostedDeprecated](https://developers.cloudflare.com/workers-ai/models/sqlcoder-7b-2/) [![RunwayML logo](https://developers.cloudflare.com/_astro/runway.Cq8Cjov4.svg)stable-diffusion-v1-5-img2imgBetaRunwayMLText-to-ImageStable Diffusion is a latent text-to-image diffusion model capable of generating photo-realistic images. Img2img generate a new image from an input image with Stable Diffusion. Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-v1-5-img2img/) [![RunwayML logo](https://developers.cloudflare.com/_astro/runway.Cq8Cjov4.svg)stable-diffusion-v1-5-inpaintingBetaRunwayMLText-to-ImageStable Diffusion Inpainting is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input, with the extra capability of inpainting the pictures by using a mask.Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-v1-5-inpainting/) [![Stability.ai logo](https://developers.cloudflare.com/_astro/stabilityai.VsBx3CKv.svg)stable-diffusion-xl-base-1.0BetaStability.aiText-to-ImageDiffusion-based text-to-image generative model by Stability AI. Generates and modify images based on text prompts.Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-base-1.0/) [![ByteDance logo](https://developers.cloudflare.com/_astro/bytedance.T1uiROQ6.svg)stable-diffusion-xl-lightningBetaByteDanceText-to-ImageSDXL-Lightning is a lightning-fast text-to-image generation model. It can generate high-quality 1024px images in a few steps.Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-lightning/) [![Unum logo](https://developers.cloudflare.com/_astro/unum.CWYcwnUh.svg)uform-gen2-qwen-500mBetaUnumImage-to-TextUForm-Gen is a small generative vision-language model primarily designed for Image Captioning and Visual Question Answering. The model was pre-trained on the internal image captioning dataset and fine-tuned on public instructions datasets: SVIT, LVIS, VQAs datasets.Cloudflare-hostedDeprecated](https://developers.cloudflare.com/workers-ai/models/uform-gen2-qwen-500m/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)whisperOpenAIAutomatic Speech RecognitionWhisper is a general-purpose speech recognition model. It is trained on a large dataset of diverse audio and is also a multitasking model that can perform multilingual speech recognition, speech translation, and language identification.Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/whisper/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)whisper-large-v3-turboOpenAIAutomatic Speech RecognitionWhisper is a pre-trained model for automatic speech recognition (ASR) and speech translation. Cloudflare-hostedBatch](https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)whisper-tiny-enBetaOpenAIAutomatic Speech RecognitionWhisper is a pre-trained model for automatic speech recognition (ASR) and speech translation. Trained on 680k hours of labelled data, Whisper models demonstrate a strong ability to generalize to many datasets and domains without the need for fine-tuning. This is the English-only version of the Whisper Tiny model which was trained on the task of speech recognition.Cloudflare-hosted](https://developers.cloudflare.com/workers-ai/models/whisper-tiny-en/) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/models/#page","headline":"Workers AI Models · Cloudflare Workers AI docs","description":"Browse the catalog of machine learning models available on Workers AI.","url":"https://developers.cloudflare.com/workers-ai/models/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-12","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Create stateful AI agents with persistent memory, real-time WebSocket connections, and scheduled tasks using the Cloudflare Agents SDK. title: Build Agents on Cloudflare image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/agents/llms.txt > Use this file to discover all available pages before exploring further. # Build Agents on Cloudflare Last updated Jun 24, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/agents/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Build and host Agents on Cloudflare, connect chat, voice, email, Slack, and webhooks to a durable agent runtime with Browser, Sandbox, AI Search, MCP, Payments, and other MCP tools. When you host agents on Cloudflare, each agent session has a durable identity, local SQL storage, real-time connections, scheduled work, and recoverable execution. Deploy once and Cloudflare runs your agents across its global network, scaling to tens of millions of instances. No infrastructure to manage, no sessions to reconstruct, no state to externalize. [Chat](https://developers.cloudflare.com/agents/communication-channels/chat/)[Email](https://developers.cloudflare.com/agents/communication-channels/email/)[Voice](https://developers.cloudflare.com/agents/communication-channels/voice/)[Slack](https://developers.cloudflare.com/agents/communication-channels/slack/)[Webhook](https://developers.cloudflare.com/agents/communication-channels/webhooks/) Agent harness Controls planning, tool use, and response flow. [Project Think](https://developers.cloudflare.com/agents/harnesses/think/)[Build-your-own agent](https://developers.cloudflare.com/agents/runtime/agents-api/) Agents SDK runtime Durable identity, state, connections, scheduling, and recovery. [Agent class](https://developers.cloudflare.com/agents/runtime/agents-api/) [State](https://developers.cloudflare.com/agents/runtime/lifecycle/state/)[Sessions](https://developers.cloudflare.com/agents/runtime/lifecycle/sessions/)[Routing](https://developers.cloudflare.com/agents/runtime/communication/routing/)[WebSockets](https://developers.cloudflare.com/agents/runtime/communication/websockets/)[Scheduling](https://developers.cloudflare.com/agents/runtime/execution/schedule-tasks/)[Fibers](https://developers.cloudflare.com/agents/runtime/execution/durable-execution/) [Sandbox](https://developers.cloudflare.com/agents/tools/sandbox/)[MCP](https://developers.cloudflare.com/agents/tools/mcp/)[Browser](https://developers.cloudflare.com/agents/tools/browser/)[AI Search](https://developers.cloudflare.com/agents/tools/ai-search/)[Payments](https://developers.cloudflare.com/agents/tools/payments/) [ObservabilityLogs · metrics · traces](https://developers.cloudflare.com/agents/runtime/operations/observability/) Agents on Cloudflare are composed from four parts: * **Communication channels** define how users and systems reach your agent, such as [chat](https://developers.cloudflare.com/agents/communication-channels/chat/), [voice](https://developers.cloudflare.com/agents/communication-channels/voice/), [email](https://developers.cloudflare.com/agents/communication-channels/email/), [Slack](https://developers.cloudflare.com/agents/communication-channels/slack/), [webhooks](https://developers.cloudflare.com/agents/communication-channels/webhooks/), and other event sources. * **The agent harness** defines the loop: how the agent calls models, selects tools, handles tool results, streams responses, and decides whether to continue. Use [Project Think](https://developers.cloudflare.com/agents/harnesses/think/) for an opinionated harness, or build your own loop directly on the [Agents SDK runtime](https://developers.cloudflare.com/agents/runtime/agents-api/). * **The Agents SDK runtime** provides durable infrastructure: the [Agent class](https://developers.cloudflare.com/agents/runtime/lifecycle/agent-class/), [state](https://developers.cloudflare.com/agents/runtime/lifecycle/state/), [sessions](https://developers.cloudflare.com/agents/runtime/lifecycle/sessions/), [routing](https://developers.cloudflare.com/agents/runtime/communication/routing/), [WebSockets](https://developers.cloudflare.com/agents/runtime/communication/websockets/), [scheduling](https://developers.cloudflare.com/agents/runtime/execution/schedule-tasks/), [fibers](https://developers.cloudflare.com/agents/runtime/execution/durable-execution/), and [observability](https://developers.cloudflare.com/agents/runtime/operations/observability/). * **Tools** give the agent capabilities: [browser automation](https://developers.cloudflare.com/agents/tools/browser/), [sandboxed code execution](https://developers.cloudflare.com/agents/tools/sandbox/), [AI Search](https://developers.cloudflare.com/agents/tools/ai-search/), [MCP tools](https://developers.cloudflare.com/agents/tools/mcp/), and [payments](https://developers.cloudflare.com/agents/tools/payments/). [Code Mode](https://developers.cloudflare.com/agents/tools/codemode/) lets models discover and orchestrate multiple tools by writing code. ### Get started Three commands to a running agent. No API keys required — the starter uses [Workers AI](https://developers.cloudflare.com/workers-ai/) by default. ```sh npx create-cloudflare@latest --template cloudflare/agents-starter cd agents-starter && npm install npm run dev ``` The starter includes streaming AI chat, server-side and client-side tools, human-in-the-loop approval, and task scheduling — a foundation you can build on or tear apart. You can also swap in [OpenAI, Anthropic, Google Gemini, or any other provider](https://developers.cloudflare.com/agents/runtime/operations/using-ai-models/). ### Example agents [Chat agent](https://developers.cloudflare.com/agents/examples/chat-agent/) Build a streaming AI chat agent with tools and human-in-the-loop approvals. [Slack agent](https://developers.cloudflare.com/agents/examples/slack-agent/) Build an agent that responds to Slack messages, mentions, and commands. [Voice agent](https://developers.cloudflare.com/agents/examples/voice-agent/) Build a real-time voice agent with speech-to-text and text-to-speech. [Browser agent](https://developers.cloudflare.com/agents/examples/browser-agent/) Build an agent that can inspect pages, capture screenshots, and use browser tools. [Email agent](https://developers.cloudflare.com/agents/examples/email-agent/) Build an agent that sends, receives, routes, and replies to email. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/agents/#page","headline":"Agents · Cloudflare Agents docs","description":"Create stateful AI agents with persistent memory, real-time WebSocket connections, and scheduled tasks using the Cloudflare Agents SDK.","url":"https://developers.cloudflare.com/agents/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-24","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- ## Execute AI model **post** `/accounts/{account_id}/ai/run/{model_name}` This endpoint provides users with the capability to run specific AI models on-demand. By submitting the required input data, users can receive real-time predictions or results generated by the chosen AI model. The endpoint supports various AI model types, ensuring flexibility and adaptability for diverse use cases. Model specific inputs available in [Cloudflare Docs](https://developers.cloudflare.com/workers-ai/models/). ### Path Parameters - `account_id: string` - `model_name: string` ### Body Parameters - `body: optional object { text } or object { prompt, guidance, height, 8 more } or object { prompt, lang } or 12 more` - `TextClassification object { text }` - `text: string` The text that you want to classify - `TextToImage object { prompt, guidance, height, 8 more }` - `prompt: string` A text description of the image you want to generate - `guidance: optional number` Controls how closely the generated image should adhere to the prompt; higher values make the image more aligned with the prompt - `height: optional number` The height of the generated image in pixels - `image: optional array of number` For use with img2img tasks. An array of integers that represent the image data constrained to 8-bit unsigned integer values - `image_b64: optional string` For use with img2img tasks. A base64-encoded string of the input image - `mask: optional array of number` An array representing An array of integers that represent mask image data for inpainting constrained to 8-bit unsigned integer values - `negative_prompt: optional string` Text describing elements to avoid in the generated image - `num_steps: optional number` The number of diffusion steps; higher values can improve quality but take longer - `seed: optional number` Random seed for reproducibility of the image generation - `strength: optional number` A value between 0 and 1 indicating how strongly to apply the transformation during img2img tasks; lower values make the output closer to the input image - `width: optional number` The width of the generated image in pixels - `TextToSpeech object { prompt, lang }` - `prompt: string` A text description of the audio you want to generate - `lang: optional string` The speech language (e.g., 'en' for English, 'fr' for French). Defaults to 'en' if not specified - `TextEmbeddings object { text }` - `text: string or array of string` The text to embed - `string` The text to embed - `array of string` Batch of text values to embed - `AutomaticSpeechRecognition object { audio, source_lang, target_lang }` - `audio: array of number` An array of integers that represent the audio data constrained to 8-bit unsigned integer values - `source_lang: optional string` The language of the recorded audio - `target_lang: optional string` The language to translate the transcription into. Currently only English is supported. - `ImageClassification object { image }` - `image: array of number` An array of integers that represent the image data constrained to 8-bit unsigned integer values - `ObjectDetection object { image }` - `image: optional array of number` An array of integers that represent the image data constrained to 8-bit unsigned integer values - `Prompt object { prompt, frequency_penalty, lora, 10 more }` - `prompt: string` The input text prompt for the model to generate a response. - `frequency_penalty: optional number` Decreases the likelihood of the model repeating the same lines verbatim. - `lora: optional string` Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. - `max_tokens: optional number` The maximum number of tokens to generate in the response. - `presence_penalty: optional number` Increases the likelihood of the model introducing new topics. - `raw: optional boolean` If true, a chat template is not applied and you must adhere to the specific model's expected formatting. - `repetition_penalty: optional number` Penalty for repeated tokens; higher values discourage repetition. - `response_format: optional object { json_schema, type }` - `json_schema: optional unknown` - `type: optional "json_object" or "json_schema"` - `"json_object"` - `"json_schema"` - `seed: optional number` Random seed for reproducibility of the generation. - `stream: optional boolean` If true, the response will be streamed back incrementally using SSE, Server Sent Events. - `temperature: optional number` Controls the randomness of the output; higher values produce more random results. - `top_k: optional number` Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. - `top_p: optional number` Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. - `TextGeneration object { messages, frequency_penalty, functions, 11 more }` - `messages: array of object { content, role }` An array of message objects representing the conversation history. - `content: string or array of object { text, type }` The content of the message as a string. - `string` The content of the message as a string. - `array of object { text, type }` Array of text content parts. - `text: optional string` Text content - `type: optional string` Type of the content (text) - `role: string` The role of the message sender (e.g., 'user', 'assistant', 'system', 'tool'). - `frequency_penalty: optional number` Decreases the likelihood of the model repeating the same lines verbatim. - `functions: optional array of object { code, name }` - `code: string` - `name: string` - `max_tokens: optional number` The maximum number of tokens to generate in the response. - `presence_penalty: optional number` Increases the likelihood of the model introducing new topics. - `raw: optional boolean` If true, a chat template is not applied and you must adhere to the specific model's expected formatting. - `repetition_penalty: optional number` Penalty for repeated tokens; higher values discourage repetition. - `response_format: optional object { json_schema, type }` - `json_schema: optional unknown` - `type: optional "json_object" or "json_schema"` - `"json_object"` - `"json_schema"` - `seed: optional number` Random seed for reproducibility of the generation. - `stream: optional boolean` If true, the response will be streamed back incrementally using SSE, Server Sent Events. - `temperature: optional number` Controls the randomness of the output; higher values produce more random results. - `tools: optional array of object { description, name, parameters } or object { function, type }` A list of tools available for the assistant to use. - `object { description, name, parameters }` - `description: string` A brief description of what the tool does. - `name: string` The name of the tool. More descriptive the better. - `parameters: object { properties, type, required }` Schema defining the parameters accepted by the tool. - `properties: map[object { description, type } ]` Definitions of each parameter. - `description: string` A description of the expected parameter. - `type: string` The data type of the parameter. - `type: string` The type of the parameters object (usually 'object'). - `required: optional array of string` List of required parameter names. - `Function object { function, type }` - `function: object { description, name, parameters }` Details of the function tool. - `description: string` A brief description of what the function does. - `name: string` The name of the function. - `parameters: object { properties, type, required }` Schema defining the parameters accepted by the function. - `properties: map[object { description, type } ]` Definitions of each parameter. - `description: string` A description of the expected parameter. - `type: string` The data type of the parameter. - `type: string` The type of the parameters object (usually 'object'). - `required: optional array of string` List of required parameter names. - `type: string` Specifies the type of tool (e.g., 'function'). - `top_k: optional number` Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. - `top_p: optional number` Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. - `Translation object { target_lang, text, source_lang }` - `target_lang: string` The language code to translate the text into (e.g., 'es' for Spanish) - `text: string` The text to be translated - `source_lang: optional string` The language code of the source text (e.g., 'en' for English). Defaults to 'en' if not specified - `Summarization object { input_text, max_length }` - `input_text: string` The text that you want the model to summarize - `max_length: optional number` The maximum length of the generated summary in tokens - `ImageToText object { image, frequency_penalty, max_tokens, 8 more }` - `image: array of number` An array of integers that represent the image data constrained to 8-bit unsigned integer values - `frequency_penalty: optional number` Decreases the likelihood of the model repeating the same lines verbatim. - `max_tokens: optional number` The maximum number of tokens to generate in the response. - `presence_penalty: optional number` Increases the likelihood of the model introducing new topics. - `prompt: optional string` The input text prompt for the model to generate a response. - `raw: optional boolean` If true, a chat template is not applied and you must adhere to the specific model's expected formatting. - `repetition_penalty: optional number` Penalty for repeated tokens; higher values discourage repetition. - `seed: optional number` Random seed for reproducibility of the generation. - `temperature: optional number` Controls the randomness of the output; higher values produce more random results. - `top_k: optional number` Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. - `top_p: optional number` Controls the creativity of the AI's responses by adjusting how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. - `object { image, prompt, frequency_penalty, 8 more }` - `image: string` Image in base64 encoded format. - `prompt: string` The input text prompt for the model to generate a response. - `frequency_penalty: optional number` Decreases the likelihood of the model repeating the same lines verbatim. - `ignore_eos: optional boolean` Whether to ignore the EOS token and continue generating tokens after the EOS token is generated. - `max_tokens: optional number` The maximum number of tokens to generate in the response. - `presence_penalty: optional number` Increases the likelihood of the model introducing new topics. - `repetition_penalty: optional number` Penalty for repeated tokens; higher values discourage repetition. - `seed: optional number` Random seed for reproducibility of the generation. - `temperature: optional number` Controls the randomness of the output; higher values produce more random results. - `top_k: optional number` Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. - `top_p: optional number` Controls the creativity of the AI's responses by adjusting how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. - `ImageTextToText object { image, messages, frequency_penalty, 8 more }` - `image: string` Image in base64 encoded format. - `messages: array of object { content, role }` An array of message objects representing the conversation history. - `content: string or array of object { type, image_url, text }` The content of the message as a string. - `string` The content of the message as a string. - `array of object { type, image_url, text }` Array of content parts (text, image_url, etc.). - `type: string` Type of the content part (e.g. 'text', 'image_url'). - `image_url: optional object { url }` Image URL object (when type is 'image_url'). - `url: string` Image URI with data (e.g. data:image/jpeg;base64,/9j/...). - `text: optional string` Text content (when type is 'text'). - `role: string` The role of the message sender (e.g., 'user', 'assistant', 'system', 'tool'). - `frequency_penalty: optional number` Decreases the likelihood of the model repeating the same lines verbatim. - `ignore_eos: optional boolean` Whether to ignore the EOS token and continue generating tokens after the EOS token is generated. - `max_tokens: optional number` The maximum number of tokens to generate in the response. - `presence_penalty: optional number` Increases the likelihood of the model introducing new topics. - `repetition_penalty: optional number` Penalty for repeated tokens; higher values discourage repetition. - `seed: optional number` Random seed for reproducibility of the generation. - `temperature: optional number` Controls the randomness of the output; higher values produce more random results. - `top_k: optional number` Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. - `top_p: optional number` Controls the creativity of the AI's responses by adjusting how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. - `MultimodalEmbeddings object { image, text }` - `image: optional string` Image in base64 encoded format. - `text: optional array of string` ### Returns - `result: optional array of object { label, score } or string or object { audio } or 12 more` An array of classification results for the input text - `TextClassification = array of object { label, score }` An array of classification results for the input text - `label: optional string` The classification label assigned to the text (e.g., 'POSITIVE' or 'NEGATIVE') - `score: optional number` Confidence score indicating the likelihood that the text belongs to the specified label - `TextToImage = string` The generated image in PNG format - `Audio object { audio }` - `audio: optional string` The generated audio in MP3 format, base64-encoded - `string` The generated audio in MP3 format - `TextEmbeddings object { data, shape }` - `data: optional array of array of number` Embeddings of the requested text values - `shape: optional array of number` - `AutomaticSpeechRecognition object { text, vtt, word_count, words }` - `text: string` The transcription - `vtt: optional string` - `word_count: optional number` - `words: optional array of object { end, start, word }` - `end: optional number` The ending second when the word completes - `start: optional number` The second this word begins in the recording - `word: optional string` - `ImageClassification = array of object { label, score }` - `label: optional string` The predicted category or class for the input image based on analysis - `score: optional number` A confidence value, between 0 and 1, indicating how certain the model is about the predicted label - `ObjectDetection = array of object { box, label, score }` An array of detected objects within the input image - `box: optional object { xmax, xmin, ymax, ymin }` Coordinates defining the bounding box around the detected object - `xmax: optional number` The x-coordinate of the bottom-right corner of the bounding box - `xmin: optional number` The x-coordinate of the top-left corner of the bounding box - `ymax: optional number` The y-coordinate of the bottom-right corner of the bounding box - `ymin: optional number` The y-coordinate of the top-left corner of the bounding box - `label: optional string` The class label or name of the detected object - `score: optional number` Confidence score indicating the likelihood that the detection is correct - `object { response, tool_calls, usage }` - `response: string` The generated text response from the model - `tool_calls: optional array of object { arguments, name }` An array of tool calls requests made during the response generation - `arguments: optional unknown` The arguments passed to be passed to the tool call request - `name: optional string` The name of the tool to be called - `usage: optional object { completion_tokens, prompt_tokens, total_tokens }` Usage statistics for the inference request - `completion_tokens: optional number` Total number of tokens in output - `prompt_tokens: optional number` Total number of tokens in input - `total_tokens: optional number` Total number of input and output tokens - `string` - `Translation object { translated_text }` - `translated_text: optional string` The translated text in the target language - `Summarization object { summary }` - `summary: optional string` The summarized version of the input text - `ImageToText object { description }` - `description: optional string` - `ImageTextToText object { description }` - `description: optional string` - `MultimodalEmbeddings object { data, shape }` - `data: optional array of array of number` - `shape: optional array of number` ### Example ```http curl https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/run/$MODEL_NAME \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` #### Response ```json { "result": [ { "label": "label", "score": 0 } ] } ``` --- --- description: Review recent changes to Cloudflare Workers AI. title: Changelog image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Changelog Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/changelog/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Subscribe to RSS](https://developers.cloudflare.com/workers-ai/changelog/index.xml) ## 2026-06-16 **GLM-5.2 now available on Workers AI** * [@cf/zai-org/glm-5.2](https://developers.cloudflare.com/workers-ai/models/glm-5.2/) is now available on Workers AI. Z.ai's flagship agentic coding model with a 262,144 token context window, function calling, and reasoning support. Read [changelog](https://developers.cloudflare.com/changelog/2026-06-16-glm-5.2-workers-ai/) to get started. ## 2026-06-12 **Moonshot AI Kimi K2.7 Code now available on Workers AI** * [@cf/moonshotai/kimi-k2.7-code](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/) now available on Workers AI. A frontier-scale 1T parameter MoE model optimized for coding, with a 262.1k context window, vision, multi-turn tool calling, and reasoning. Read [changelog](https://developers.cloudflare.com/changelog/post/2026-06-12-kimi-k2-7-code-workers-ai/) to get started. ## 2026-05-08 **Planned model deprecations** * We are refreshing the Workers AI model catalog to make room for newer releases. Please update your apps to remove references to the models listed below before the deprecation date. Refer to the [changelog](https://developers.cloudflare.com/changelog/post/2026-05-08-planned-model-deprecations/) for full details. * We recommend migrating to newer models such as [@cf/zai-org/glm-4.7-flash](https://developers.cloudflare.com/workers-ai/models/glm-4.7-flash/) for fast tool-calling, [@cf/google/gemma-4-26b-a4b-it](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/) for an efficient open model, or [@cf/moonshotai/kimi-k2.6](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/) for a capable tool-calling and vision model. * On May 30, 2026, requests to [@cf/moonshotai/kimi-k2.5](https://developers.cloudflare.com/workers-ai/models/kimi-k2.5/) will be automatically aliased to [@cf/moonshotai/kimi-k2.6](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/), which has a higher price. The deprecation date was extended from May 10, 2026\. Please review the [K2.6 pricing and model capabilities](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/) prior to May 30, 2026. * On May 30, 2026, the following models will be deprecated: * `@cf/moonshotai/kimi-k2.5` \--> `@cf/moonshotai/kimi-k2.6` * `@hf/meta-llama/meta-llama-3-8b-instruct` * `@cf/meta/llama-3-8b-instruct` * `@cf/meta/llama-3-8b-instruct-awq` * `@cf/meta/llama-3.1-8b-instruct` * `@cf/meta/llama-3.1-8b-instruct-awq` * `@cf/meta/llama-3.1-70b-instruct` * `@cf/meta/llama-2-7b-chat-int8` * `@cf/meta/llama-2-7b-chat-fp16` * `@cf/mistral/mistral-7b-instruct-v0.1` * `@hf/mistral/mistral-7b-instruct-v0.2` * `@hf/google/gemma-7b-it` * `@cf/google/gemma-3-12b-it` * `@hf/nousresearch/hermes-2-pro-mistral-7b` * `@cf/microsoft/phi-2` * `@cf/defog/sqlcoder-7b-2` * `@cf/unum/uform-gen2-qwen-500m` * `@cf/facebook/bart-large-cnn` * The `-fast` and `-lora` variants of models will remain active. LoRA models may be deprecated in the future, and we will communicate when new LoRA models come online to give users time to train new LoRAs before we deprecate old ones. ## 2026-04-20 **Moonshot AI Kimi K2.6 now available on Workers AI** * [@cf/moonshotai/kimi-k2.6](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/) now available on Workers AI. The latest frontier-scale model from Moonshot AI with improved reasoning, coding, and agentic capabilities. Read [changelog](https://developers.cloudflare.com/changelog/post/2026-04-20-kimi-k2-6-workers-ai/) to get started. * K2.6 uses the `chat_template_kwargs.thinking` parameter to control reasoning (instead of `chat_template_kwargs.enable_thinking`) and returns reasoning content in the `reasoning` field (instead of `reasoning_content`). ## 2026-04-04 **Google Gemma 4 26B A4B now available on Workers AI** * [@cf/google/gemma-4-26b-a4b-it](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/) now available on Workers AI. A Mixture-of-Experts model with 26B total parameters and 4B active, featuring a 256K context window, vision, built-in thinking mode, and function calling. Read [changelog](https://developers.cloudflare.com/changelog/post/2026-04-04-gemma-4-26b-a4b-workers-ai/) to get started. ## 2026-03-19 **Moonshot AI Kimi K2.5 now available on Workers AI** * [@cf/moonshotai/kimi-k2.5](https://developers.cloudflare.com/workers-ai/models/kimi-k2.5/) now available on Workers AI. A frontier-scale open-source model with a 256k context window, multi-turn tool calling, vision inputs, and structured outputs for agentic workloads. Read [changelog](https://developers.cloudflare.com/changelog/post/2026-03-19-kimi-k2-5-workers-ai/) to get started. * New [Prompt caching](https://developers.cloudflare.com/workers-ai/features/prompt-caching/) documentation. Send the `x-session-affinity` header to route requests to the same model instance and maximize prefix cache hit rates across multi-turn conversations. * Redesigned [Asynchronous Batch API](https://developers.cloudflare.com/workers-ai/features/batch-api/) with a pull-based system that processes queued requests as capacity becomes available, avoiding out-of-capacity errors for durable workflows. ## 2026-03-11 **NVIDIA Nemotron 3 Super now available on Workers AI** * [@cf/nvidia/nemotron-3-120b-a12b](https://developers.cloudflare.com/workers-ai/models/nemotron-3-120b-a12b/) now available on Workers AI! A hybrid MoE model with 120B total parameters and 12B active, optimized for multi-agent and agentic AI workloads. Read [changelog](https://developers.cloudflare.com/changelog/post/2026-03-11-nemotron-3-super-workers-ai/) to get started. ## 2026-03-06 **Deepgram Nova-3 now supports 10 languages with regional variants** * [@cf/deepgram/nova-3](https://developers.cloudflare.com/workers-ai/models/nova-3/) now supports 10 languages with regional variants for real-time transcription. Supported languages include English, Spanish, French, German, Hindi, Russian, Portuguese, Japanese, Italian, and Dutch — with regional variants like `en-GB`, `fr-CA`, and `pt-BR`. ## 2026-02-17 **Chat Completions API support for gpt-oss models and tool calling improvements** * [@cf/openai/gpt-oss-120b](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/) and [@cf/openai/gpt-oss-20b](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/) now support Chat Completions API format. Use `/v1/chat/completions` with a `messages` array, or use `/ai/run` which dynamically detects your input format and accepts Chat Completions (`messages`), legacy Completions (`prompt`), or Responses API (`input`). * **\[Bug fix\]** Fixed a bug in the schema for multiple text generation models where the `content` field in message objects only accepted string values. The field now properly accepts both string content and array content (structured content parts for multi-modal inputs). This fix applies to all affected chat models including GPT-OSS models, Llama 3.x, Mistral, Qwen, and others. * **\[Bug fix\]** Tool call round-trips now work correctly. The binding no longer rejects `tool_call_id` values that it generated itself, fixing issues with multi-turn tool calling conversations. * **\[Bug fix\]** Assistant messages with `content: null` and `tool_calls` are now accepted in both the Workers AI binding and REST API (`/v1/chat/completions`), fixing tool call round-trip failures. * **\[Bug fix\]** Streaming responses now correctly report `finish_reason` only on the usage chunk, matching OpenAI's streaming behavior and preventing duplicate finish events. * **\[Bug fix\]** `/v1/chat/completions` now preserves original tool call IDs from models instead of regenerating them. Previously, the endpoint was generating new IDs which broke multi-turn tool calling because AI SDK clients could not match tool results to their original calls. * **\[Bug fix\]** `/v1/chat/completions` now correctly reports `finish_reason: "tool_calls"` in the final usage chunk when tools are used. Previously, it was hardcoding `finish_reason: "stop"` which caused AI SDK clients to think the conversation was complete instead of executing tool calls. ## 2026-02-13 **GLM-4.7-Flash, @cloudflare/tanstack-ai, and workers-ai-provider v3.1.1** * [@cf/zai-org/glm-4.7-flash](https://developers.cloudflare.com/workers-ai/models/glm-4.7-flash/) is now available on Workers AI! A fast and efficient multilingual text generation model optimized for multi-turn tool calling across 100+ languages. Read [changelog](https://developers.cloudflare.com/changelog/2026-02-13-glm-4.7-flash-workers-ai/) to get started. * New [@cloudflare/tanstack-ai](https://www.npmjs.com/package/@cloudflare/tanstack-ai) package for using Workers AI and AI Gateway with TanStack AI. * [workers-ai-provider v3.1.1](https://www.npmjs.com/package/workers-ai-provider) adds transcription, text-to-speech, and reranking capabilities. ## 2026-01-28 **Black Forest Labs FLUX.2 \[klein\] 9B now available** * [@cf/black-forest-labs/flux-2-klein-9b](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-9b/) now available on Workers AI! Read [changelog](https://developers.cloudflare.com/changelog/2026-01-28-flux-2-klein-9b-workers-ai/) to get started ## 2026-01-15 **Black Forest Labs FLUX.2 \[klein\] 4b now available** * [@cf/black-forest-labs/flux-2-klein-4b](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-4b/) now available on Workers AI! Read [changelog](https://developers.cloudflare.com/changelog/2026-01-15-flux-2-klein-4b-workers-ai/) to get started ## 2025-12-03 **Deepgram Flux promotional period over on Dec 8, 2025 - now has pricing** * Check out updated pricing on the [@cf/deepgram/flux](https://developers.cloudflare.com/workers-ai/models/flux/) model page or [pricing](https://developers.cloudflare.com/workers-ai/platform/pricing/) page * Pricing will start Dec 8, 2025 ## 2025-11-25 **Black Forest Labs FLUX.2 dev now available** * [@cf/black-forest-labs/flux-2-dev](https://developers.cloudflare.com/workers-ai/models/flux-2-dev/) now available on Workers AI! Read [changelog](https://developers.cloudflare.com/changelog/2025-11-25-flux-2-dev-workers-ai/) to get started ## 2025-11-13 **Qwen3 LLM and Embeddings available on Workers AI** * [@cf/qwen/qwen3-30b-a3b-fp8](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/) and [@cf/qwen/qwen3-embedding-0.6b](https://developers.cloudflare.com/workers-ai/models/qwen3-embedding-0.6b) now available on Workers AI ## 2025-10-21 **New voice and LLM models on Workers AI** * Deepgram Aura 2 brings new text-to-speech capabilities to Workers AI. Check out [@cf/deepgram/aura-2-en](https://developers.cloudflare.com/workers-ai/models/aura-2-en/) and [@cf/deepgram/aura-2-es](https://developers.cloudflare.com/workers-ai/models/aura-2-es/) on how to use the new models. * IBM Granite model is also up! This new LLM model is small but mighty, take a look at the docs for more [@cf/ibm-granite/granite-4.0-h-micro](https://developers.cloudflare.com/workers-ai/models/granite-4.0-h-micro/) ## 2025-10-02 **Deepgram Flux now available on Workers AI** * We're excited to be a launch partner with Deepgram and offer their new Speech Recognition model built specifically for enabling voice agents. Check out [Deepgram's blog](https://deepgram.com/flux) for more details on the release. * Access the model through [@cf/deepgram/flux](https://developers.cloudflare.com/workers-ai/models/flux/) and check out the [changelog](https://developers.cloudflare.com/changelog/2025-10-02-deepgram-flux/) for in-depth examples. ## 2025-09-24 **New local models available on Workers AI** * We've added support for some regional models on Workers AI in support of uplifting local AI labs and AI sovereignty. Check out the [full blog post here](https://blog.cloudflare.com/sovereign-ai-and-choice). * [@cf/pfnet/plamo-embedding-1b](https://developers.cloudflare.com/workers-ai/models/plamo-embedding-1b) creates embeddings from Japanese text. * [@cf/aisingapore/gemma-sea-lion-v4-27b-it](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it) is a fine-tuned model that supports multiple South East Asian languages, including Burmese, English, Indonesian, Khmer, Lao, Malay, Mandarin, Tagalog, Tamil, Thai, and Vietnamese. * [@cf/ai4bharat/indictrans2-en-indic-1B](https://developers.cloudflare.com/workers-ai/models/indictrans2-en-indic-1B) is a translation model that can translate between 22 indic languages, including Bengali, Gujarati, Hindi, Tamil, Sanskrit and even traditionally low-resourced languages like Kashmiri, Manipuri and Sindhi. ## 2025-09-23 **New document formats supported by Markdown conversion utility** * Our [Markdown conversion utility](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/) now supports converting `.docx` and `.odt` files. ## 2025-09-18 **Model Catalog updates (types, EmbeddingGemma, model deprecation)** * Workers AI types got updated in the upcoming wrangler release, please use `npm i -D wrangler@latest` to update your packages. * EmbeddingGemma model accuracy has been improved, we recommend re-indexing data to take advantage of the improved accuracy * Some older Workers AI models are being deprecated on October 1st, 2025\. We reccommend you use the newer models such as [Llama 4](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/) and [gpt-oss](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/). The following models are being deprecated: * @hf/thebloke/zephyr-7b-beta-awq * @hf/thebloke/mistral-7b-instruct-v0.1-awq * @hf/thebloke/llama-2-13b-chat-awq * @hf/thebloke/openhermes-2.5-mistral-7b-awq * @hf/thebloke/neural-chat-7b-v3-1-awq * @hf/thebloke/llamaguard-7b-awq * @hf/thebloke/deepseek-coder-6.7b-base-awq * @hf/thebloke/deepseek-coder-6.7b-instruct-awq * @cf/deepseek-ai/deepseek-math-7b-instruct * @cf/openchat/openchat-3.5-0106 * @cf/tiiuae/falcon-7b-instruct * @cf/thebloke/discolm-german-7b-v1-awq * @cf/qwen/qwen1.5-0.5b-chat * @cf/qwen/qwen1.5-7b-chat-awq * @cf/qwen/qwen1.5-14b-chat-awq * @cf/tinyllama/tinyllama-1.1b-chat-v1.0 * @cf/qwen/qwen1.5-1.8b-chat * @hf/nexusflow/starling-lm-7b-beta * @cf/fblgit/una-cybertron-7b-v2-bf16 ## 2025-09-05 **Introducing EmbeddingGemma from Google** * We’re excited to be a launch partner alongside Google to bring their newest embedding model to Workers AI. We're excited to introduce EmbeddingGemma delivers best-in-class performance for its size, enabling RAG and semantic search use cases. Take a look at [@cf/google/embeddinggemma-300m](https://developers.cloudflare.com/workers-ai/models/embeddinggemma-300m) for more details. Now available to use for embedding in AI Search too. ## 2025-08-27 **Introducing Partner models to the Workers AI catalog** * Read the [blog](https://blog.cloudflare.com/workers-ai-partner-models) for more details * [@cf/deepgram/aura-1](https://developers.cloudflare.com/workers-ai/models/aura-1) is a text-to-speech model that allows you to input text and have it come to life in a customizable voice * [@cf/deepgram/nova-3](https://developers.cloudflare.com/workers-ai/models/nova-3) is speech-to-text model that transcribes multilingual audio at a blazingly fast speed * [@cf/pipecat-ai/smart-turn-v2](https://developers.cloudflare.com/workers-ai/models/smart-turn-v2) helps you detect when someone is done speaking * [@cf/leonardo/lucid-origin](https://developers.cloudflare.com/workers-ai/models/lucid-origin) is a text-to-image model that generates images with sharp graphic design, stunning full-HD renders, or highly specific creative direction * [@cf/leonardo/phoenix-1.0](https://developers.cloudflare.com/workers-ai/models/phoenix-1.0) is a text-to-image model with exceptional prompt adherence and coherent text * WebSocket support added for audio models like `@cf/deepgram/aura-1`, `@cf/deepgram/nova-3`, `@cf/pipecat-ai/smart-turn-v2` ## 2025-08-05 **Adding gpt-oss models to our catalog** * Check out the [blog](https://blog.cloudflare.com/openai-gpt-oss-on-workers-ai) for more details about the new models * Take a look at the [gpt-oss-120b](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b) and [gpt-oss-20b](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b) model pages for more information about schemas, pricing, and context windows ## 2025-04-09 **Pricing correction for @cf/myshell-ai/melotts** * We've updated our documentation to reflect the correct pricing for melotts: $0.0002 per audio minute, which is actually cheaper than initially stated. The documented pricing was incorrect, where it said users would be charged based on input tokens. ## 2025-03-17 **Minor updates to the model schema for llama-3.2-1b-instruct, whisper-large-v3-turbo, llama-guard** * [llama-3.2-1b-instruct](https://developers.cloudflare.com/workers-ai/models/llama-3.2-1b-instruct/) \- updated context window to the accurate 60,000 * [whisper-large-v3-turbo](https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/) \- new hyperparameters available * [llama-guard-3-8b](https://developers.cloudflare.com/workers-ai/models/llama-guard-3-8b/) \- the messages array must alternate between `user` and `assistant` to function correctly ## 2025-02-21 **Workers AI bug fixes** * We fixed a bug where `max_tokens` defaults were not properly being respected - `max_tokens` now correctly defaults to `256` as displayed on the model pages. Users relying on the previous behaviour may observe this as a breaking change. If you want to generate more tokens, please set the `max_tokens` parameter to what you need. * We updated model pages to show context windows - which is defined as the tokens used in the prompt + tokens used in the response. If your prompt + response tokens exceed the context window, the request will error. Please set `max_tokens` accordingly depending on your prompt length and the context window length to ensure a successful response. ## 2024-09-26 **Workers AI Birthday Week 2024 announcements** * Meta Llama 3.2 1B, 3B, and 11B vision is now available on Workers AI * `@cf/black-forest-labs/flux-1-schnell` is now available on Workers AI * Workers AI is fast! Powered by new GPUs and optimizations, you can expect faster inference on Llama 3.1, Llama 3.2, and FLUX models. * No more neurons. Workers AI is moving towards [unit-based pricing](https://developers.cloudflare.com/workers-ai/platform/pricing) * Model pages get a refresh with better documentation on parameters, pricing, and model capabilities * Closed beta for our Run Any\* Model feature, [sign up here](https://forms.gle/h7FcaTF4Zo5dzNb68) * Check out the [product announcements blog post](https://blog.cloudflare.com/workers-ai) for more information * And the [technical blog post](https://blog.cloudflare.com/workers-ai/making-workers-ai-faster) if you want to learn about how we made Workers AI fast ## 2024-07-23 **Meta Llama 3.1 now available on Workers AI** Workers AI now suppoorts [Meta Llama 3.1](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct/). ## 2024-06-27 **Introducing embedded function calling** * A new way to do function calling with [Embedded function calling](https://developers.cloudflare.com/workers-ai/function-calling/embedded) * Published new [@cloudflare/ai-utils](https://www.npmjs.com/package/@cloudflare/ai-utils) npm package * Open-sourced [ai-utils on Github](https://github.com/cloudflare/ai-utils) ## 2024-06-19 **Added support for traditional function calling** * [Function calling](https://developers.cloudflare.com/workers-ai/function-calling/) is now supported on enabled models * Properties added on [models](https://developers.cloudflare.com/workers-ai/models/) page to show which models support function calling ## 2024-06-18 **Native support for AI Gateways** Workers AI now natively supports [AI Gateway](https://developers.cloudflare.com/ai-gateway/usage/providers/workersai/#worker). ## 2024-06-11 **Deprecation announcement for \`@cf/meta/llama-2-7b-chat-int8\`** We will be deprecating `@cf/meta/llama-2-7b-chat-int8` on 2024-06-30. Replace the model ID in your code with a new model of your choice: * [@cf/meta/llama-3-8b-instruct](https://developers.cloudflare.com/workers-ai/models/llama-3-8b-instruct/) is the newest model in the Llama family (and is currently free for a limited time on Workers AI). * [@cf/meta/llama-3-8b-instruct-awq](https://developers.cloudflare.com/workers-ai/models/llama-3-8b-instruct-awq/) is the new Llama 3 in a similar precision to your currently selected model. This model is also currently free for a limited time. If you do not switch to a different model by June 30th, we will automatically start returning inference from `@cf/meta/llama-3-8b-instruct-awq`. ## 2024-05-29 **Add new public LoRAs and note on LoRA routing** * Added documentation on [new public LoRAs](https://developers.cloudflare.com/workers-ai/fine-tunes/public-loras/). * Noted that you can now run LoRA inference with the base model rather than explicitly calling the `-lora` version ## 2024-05-17 **Add OpenAI compatible API endpoints** Added OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/). ## 2024-04-11 **Add AI native binding** * Added new AI native binding, you can now run models with `const resp = await env.AI.run(modelName, inputs)` * Deprecated `@cloudflare/ai` npm package. While existing solutions using the @cloudflare/ai package will continue to work, no new Workers AI features will be supported. Moving to native AI bindings is highly recommended Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"BlogPosting","@id":"https://developers.cloudflare.com/workers-ai/changelog/#page","headline":"Changelog · Cloudflare Workers AI docs","description":"Review recent changes to Cloudflare Workers AI.","url":"https://developers.cloudflare.com/workers-ai/changelog/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Use Workers AI with the Vercel AI SDK for streaming text generation, tool calls, and structured output. title: Vercel AI SDK image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Vercel AI SDK Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/configuration/ai-sdk/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Workers AI can be used with the [Vercel AI SDK ↗](https://sdk.vercel.ai/) for JavaScript and TypeScript codebases. ## Setup Install the [workers-ai-provider provider ↗](https://sdk.vercel.ai/providers/community-providers/cloudflare-workers-ai): npmyarnpnpmbun ``` npm i workers-ai-provider ``` ``` yarn add workers-ai-provider ``` ``` pnpm add workers-ai-provider ``` ``` bun add workers-ai-provider ``` Then, add an AI binding in your Workers project Wrangler file: ```toml [ai] binding = "AI" ``` ## Models The AI SDK can be configured to work with [any AI model](https://developers.cloudflare.com/workers-ai/models/). ```js import { createWorkersAI } from "workers-ai-provider"; const workersai = createWorkersAI({ binding: env.AI }); // Choose any model: https://developers.cloudflare.com/workers-ai/models/ const model = workersai("@cf/meta/llama-3.1-8b-instruct", {}); ``` ## Generate Text Once you have selected your model, you can generate text from a given prompt. ```js import { createWorkersAI } from 'workers-ai-provider'; import { generateText } from 'ai'; type Env = { AI: Ai; }; export default { async fetch(_: Request, env: Env) { const workersai = createWorkersAI({ binding: env.AI }); const result = await generateText({ model: workersai('@cf/meta/llama-2-7b-chat-int8'), prompt: 'Write a 50-word essay about hello world.', }); return new Response(result.text); }, }; ``` ## Stream Text For longer responses, consider streaming responses to provide as the generation completes. ```js import { createWorkersAI } from 'workers-ai-provider'; import { streamText } from 'ai'; type Env = { AI: Ai; }; export default { async fetch(_: Request, env: Env) { const workersai = createWorkersAI({ binding: env.AI }); const result = streamText({ model: workersai('@cf/meta/llama-2-7b-chat-int8'), prompt: 'Write a 50-word essay about hello world.', }); return result.toTextStreamResponse({ headers: { // add these headers to ensure that the // response is chunked and streamed 'Content-Type': 'text/x-unknown', 'content-encoding': 'identity', 'transfer-encoding': 'chunked', }, }); }, }; ``` ## Generate Structured Objects You can provide a Zod schema to generate a structured JSON response. ```js import { createWorkersAI } from 'workers-ai-provider'; import { generateObject } from 'ai'; import { z } from 'zod'; type Env = { AI: Ai; }; export default { async fetch(_: Request, env: Env) { const workersai = createWorkersAI({ binding: env.AI }); const result = await generateObject({ model: workersai('@cf/meta/llama-3.1-8b-instruct'), prompt: 'Generate a Lasagna recipe', schema: z.object({ recipe: z.object({ ingredients: z.array(z.string()), description: z.string(), }), }), }); return Response.json(result.object); }, }; ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/configuration/ai-sdk/#page","headline":"Vercel AI SDK · Cloudflare Workers AI docs","description":"Use Workers AI with the Vercel AI SDK for streaming text generation, tool calls, and structured output.","url":"https://developers.cloudflare.com/workers-ai/configuration/ai-sdk/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Create an AI binding to connect your Cloudflare Worker to Workers AI. title: Workers Bindings image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Workers Bindings Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/configuration/bindings/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) ## Workers [Workers](https://developers.cloudflare.com/workers/) provides a serverless execution environment that allows you to create new applications or augment existing ones. To use Workers AI with Workers, you must create a Workers AI [binding](https://developers.cloudflare.com/workers/runtime-apis/bindings/). Bindings allow your Workers to interact with resources, like Workers AI, on the Cloudflare Developer Platform. You create bindings on the Cloudflare dashboard or by updating your [Wrangler file](https://developers.cloudflare.com/workers/wrangler/configuration/). To bind Workers AI to your Worker, add the following to the end of your Wrangler file: ```jsonc { "ai": { "binding": "AI" // i.e. available in your Worker on env.AI } } ``` ```toml [ai] binding = "AI" ``` ## Pages Functions [Pages Functions](https://developers.cloudflare.com/pages/functions/) allow you to build full-stack applications with Cloudflare Pages by executing code on the Cloudflare network. Functions are Workers under the hood. To configure a Workers AI binding in your Pages Function, you must use the Cloudflare dashboard. Refer to [Workers AI bindings](https://developers.cloudflare.com/pages/functions/bindings/#workers-ai) for instructions. ## Methods ### async env.AI.run() `async env.AI.run()` runs a model. Takes a model as the first parameter, and an object as the second parameter. ```javascript const answer = await env.AI.run('@cf/meta/llama-3.1-8b-instruct', { prompt: "What is the origin of the phrase 'Hello, World'" }); ``` **Parameters** * `model` `string`required * The model to run. **Supported options** * `stream` `boolean`optional * Returns a stream of results as they are available. ```javascript const answer = await env.AI.run('@cf/meta/llama-3.1-8b-instruct', { prompt: "What is the origin of the phrase 'Hello, World'", stream: true }); return new Response(answer, { headers: { "content-type": "text/event-stream" } }); ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/configuration/bindings/#page","headline":"Workers Bindings · Cloudflare Workers AI docs","description":"Create an AI binding to connect your Cloudflare Worker to Workers AI.","url":"https://developers.cloudflare.com/workers-ai/configuration/bindings/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Connect Workers AI models to Hugging Face's open-source Chat UI interface. title: Hugging Face Chat UI image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Hugging Face Chat UI Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/configuration/hugging-face-chat-ui/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Use Workers AI with [Chat UI ↗](https://github.com/huggingface/chat-ui?tab=readme-ov-file#text-embedding-models), an open-source chat interface offered by Hugging Face. ## Prerequisites You will need the following: * A [Cloudflare account ↗](https://dash.cloudflare.com) * Your [Account ID](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/) * An [API token](https://developers.cloudflare.com/workers-ai/get-started/rest-api/#1-get-api-token-and-account-id) for Workers AI ## Setup First, decide how to reference your Account ID and API token (either directly in your `.env.local` using the `CLOUDFLARE_ACCOUNT_ID` and `CLOUDFLARE_API_TOKEN` variables or in the endpoint configuration). Then, follow the rest of the setup instructions in the [Chat UI GitHub repository ↗](https://github.com/huggingface/chat-ui?tab=readme-ov-file#text-embedding-models). When setting up your models, specify the `cloudflare` endpoint. ```json { "name" : "nousresearch/hermes-2-pro-mistral-7b", "tokenizer": "nousresearch/hermes-2-pro-mistral-7b", "parameters": { "stop": ["<|im_end|>"] }, "endpoints" : [ { "type": "cloudflare", // optionally specify these if not included in .env.local "accountId": "your-account-id", "apiToken": "your-api-token" // } ] } ``` ## Supported models This template works with any [text generation models](https://developers.cloudflare.com/workers-ai/models/) that begin with the `@hf` parameter. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/configuration/hugging-face-chat-ui/#page","headline":"Hugging Face Chat UI · Cloudflare Workers AI docs","description":"Connect Workers AI models to Hugging Face's open-source Chat UI interface.","url":"https://developers.cloudflare.com/workers-ai/configuration/hugging-face-chat-ui/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Use the OpenAI SDK to call Workers AI models through compatible API endpoints. title: OpenAI compatible API endpoints image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # OpenAI compatible API endpoints Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Workers AI supports OpenAI compatible endpoints for [text generation](https://developers.cloudflare.com/workers-ai/models/) (`/v1/chat/completions`) and [text embedding models](https://developers.cloudflare.com/workers-ai/models/) (`/v1/embeddings`). This allows you to use the same code as you would for your OpenAI commands, but swap in Workers AI easily. ## Usage ### Workers AI Normally, Workers AI requires you to specify the model name in the cURL endpoint or within the `env.AI.run` function. With OpenAI compatible endpoints, you can leverage the [openai-node sdk ↗](https://github.com/openai/openai-node) to make calls to Workers AI. This allows you to use Workers AI by simply changing the base URL and the model name. ```js import OpenAI from "openai"; const openai = new OpenAI({ apiKey: env.CLOUDFLARE_API_KEY, baseURL: `https://api.cloudflare.com/client/v4/accounts/${env.CLOUDFLARE_ACCOUNT_ID}/ai/v1`, }); // Use chat completions const chatCompletion = await openai.chat.completions.create({ messages: [{ role: "user", content: "Make some robot noises" }], model: "@cf/meta/llama-3.1-8b-instruct", }); // Use responses const response = await openai.responses.create({ model: "@cf/openai/gpt-oss-120b", input: "Talk to me about open source", }); const embeddings = await openai.embeddings.create({ model: "@cf/baai/bge-large-en-v1.5", input: "I love matcha", }); ``` ```bash curl --request POST \ --url https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions \ --header "Authorization: Bearer {api_token}" \ --header "Content-Type: application/json" \ --data ' { "model": "@cf/meta/llama-3.1-8b-instruct", "messages": [ { "role": "user", "content": "how to build a wooden spoon in 3 short steps? give as short as answer as possible" } ] } ' ``` ### AI Gateway These endpoints are also compatible with [AI Gateway](https://developers.cloudflare.com/ai-gateway/usage/providers/workersai/#openai-compatible-endpoints). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/#page","headline":"OpenAI compatible API endpoints · Cloudflare Workers AI docs","description":"Use the OpenAI SDK to call Workers AI models through compatible API endpoints.","url":"https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Queue large inference workloads for asynchronous processing with the Workers AI Batch API. title: Asynchronous Batch API image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Asynchronous Batch API Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/batch-api/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Asynchronous batch processing lets you send a collection (batch) of inference requests in a single call. Instead of expecting immediate responses for every request, the system queues them for processing and returns the results later. Batch processing is useful for large workloads such as summarization or embeddings when there is no human interaction. Using the batch API will guarantee that your requests are fulfilled eventually, rather than erroring out if Cloudflare does not have enough capacity at a given time. When you send a batch request, the API immediately acknowledges receipt with a status like `queued` and provides a unique `request_id`. This ID is later used to poll for the final responses once the processing is complete. You can use the Batch API by either creating and deploying a Cloudflare Worker that leverages the [Batch API with the AI binding](https://developers.cloudflare.com/workers-ai/features/batch-api/workers-binding/), using the [REST API](https://developers.cloudflare.com/workers-ai/features/batch-api/rest-api/) directly or by starting from a [template ↗](https://github.com/craigsdennis/batch-please-workers-ai). Note Ensure that the total payload is under 10 MB. ## Demo application If you want to get started quickly, click the button below: [![Deploy to Workers](https://deploy.workers.cloudflare.com/button)](https://deploy.workers.cloudflare.com/?url=https://github.com/craigsdennis/batch-please-workers-ai) This will create a repository in your GitHub account and deploy a ready-to-use Worker that demonstrates how to use Cloudflare's Asynchronous Batch API. The template includes preconfigured AI bindings, and examples for sending and retrieving batch requests with and without external references. Once deployed, you can visit the live Worker and start experimenting with the Batch API immediately. ## Supported Models Refer to our [model catalog](https://developers.cloudflare.com/workers-ai/models/?capabilities=Batch) for supported models. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/batch-api/#page","headline":"Asynchronous Batch API · Cloudflare Workers AI docs","description":"Queue large inference workloads for asynchronous processing with the Workers AI Batch API.","url":"https://developers.cloudflare.com/workers-ai/features/batch-api/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Send and retrieve batch inference requests using the Workers AI REST API. title: REST API image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # REST API Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/batch-api/rest-api/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) If you prefer to work directly with the REST API instead of a [Cloudflare Worker](https://developers.cloudflare.com/workers-ai/features/batch-api/workers-binding/), below are the steps on how to do it: ## 1\. Sending a Batch Request Make a POST request using the following pattern. You can pass `external_reference` as a unique ID per-request that will be returned in the response. ```bash curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/run/@cf/baai/bge-m3?queueRequest=true" \ --header "Authorization: Bearer $API_TOKEN" \ --header 'Content-Type: application/json' \ --json '{ "requests": [ { "query": "This is a story about Cloudflare", "contexts": [ { "text": "This is a story about an orange cloud" }, { "text": "This is a story about a llama" }, { "text": "This is a story about a hugging emoji" } ], "external_reference": "reference-1" } ] }' ``` ```json { "result": { "status": "queued", "request_id": "768f15b7-4fd6-4498-906e-ad94ffc7f8d2", "model": "@cf/baai/bge-m3" }, "success": true, "errors": [], "messages": [] } ``` ## 2\. Retrieving the Batch Response After receiving a `request_id` from your initial POST, you can poll for or retrieve the results with another POST request: ```bash curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/run/@cf/baai/bge-m3?queueRequest=true" \ --header "Authorization: Bearer $API_TOKEN" \ --header 'Content-Type: application/json' \ --json '{ "request_id": "" }' ``` ```json { "result": { "responses": [ { "id": 0, "result": { "response": [ { "id": 0, "score": 0.73974609375 }, { "id": 1, "score": 0.642578125 }, { "id": 2, "score": 0.6220703125 } ] }, "success": true, "external_reference": "reference-1" } ], "usage": { "prompt_tokens": 12, "completion_tokens": 0, "total_tokens": 12 } }, "success": true, "errors": [], "messages": [] } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/batch-api/rest-api/#page","headline":"REST API · Cloudflare Workers AI docs","description":"Send and retrieve batch inference requests using the Workers AI REST API.","url":"https://developers.cloudflare.com/workers-ai/features/batch-api/rest-api/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Send and retrieve batch inference requests using a Workers AI binding. title: Workers Binding image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Workers Binding Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/batch-api/workers-binding/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) You can use Workers Bindings to interact with the Batch API. ## Send a Batch request Send your initial batch inference request by composing a JSON payload containing an array of individual inference requests and the `queueRequest: true` property (which is what controls queueing behavior). Note Ensure that the total payload is under 10 MB. ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const embeddings = await env.AI.run( "@cf/baai/bge-m3", { requests: [ { query: "This is a story about Cloudflare", contexts: [ { text: "This is a story about an orange cloud", }, { text: "This is a story about a llama", }, { text: "This is a story about a hugging emoji", }, ], }, ], }, { queueRequest: true }, ); return Response.json(embeddings); }, } satisfies ExportedHandler; ``` ```json { "status": "queued", "model": "@cf/baai/bge-m3", "request_id": "000-000-000" } ``` You will get a response with the following values: * **`status`**: Indicates that your request is queued. * **`request_id`**: A unique identifier for the batch request. * **`model`**: The model used for the batch inference. Of these, the `request_id` is important for when you need to [poll the batch status](#poll-batch-status). ### Poll batch status Once your batch request is queued, use the `request_id` to poll for its status. During processing, the API returns a status `queued` or `running` indicating that the request is still in the queue or being processed. ```typescript export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const status = await env.AI.run("@cf/baai/bge-m3", { request_id: "000-000-000", }); return Response.json(status); }, } satisfies ExportedHandler; ``` ```json { "responses": [ { "id": 0, "result": { "response": [ { "id": 0, "score": 0.73974609375 }, { "id": 1, "score": 0.642578125 }, { "id": 2, "score": 0.6220703125 } ] }, "success": true, "external_reference": "reference-1" } ], "usage": { "prompt_tokens": 12, "completion_tokens": 0, "total_tokens": 12 } } ``` When the inference is complete, the API returns a final HTTP status code of `200` along with an array of responses. Each response object corresponds to an individual input prompt, identified by an `id` that maps to the index of the prompt in your original request. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/batch-api/workers-binding/#page","headline":"Workers Binding · Cloudflare Workers AI docs","description":"Send and retrieve batch inference requests using a Workers AI binding.","url":"https://developers.cloudflare.com/workers-ai/features/batch-api/workers-binding/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Run fine-tuned inference on Workers AI using LoRA adapters. title: Fine-tunes image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Fine-tunes Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/fine-tunes/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Learn how to use Workers AI to get fine-tuned inference. [Fine-tuned inference with LoRAs](https://developers.cloudflare.com/workers-ai/features/fine-tunes/loras/) Upload a LoRA adapter and run fine-tuned inference with one of our base models. Run inference with LoRAs --- ## What is fine-tuning? Fine-tuning is a general term for modifying an AI model by continuing to train it with additional data. The goal of fine-tuning is to increase the probability that a generation is similar to your dataset. Training a model from scratch is not practical for many use cases given how expensive and time consuming they can be to train. By fine-tuning an existing pre-trained model, you benefit from its capabilities while also accomplishing your desired task. [Low-Rank Adaptation ↗](https://arxiv.org/abs/2106.09685) (LoRA) is a specific fine-tuning method that can be applied to various model architectures, not just LLMs. It is common that the pre-trained model weights are directly modified or fused with additional fine-tune weights in traditional fine-tuning methods. LoRA, on the other hand, allows for the fine-tune weights and pre-trained model to remain separate, and for the pre-trained model to remain unchanged. The end result is that you can train models to be more accurate at specific tasks, such as generating code, having a specific personality, or generating images in a specific style. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/features/fine-tunes/#page","headline":"Fine-tunes · Cloudflare Workers AI docs","description":"Run fine-tuned inference on Workers AI using LoRA adapters.","url":"https://developers.cloudflare.com/workers-ai/features/fine-tunes/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Upload and use LoRA adapters to get fine-tuned inference on Workers AI. title: Using LoRA adapters image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Using LoRA adapters Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/fine-tunes/loras/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Workers AI supports fine-tuned inference with adapters trained with [Low-Rank Adaptation ↗](https://blog.cloudflare.com/fine-tuned-inference-with-loras). This feature is in open beta and free during this period. ## Limitations * We only support LoRAs for a [variety of models](https://developers.cloudflare.com/workers-ai/models/?capabilities=LoRA) (must not be quantized) * Adapter must be trained with rank `r <=8` as well as larger ranks if up to 32\. You can check the rank of a pre-trained LoRA adapter through the adapter's `config.json` file * LoRA adapter file must be < 300MB * LoRA adapter files must be named `adapter_config.json` and `adapter_model.safetensors` exactly * You can test up to 100 LoRA adapters per account --- ## Choosing compatible LoRA adapters ### Finding open-source LoRA adapters We have started a [Hugging Face Collection ↗](https://huggingface.co/collections/Cloudflare/workers-ai-compatible-loras-6608dd9f8d305a46e355746e) that lists a few LoRA adapters that are compatible with Workers AI. Generally, any LoRA adapter that fits our limitations above should work. ### Training your own LoRA adapters To train your own LoRA adapter, follow the [tutorial](https://developers.cloudflare.com/workers-ai/guides/tutorials/fine-tune-models-with-autotrain/). --- ## Uploading LoRA adapters In order to run inference with LoRAs on Workers AI, you'll need to create a new fine tune on your account and upload your adapter files. You should have a `adapter_model.safetensors` file with model weights and `adapter_config.json` with your config information. _Note that we only accept adapter files in these types._ Right now, you can't edit a fine tune's asset files after you upload it. We will support this soon, but for now you will need to create a new fine tune and upload files again if you would like to use a new LoRA. Before you upload your LoRA adapter, you'll need to edit your `adapter_config.json` file to include `model_type` as one of `mistral`, `gemma` or `llama` like below. ```json { "alpha_pattern": {}, "auto_mapping": null, ... "target_modules": [ "q_proj", "v_proj" ], "task_type": "CAUSAL_LM", "model_type": "mistral", } ``` ### Wrangler You can create a finetune and upload your LoRA adapter via wrangler with the following commands: ```bash npx wrangler ai finetune create #🌀 Creating new finetune "test-lora" for model "@cf/mistral/mistral-7b-instruct-v0.2-lora"... #🌀 Uploading file "/Users/abcd/Downloads/adapter_config.json" to "test-lora"... #🌀 Uploading file "/Users/abcd/Downloads/adapter_model.safetensors" to "test-lora"... #✅ Assets uploaded, finetune "test-lora" is ready to use. npx wrangler ai finetune list ┌──────────────────────────────────────┬─────────────────┬─────────────┐ │ finetune_id │ name │ description │ ├──────────────────────────────────────┼─────────────────┼─────────────┤ │ 00000000-0000-0000-0000-000000000000 │ test-lora │ │ └──────────────────────────────────────┴─────────────────┴─────────────┘ ``` ### REST API Alternatively, you can use our REST API to create a finetune and upload your adapter files. You will need a Cloudflare API Token with `Workers AI: Edit` permissions to make calls to our REST API, which you can generate via the Cloudflare Dashboard. #### Creating a fine-tune on your account Required API token permissions At least one of the following [token permissions](https://developers.cloudflare.com/fundamentals/api/reference/permissions/) is required: * `Workers AI Write` ```bash curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/finetunes" \ --request POST \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --json '{ "model": "SUPPORTED_MODEL_NAME", "name": "FINETUNE_NAME", "description": "OPTIONAL_DESCRIPTION" }' ``` #### Uploading your adapter weights and config You have to call the upload endpoint each time you want to upload a new file, so you usually run this once for `adapter_model.safetensors` and once for `adapter_config.json`. Make sure you include the `@` before your path to files. You can either use the finetune `name` or `id` that you used when you created the fine tune. ```bash ## Input: finetune_id, adapter_model.safetensors, then adapter_config.json ## Output: success true/false curl -X POST https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/finetunes/{FINETUNE_ID}/finetune-assets/ \ -H 'Authorization: Bearer {API_TOKEN}' \ -H 'Content-Type: multipart/form-data' \ -F 'file_name=adapter_model.safetensors' \ -F 'file=@{PATH/TO/adapter_model.safetensors}' ``` #### List fine-tunes in your account You can call this method to confirm what fine-tunes you have created in your account Required API token permissions At least one of the following [token permissions](https://developers.cloudflare.com/fundamentals/api/reference/permissions/) is required: * `Workers AI Write` * `Workers AI Read` ```bash curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/finetunes" \ --request GET \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` ```json { "success": true, "result": [ [ { "id": "00000000-0000-0000-0000-000000000", "model": "@cf/meta-llama/llama-2-7b-chat-hf-lora", "name": "llama2-finetune", "description": "test" }, { "id": "00000000-0000-0000-0000-000000000", "model": "@cf/mistralai/mistral-7b-instruct-v0.2-lora", "name": "mistral-finetune", "description": "test" } ] ] } ``` --- ## Running inference with LoRAs To make inference requests and apply the LoRA adapter, you will need your model and finetune `name` or `id`. You should use the chat template that your LoRA was trained on, but you can try running it with `raw: true` and the messages template like below. ```javascript const response = await env.AI.run( "@cf/mistralai/mistral-7b-instruct-v0.2-lora", //the model supporting LoRAs { messages: [{ role: "user", content: "Hello world" }], raw: true, //skip applying the default chat template lora: "00000000-0000-0000-0000-000000000", //the finetune id OR name }, ); ``` ```bash curl https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/mistral/mistral-7b-instruct-v0.2-lora \ -H 'Authorization: Bearer {API_TOKEN}' \ -d '{ "messages": [{"role": "user", "content": "Hello world"}], "raw": "true", "lora": "00000000-0000-0000-0000-000000000" }' ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/fine-tunes/loras/#page","headline":"Fine-tuned inference with LoRA adapters · Cloudflare Workers AI docs","description":"Upload and use LoRA adapters to get fine-tuned inference on Workers AI.","url":"https://developers.cloudflare.com/workers-ai/features/fine-tunes/loras/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Cloudflare offers a few public LoRA adapters that are immediately ready for use. title: Public LoRA adapters image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Public LoRA adapters Last updated Jun 5, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/fine-tunes/public-loras/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Cloudflare offers a few public LoRA adapters that can immediately be used for fine-tuned inference. You can try them out immediately via our [playground ↗](https://playground.ai.cloudflare.com). Public LoRAs will have the name `cf-public-x`, and the prefix will be reserved for Cloudflare. Note Have more LoRAs you would like to see? Let us know on [Discord ↗](https://discord.cloudflare.com). | Name | Description | Compatible with | | ---------------------------------------------------------------------------- | ---------------------------------- | ------------------------------------------------------------------------- | | [cf-public-magicoder ↗](https://huggingface.co/predibase/magicoder) | Coding tasks in multiple languages | @cf/mistral/mistral-7b-instruct-v0.1 @hf/mistral/mistral-7b-instruct-v0.2 | | [cf-public-jigsaw-classification ↗](https://huggingface.co/predibase/jigsaw) | Toxic comment classification | @cf/mistral/mistral-7b-instruct-v0.1 @hf/mistral/mistral-7b-instruct-v0.2 | | [cf-public-cnn-summarization ↗](https://huggingface.co/predibase/cnn) | Article summarization | @cf/mistral/mistral-7b-instruct-v0.1 @hf/mistral/mistral-7b-instruct-v0.2 | You can also list these public LoRAs with an API call: Required API token permissions At least one of the following [token permissions](https://developers.cloudflare.com/fundamentals/api/reference/permissions/) is required: * `Workers AI Write` * `Workers AI Read` ```bash curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/finetunes/public" \ --request GET \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` ## Running inference with public LoRAs To run inference with public LoRAs, you just need to define the LoRA name in the request. We recommend that you use the prompt template that the LoRA was trained on. You can find this in the HuggingFace repos linked above for each adapter. ### cURL ```bash curl https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/run/@cf/mistral/mistral-7b-instruct-v0.1 \ --header 'Authorization: Bearer {cf_token}' \ --data '{ "messages": [ { "role": "user", "content": "Write a python program to check if a number is even or odd." } ], "lora": "cf-public-magicoder" }' ``` ### JavaScript ```js const answer = await env.AI.run("@cf/mistral/mistral-7b-instruct-v0.1", { stream: true, raw: true, messages: [ { role: "user", content: "Summarize the following: Some newspapers, TV channels and well-known companies publish false news stories to fool people on 1 April. One of the earliest examples of this was in 1957 when a programme on the BBC, the UKs national TV channel, broadcast a report on how spaghetti grew on trees. The film showed a family in Switzerland collecting spaghetti from trees and many people were fooled into believing it, as in the 1950s British people didn't eat much pasta and many didn't know how it was made! Most British people wouldnt fall for the spaghetti trick today, but in 2008 the BBC managed to fool their audience again with their Miracles of Evolution trailer, which appeared to show some special penguins that had regained the ability to fly. Two major UK newspapers, The Daily Telegraph and the Daily Mirror, published the important story on their front pages.", }, ], lora: "cf-public-cnn-summarization", }); ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/fine-tunes/public-loras/#page","headline":"Public LoRA adapters · Cloudflare Workers AI docs","description":"Cloudflare offers a few public LoRA adapters that are immediately ready for use.","url":"https://developers.cloudflare.com/workers-ai/features/fine-tunes/public-loras/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-05","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Enable Workers AI models to execute functions and interact with external APIs. title: Function calling image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Function calling Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Function calling enables people to take Large Language Models (LLMs) and use the model response to execute functions or interact with external APIs. The developer usually defines a set of functions and the required input schema for each function, which we call `tools`. The model then intelligently understands when it needs to do a tool call, and it returns a JSON output which the user needs to feed to another function or API. In essence, function calling allows you to perform actions with LLMs by executing code or making additional API calls. ## How can I use function calling? Workers AI has [embedded function calling](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/) which allows you to execute function code alongside your inference calls. We have a package called [@cloudflare/ai-utils ↗](https://www.npmjs.com/package/@cloudflare/ai-utils) to help facilitate this, which we have open-sourced on [Github ↗](https://github.com/cloudflare/ai-utils). For industry-standard function calling, take a look at the documentation on [Traditional Function Calling](https://developers.cloudflare.com/workers-ai/features/function-calling/traditional/). To show you the value of embedded function calling, take a look at the example below that compares traditional function calling with embedded function calling. Embedded function calling allowed us to cut down the lines of code from 77 to 31. ```sh # The ai-utils package enables embedded function calling npm i @cloudflare/ai-utils ``` ```js import { createToolsFromOpenAPISpec, runWithTools, autoTrimTools, } from "@cloudflare/ai-utils"; export default { async fetch(request, env, ctx) { const response = await runWithTools( env.AI, "@hf/nousresearch/hermes-2-pro-mistral-7b", { messages: [{ role: "user", content: "Who is Cloudflare on github?" }], tools: [ // You can pass the OpenAPI spec link or contents directly ...(await createToolsFromOpenAPISpec( "https://gist.githubusercontent.com/mchenco/fd8f20c8f06d50af40b94b0671273dc1/raw/f9d4b5cd5944cc32d6b34cad0406d96fd3acaca6/partial_api.github.com.json", { overrides: [ { // for all requests on *.github.com, we'll need to add a User-Agent. matcher: ({ url, method }) => { return url.hostname === "api.github.com"; }, values: { headers: { "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36", }, }, }, ], }, )), ], }, ).then((response) => { return response; }); return new Response(JSON.stringify(response)); }, }; ``` ```js export default { async fetch(request, env, ctx) { const response = await env.AI.run( "@hf/nousresearch/hermes-2-pro-mistral-7b", { messages: [{ role: "user", content: "Who is Cloudflare on GitHub?" }], tools: [ { name: "getGithubUser", description: "Provides publicly available information about someone with a GitHub account.", parameters: { type: "object", properties: { username: { type: "string", description: "The handle for the GitHub user account.", }, }, required: ["username"], }, }, ], }, ); const selected_tool = response.tool_calls[0]; let res; if (selected_tool.name == "getGithubUser") { try { const username = selected_tool.arguments.username; const url = `https://api.github.com/users/${username}`; res = await fetch(url, { headers: { // Github API requires a User-Agent header "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36", }, }).then((res) => res.json()); } catch (error) { return error; } } const finalResponse = await env.AI.run( "@hf/nousresearch/hermes-2-pro-mistral-7b", { messages: [ { role: "user", content: "Who is Cloudflare on GitHub?", }, { role: "assistant", content: JSON.stringify(selected_tool), }, { role: "tool", content: JSON.stringify(res), }, ], tools: [ { name: "getGithubUser", description: "Provides publicly available information about someone with a GitHub account.", parameters: { type: "object", properties: { username: { type: "string", description: "The handle for the GitHub user account.", }, }, required: ["username"], }, }, ], }, ); return new Response(JSON.stringify(finalResponse)); }, }; ``` ## What models support function calling? There are open-source models which have been fine-tuned to do function calling. When browsing our [model catalog](https://developers.cloudflare.com/workers-ai/models/), look for models with the function calling property beside it. For example, [@hf/nousresearch/hermes-2-pro-mistral-7b](https://developers.cloudflare.com/workers-ai/models/hermes-2-pro-mistral-7b/) is a fine-tuned variant of Mistral 7B that you can use for function calling. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/#page","headline":"Function calling · Cloudflare Workers AI docs","description":"Enable Workers AI models to execute functions and interact with external APIs.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["LLM"]} ``` --- --- description: Execute function code alongside inference calls using Workers AI embedded function calling. title: Embedded image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Embedded Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Cloudflare has a unique [embedded function calling ↗](https://blog.cloudflare.com/embedded-function-calling) feature that allows you to execute function code alongside your tool call inference. Our npm package [@cloudflare/ai-utils ↗](https://www.npmjs.com/package/@cloudflare/ai-utils) is the developer toolkit to get started. Embedded function calling can be used to easily make complex agents that interact with websites and APIs, like using natural language to create meetings on Google Calendar, saving data to Notion, automatically routing requests to other APIs, saving data to an R2 bucket - or all of this at the same time. All you need is a prompt and an OpenAPI spec to get started. REST API support Embedded function calling depends on features native to the Workers platform. This means that embedded function calling is only supported via [Cloudflare Workers](https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/), not via the [REST API](https://developers.cloudflare.com/workers-ai/get-started/rest-api/). ## Resources * [Get Started](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/get-started/) * [Examples](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/) * [API Reference](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/api-reference/) * [Troubleshooting](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/troubleshooting/) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/#page","headline":"Embedded function calling · Cloudflare Workers AI docs","description":"Execute function code alongside inference calls using Workers AI embedded function calling.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Reference for the runWithTools and autoTrimTools methods in embedded function calling. title: API Reference image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # API Reference Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/api-reference/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Learn more about the API reference for [embedded function calling](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded). ## runWithTools This wrapper method enables you to do embedded function calling. You pass it the AI binding, model, inputs (`messages` array and `tools` array), and optional configurations. * `AI Binding`Ai * The AI binding, such as `env.AI`. * `model`BaseAiTextGenerationModels * The ID of the model that supports function calling. For example, `@hf/nousresearch/hermes-2-pro-mistral-7b`. * `input`Object * `messages`RoleScopedChatInput\[\] * `tools`AiTextGenerationToolInputWithFunction\[\] * `config`Object * `streamFinalResponse`boolean optional * `maxRecursiveToolRuns`number optional * `strictValidation`boolean optional * `verbose`boolean optional * `trimFunction`boolean optional - For the `trimFunction`, you can pass it `autoTrimTools`, which is another helper method we've devised to automatically choose the correct tools (using an LLM) before sending it off for inference. This means that your final inference call will have fewer input tokens. ## createToolsFromOpenAPISpec This method lets you automatically create tool schemas based on OpenAPI specs, so you don't have to manually write or hardcode the tool schemas. You can pass the OpenAPI spec for any API in JSON or YAML format. `createToolsFromOpenAPISpec` has a config input that allows you to perform overrides if you need to provide headers like Authentication or User-Agent. * `spec`string * The OpenAPI specification in either JSON or YAML format, or a URL to a remote OpenAPI specification. * `config`Config optional - Configuration options for the createToolsFromOpenAPISpec function * `overrides`ConfigRule\[\] optional * `matchPatterns`RegExp\[\] optional * `options` Object optional { `verbose` boolean optional } Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/api-reference/#page","headline":"API Reference - Embedded function calling · Cloudflare Workers AI docs","description":"Reference for the runWithTools and autoTrimTools methods in embedded function calling.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/api-reference/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Learn how to use the fetch() handler in Cloudflare Workers AI to enable LLMs to perform API calls, like retrieving a 5-day weather forecast using function calling. title: Use fetch() handler image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Use fetch() handler Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/fetch/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) A very common use case is to provide the LLM with the ability to perform API calls via function calling. In this example the LLM will retrieve the weather forecast for the next 5 days. To do so a `getWeather` function is defined that is passed to the LLM as tool. The `getWeather`function extracts the user's location from the request and calls the external weather API via the Workers' [Fetch API](https://developers.cloudflare.com/workers/runtime-apis/fetch/) and returns the result. ```ts import { runWithTools } from '@cloudflare/ai-utils'; type Env = { AI: Ai; }; export default { async fetch(request, env, ctx) { // Define function const getWeather = async (args: { numDays: number }) => { const { numDays } = args; // Location is extracted from request based on // https://developers.cloudflare.com/workers/runtime-apis/request/#incomingrequestcfproperties const lat = request.cf?.latitude const long = request.cf?.longitude // Interpolate values for external API call const response = await fetch( `https://api.open-meteo.com/v1/forecast?latitude=${lat}&longitude=${long}&daily=temperature_2m_max,precipitation_sum&timezone=GMT&forecast_days=${numDays}` ); return response.text(); }; // Run AI inference with function calling const response = await runWithTools( env.AI, // Model with function calling support '@hf/nousresearch/hermes-2-pro-mistral-7b', { // Messages messages: [ { role: 'user', content: 'What the weather like the next 5 days? Respond as text', }, ], // Definition of available tools the AI model can leverage tools: [ { name: 'getWeather', description: 'Get the weather for the next [numDays] days', parameters: { type: 'object', properties: { numDays: { type: 'numDays', description: 'number of days for the weather forecast' }, }, required: ['numDays'], }, // reference to previously defined function function: getWeather, }, ], } ); return new Response(JSON.stringify(response)); }, } satisfies ExportedHandler; ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/fetch/#page","headline":"Use fetch() handler · Cloudflare Workers AI docs","description":"Learn how to use the fetch() handler in Cloudflare Workers AI to enable LLMs to perform API calls, like retrieving a 5-day weather forecast using function calling.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/fetch/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: Learn how to use Cloudflare Workers AI to interact with KV storage, enabling persistent data handling with embedded function calling in a few lines of code. title: Use KV API image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Use KV API Last updated Oct 13, 2025|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/kv/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Interact with persistent storage to retrieve or store information enables for powerful use cases. In this example we show how embedded function calling can interact with other resources on the Cloudflare Developer Platform with a few lines of code. ## Pre-Requisites For this example to work, you need to provision a [KV](https://developers.cloudflare.com/kv/) namespace first. To do so, follow the [KV - Get started ](https://developers.cloudflare.com/kv/get-started/) guide. Importantly, your Wrangler file must be updated to include the `KV` binding definition to your respective namespace. ## Worker code ```ts import { runWithTools } from "@cloudflare/ai-utils"; type Env = { AI: Ai; KV: KVNamespace; }; export default { async fetch(request, env, ctx) { // Define function const updateKvValue = async ({ key, value, }: { key: string; value: string; }) => { const response = await env.KV.put(key, value); return `Successfully updated key-value pair in database: ${response}`; }; // Run AI inference with function calling const response = await runWithTools( env.AI, "@hf/nousresearch/hermes-2-pro-mistral-7b", { messages: [ { role: "system", content: "Put user given values in KV" }, { role: "user", content: "Set the value of banana to yellow." }, ], tools: [ { name: "KV update", description: "Update a key-value pair in the database", parameters: { type: "object", properties: { key: { type: "string", description: "The key to update", }, value: { type: "string", description: "The value to update", }, }, required: ["key", "value"], }, function: updateKvValue, }, ], }, ); return new Response(JSON.stringify(response)); }, } satisfies ExportedHandler; ``` ## Verify results To verify the results, run the following command ```sh npx wrangler kv key get banana --binding KV --local ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/kv/#page","headline":"Use KV API · Cloudflare Workers AI docs","description":"Learn how to use Cloudflare Workers AI to interact with KV storage, enabling persistent data handling with embedded function calling in a few lines of code.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/kv/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2025-10-13","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: Generate Workers AI function calling tools from an OpenAPI specification using the ai-utils package. title: Tools based on OpenAPI Spec image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Tools based on OpenAPI Spec Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/openapi/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Oftentimes APIs are defined and documented via [OpenAPI specification ↗](https://swagger.io/specification/). The Cloudflare `ai-utils` package's `createToolsFromOpenAPISpec` function creates tools from the OpenAPI spec, which the LLM can then leverage to fulfill the prompt. In this example the LLM will describe the a Github user, based Github's API and its OpenAPI spec. ```ts import { createToolsFromOpenAPISpec, runWithTools } from '@cloudflare/ai-utils'; type Env = { AI: Ai; }; const APP_NAME = 'cf-fn-calling-example-app'; export default { async fetch(request, env, ctx) { const toolsFromOpenAPISpec = [ // You can pass the OpenAPI spec link or contents directly ...(await createToolsFromOpenAPISpec( 'https://gist.githubusercontent.com/mchenco/fd8f20c8f06d50af40b94b0671273dc1/raw/f9d4b5cd5944cc32d6b34cad0406d96fd3acaca6/partial_api.github.com.json', { overrides: [ { matcher: ({ url }) => { return url.hostname === 'api.github.com'; }, // for all requests on *.github.com, we'll need to add a User-Agent. values: { headers: { 'User-Agent': APP_NAME, }, }, }, ], } )), ]; const response = await runWithTools( env.AI, '@hf/nousresearch/hermes-2-pro-mistral-7b', { messages: [ { role: 'user', content: 'Who is cloudflare on Github and how many repos does the organization have?', }, ], tools: toolsFromOpenAPISpec, } ); return new Response(JSON.stringify(response)); }, } satisfies ExportedHandler; ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/openapi/#page","headline":"Tools based on OpenAPI Spec · Cloudflare Workers AI docs","description":"Generate Workers AI function calling tools from an OpenAPI specification using the ai-utils package.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/openapi/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: Set up and deploy your first Workers AI project with embedded function calling. title: Get Started image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Get Started Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/get-started/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) This guide will instruct you through setting up and deploying your first Workers AI project with embedded function calling. You will use Workers, a Workers AI binding, the [ai-utils package ↗](https://github.com/cloudflare/ai-utils), and a large language model (LLM) to deploy your first AI-powered application on the Cloudflare global network with embedded function calling. ## 1\. Create a Worker project with Workers AI Follow the [Workers AI Get Started Guide](https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/) until step 2. ## 2\. Install additional npm package Next, run the following command in your project repository to install the Worker AI utilities package. npmyarnpnpmbun ``` npm i @cloudflare/ai-utils ``` ``` yarn add @cloudflare/ai-utils ``` ``` pnpm add @cloudflare/ai-utils ``` ``` bun add @cloudflare/ai-utils ``` ## 3\. Add Workers AI Embedded function calling Update the `index.ts` file in your application directory with the following code: ```js import { runWithTools } from "@cloudflare/ai-utils"; export default { async fetch(request, env, ctx) { // Define function const sum = (args) => { const { a, b } = args; return Promise.resolve((a + b).toString()); }; // Run AI inference with function calling const response = await runWithTools( env.AI, // Model with function calling support "@hf/nousresearch/hermes-2-pro-mistral-7b", { // Messages messages: [ { role: "user", content: "What the result of 123123123 + 10343030?", }, ], // Definition of available tools the AI model can leverage tools: [ { name: "sum", description: "Sum up two numbers and returns the result", parameters: { type: "object", properties: { a: { type: "number", description: "the first number" }, b: { type: "number", description: "the second number" }, }, required: ["a", "b"], }, // reference to previously defined function function: sum, }, ], }, ); return new Response(JSON.stringify(response)); }, }; ``` ```ts import { runWithTools } from "@cloudflare/ai-utils"; type Env = { AI: Ai; }; export default { async fetch(request, env, ctx) { // Define function const sum = (args: { a: number; b: number }): Promise => { const { a, b } = args; return Promise.resolve((a + b).toString()); }; // Run AI inference with function calling const response = await runWithTools( env.AI, // Model with function calling support "@hf/nousresearch/hermes-2-pro-mistral-7b", { // Messages messages: [ { role: "user", content: "What the result of 123123123 + 10343030?", }, ], // Definition of available tools the AI model can leverage tools: [ { name: "sum", description: "Sum up two numbers and returns the result", parameters: { type: "object", properties: { a: { type: "number", description: "the first number" }, b: { type: "number", description: "the second number" }, }, required: ["a", "b"], }, // reference to previously defined function function: sum, }, ], }, ); return new Response(JSON.stringify(response)); }, } satisfies ExportedHandler; ``` This example imports the utils with `import { runWithTools} from "@cloudflare/ai-utils"` and follows the API reference below. Moreover, in this example we define and describe a list of tools that the LLM can leverage to respond to the user query. Here, the list contains of only one tool, the `sum` function. Abstracted by the `runWithTools` function, the following steps occur: sequenceDiagram participant Worker as Worker participant WorkersAI as Workers AI Worker->>+WorkersAI: Send messages, function calling prompt, and available tools WorkersAI->>+Worker: Select tools and arguments for function calling Worker-->>-Worker: Execute function Worker-->>+WorkersAI: Send messages, function calling prompt and function result WorkersAI-->>-Worker: Send response incorporating function output The `ai-utils package` is also open-sourced on [Github ↗](https://github.com/cloudflare/ai-utils). ## 4\. Local development & deployment Follow steps 4 and 5 of the [Workers AI Get Started Guide](https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/) for local development and deployment. Workers AI Embedded Function Calling charges Embedded function calling runs Workers AI inference requests. Standard charges for inference (e.g. tokens) usage will be charged. Resources consumed (e.g. CPU time) during embedded functions' code execution will be charged just as any other Worker's code execution. ## API reference For more details, refer to [API reference](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/api-reference/). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/get-started/#page","headline":"Get Started · Cloudflare Workers AI docs","description":"Set up and deploy your first Workers AI project with embedded function calling.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/get-started/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Debug and resolve common issues with Workers AI embedded function calling. title: Troubleshooting image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Troubleshooting Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/troubleshooting/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) This section will describe tools for troubleshooting and address common errors. ## Logging General [logging](https://developers.cloudflare.com/workers/observability/logs/) capabilities for Workers also apply to embedded function calling. ### Function invocations The invocations of tools can be logged as in any Worker using `console.log()`: ```ts export default { async fetch(request, env, ctx) { const sum = (args: { a: number; b: number }): Promise => { const { a, b } = args; // Logging from within embedded function invocations console.log(`The sum function has been invoked with the arguments a: ${a} and b: ${b}`) return Promise.resolve((a + b).toString()); }; ... } } ``` ### Logging within `runWithTools` The `runWithTools` function has a `verbose` mode that emits helpful logs for debugging of function calls as well input and output statistics. ```ts const response = await runWithTools( env.AI, '@hf/nousresearch/hermes-2-pro-mistral-7b', { messages: [ ... ], tools: [ ... ], }, // Enable verbose mode { verbose: true } ); ``` ## Performance To respond to a LLM prompt with embedded function, potentially multiple AI inference requests and function invocations are needed, which can have an impact on user experience. Consider the following to improve performance: * Shorten prompts (to reduce time for input processing) * Reduce number of tools provided * Stream the final response to the end user (to minimize the time to interaction). See example below: ```ts async fetch(request, env, ctx) { const response = (await runWithTools( env.AI, '@hf/nousresearch/hermes-2-pro-mistral-7b', { messages: [ ... ], tools: [ ... ], }, { // Enable response streaming streamFinalResponse: true, } )) as ReadableStream; // Set response headers for streaming return new Response(response, { headers: { 'content-type': 'text/event-stream', }, }); } ``` ## Common Errors If you are getting a `BadInput` error, your inputs may exceed our current context window for our models. Try reducing input tokens to resolve this error. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/troubleshooting/#page","headline":"Troubleshooting · Cloudflare Workers AI docs","description":"Debug and resolve common issues with Workers AI embedded function calling.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/troubleshooting/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Define tools and schemas for industry-standard function calling with Workers AI models. title: Traditional image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Traditional Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/traditional/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) This page shows how you can do traditional function calling, as defined by industry standards. Workers AI also offers [embedded function calling](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/), which is drastically easier than traditional function calling. With traditional function calling, you define an array of tools with the name, description, and tool arguments. The example below shows how you would pass a tool called `getWeather` in an inference request to a model. ```js const response = await env.AI.run("@hf/nousresearch/hermes-2-pro-mistral-7b", { messages: [ { role: "user", content: "what is the weather in london?", }, ], tools: [ { name: "getWeather", description: "Return the weather for a latitude and longitude", parameters: { type: "object", properties: { latitude: { type: "string", description: "The latitude for the given location", }, longitude: { type: "string", description: "The longitude for the given location", }, }, required: ["latitude", "longitude"], }, }, ], }); return new Response(JSON.stringify(response.tool_calls)); ``` The LLM will then return a JSON object with the required arguments and the name of the tool that was called. You can then pass this JSON object to make an API call. ```json [ { "arguments": { "latitude": "51.5074", "longitude": "-0.1278" }, "name": "getWeather" } ] ``` For a working example on how to do function calling, take a look at our [demo app ↗](https://github.com/craigsdennis/lightbulb-moment-tool-calling/blob/main/src/index.ts). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/traditional/#page","headline":"Traditional function calling · Cloudflare Workers AI docs","description":"Define tools and schemas for industry-standard function calling with Workers AI models.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/traditional/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Force Workers AI text generation models to return valid JSON output using response_format or JSON schemas. title: JSON Mode image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # JSON Mode Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/json-mode/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) When we want text-generation AI models to interact with databases, services, and external systems programmatically, typically when using tool calling or building AI agents, we must have structured response formats rather than natural language. Workers AI supports JSON Mode, enabling applications to request a structured output response when interacting with AI models. ## Schema JSON Mode is compatible with OpenAI’s implementation; to enable add the `response_format` property to the request object using the following convention: ```json { response_format: { title: "JSON Mode", type: "object", properties: { type: { type: "string", enum: ["json_object", "json_schema"], }, json_schema: {}, } } } ``` Where `json_schema` must be a valid [JSON Schema ↗](https://json-schema.org/) declaration. ## JSON Mode example When using JSON Format, pass the schema as in the example below as part of the request you send to the LLM. ```json { "messages": [ { "role": "system", "content": "Extract data about a country." }, { "role": "user", "content": "Tell me about India." } ], "response_format": { "type": "json_schema", "json_schema": { "type": "object", "properties": { "name": { "type": "string" }, "capital": { "type": "string" }, "languages": { "type": "array", "items": { "type": "string" } } }, "required": [ "name", "capital", "languages" ] } } } ``` The LLM will follow the schema, and return a response such as below: ```json { "response": { "name": "India", "capital": "New Delhi", "languages": [ "Hindi", "English", "Bengali", "Telugu", "Marathi", "Tamil", "Gujarati", "Urdu", "Kannada", "Odia", "Malayalam", "Punjabi", "Sanskrit" ] } } ``` As you can see, the model is complying with the JSON schema definition in the request and responding with a validated JSON object. ## Supported Models This is the list of models that now support JSON Mode: * [@cf/meta/llama-3.1-8b-instruct-fast](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fast/) * [@cf/meta/llama-3.1-70b-instruct](https://developers.cloudflare.com/workers-ai/models/llama-3.1-70b-instruct/) * [@cf/meta/llama-3.3-70b-instruct-fp8-fast](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/) * [@cf/meta/llama-3-8b-instruct](https://developers.cloudflare.com/workers-ai/models/llama-3-8b-instruct/) * [@cf/meta/llama-3.1-8b-instruct](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct/) * [@cf/meta/llama-3.2-11b-vision-instruct](https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct/) * [@hf/nousresearch/hermes-2-pro-mistral-7b](https://developers.cloudflare.com/workers-ai/models/hermes-2-pro-mistral-7b/) * [@hf/thebloke/deepseek-coder-6.7b-instruct-awq](https://developers.cloudflare.com/workers-ai/models/deepseek-coder-6.7b-instruct-awq/) * [@cf/deepseek-ai/deepseek-r1-distill-qwen-32b](https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/) We will continue extending this list to keep up with new, and requested models. Note that Workers AI can't guarantee that the model responds according to the requested JSON Schema. Depending on the complexity of the task and adequacy of the JSON Schema, the model may not be able to satisfy the request in extreme situations. If that's the case, then an error `JSON Mode couldn't be met` is returned and must be handled. JSON Mode currently doesn't support streaming. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/features/json-mode/#page","headline":"JSON Mode · Cloudflare Workers AI docs","description":"Force Workers AI text generation models to return valid JSON output using response\\_format or JSON schemas.","url":"https://developers.cloudflare.com/workers-ai/features/json-mode/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["JSON"]} ``` --- --- description: Convert documents in multiple formats to Markdown using the Workers AI toMarkdown method. title: Markdown Conversion image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Markdown Conversion Last updated Apr 21, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Markdown ↗](https://en.wikipedia.org/wiki/Markdown) is essential for text generation and large language models (LLMs) in training and inference because it can provide structured, semantic, human, and machine-readable input. Likewise, Markdown facilitates chunking and structuring input data for better retrieval and synthesis in the context of RAGs, and its simplicity and ease of parsing and rendering make it ideal for AI Agents. For these reasons, document conversion plays an important role when designing and developing AI applications. Workers AI provides the `toMarkdown` utility method that developers can use from the [env.AI](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/usage/binding/) binding or the [REST APIs](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/usage/rest-api/) for quick, easy, and convenient conversion and summary of documents in multiple formats to Markdown language. ## Pricing `toMarkdown` is free for most format conversions. In some cases, like image conversion, it can use Workers AI models for object detection and summarization, which may incur additional costs if it exceeds the Workers AI free allocation limits. Refer to [what models we use](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/how-it-works/) and the [Workers AI pricing page](https://developers.cloudflare.com/workers-ai/platform/pricing/) for more details. ## Other Markdown conversion features * The Browser Run [/markdown](https://developers.cloudflare.com/browser-run/quick-actions/markdown-endpoint/) endpoint supports markdown conversion if you need to render a dynamic page or application in a real browser before converting it. * [Markdown for Agents](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/) allows real-time document conversion for Cloudflare zones using content negotiation headers. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/features/markdown-conversion/#page","headline":"Markdown Conversion · Cloudflare Workers AI docs","description":"Convert documents in multiple formats to Markdown using the Workers AI toMarkdown method.","url":"https://developers.cloudflare.com/workers-ai/features/markdown-conversion/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Configure per-format options for Workers AI Markdown Conversion, including HTML and image settings. title: Conversion Options image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Conversion Options Last updated Jul 13, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/conversion-options/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) By default, the `toMarkdown` service extracts text content from your files. To further extend the capabilities of the conversion process, you can pass options to the service to control how specific file types are converted. Options are organized by file type and are all optional. ## Available options ### Output ```typescript { output?: { format?: 'markdown' | 'text'; } } ``` * `format`: controls the format of the converted content. Defaults to `markdown`. Set to `text` to receive plain text with Markdown syntax removed. When `format` is `text`, the `format` field of the [ConversionResult](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/usage/binding/#conversionresult-definition) is also set to `text`. ### Images ```typescript { image?: { descriptionLanguage?: 'en' | 'it' | 'de' | 'es' | 'fr' | 'pt'; } } ``` * `descriptionLanguage`: controls the language of the AI-generated image descriptions. Caution This option works on a _best-effort_ basis: it is not guaranteed that the resulting text will be in the desired language. ### HTML ```typescript { html?: { hostname?: string; cssSelector?: string; } } ``` * `hostname`: string to use as a host when resolving relative links inside the HTML. * `cssSelector`: string containing a CSS selector pattern to pick specific elements from your HTML. Refer to [how HTML is processed](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/how-it-works/#html) for more details. ### PDF ```typescript { pdf?: { metadata?: boolean; } } ``` * `metadata`: Previously, all converted PDF files always included metadata information when converted. This option allows you to opt-out of this behavior. ## Examples ### Binding To configure custom options, pass a `conversionOptions` object inside the second argument of the binding call, like this: ```typescript await env.AI.toMarkdown(..., { conversionOptions: { html: { ... }, pdf: { ... }, ... } }) ``` ### REST API Since the REST API uses file uploads, the request's `Content-Type` will be `multipart/form-data`. As such, include a new form field with your stringified object as a value: ```bash curl https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/tomarkdown \ -X POST \ -H 'Authorization: Bearer {API_TOKEN}' \ ... -F 'conversionOptions={ "html": { ... }, ... }' ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/markdown-conversion/conversion-options/#page","headline":"Conversion Options · Cloudflare Workers AI docs","description":"Configure per-format options for Workers AI Markdown Conversion, including HTML and image settings.","url":"https://developers.cloudflare.com/workers-ai/features/markdown-conversion/conversion-options/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-07-13","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Learn how Workers AI pre-processes and converts HTML, images, and other files to Markdown. title: How it works image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # How it works Last updated Jul 8, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/how-it-works/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) ## Pre-processing When parsing files before converting them to Markdown, there are some cleanup tasks we do depending on the type of file you are trying to convert. ### HTML When we detect an HTML file, a series of things happen to the HTML content before it is converted: * Some elements are ignored, including `script` and `style` tags. * Meta tags are extracted. These include `title`, `description`, `og:title`, `og:description` and `og:image`. * [JSON-LD ↗](https://json-ld.org/) content is extracted, if it exists. This will be appended at the end of the converted markdown. * The base URL to use for resolving relative links is extracted from the `` element1, if it exists, according to the spec (that is, only the first instance of the base URL is counted). * If the `cssSelector` option is: * present, then only those elements that match the selector are kept for further processing; * missing, then elements such as `
`, `