Use the @mux/ai library to automatically generate titles, descriptions, and tags for your videos via LLMs
Automatically generating video metadata like titles, descriptions, and tags helps you build better search experiences, improve content discovery, and save time on manual content curation. The @mux/ai library makes this straightforward by analyzing video transcripts and storyboard images to generate metadata.
Before starting, make sure you have:
npm install @mux/aiSet your environment variables:
# Required
MUX_TOKEN_ID=your_mux_token_id
MUX_TOKEN_SECRET=your_mux_token_secret
# You only need the API key for the provider you're using
OPENAI_API_KEY=your_openai_api_key # OR
ANTHROPIC_API_KEY=your_anthropic_api_key # OR
GOOGLE_GENERATIVE_AI_API_KEY=your_google_api_keyimport { getSummaryAndTags } from "@mux/ai/workflows";
const result = await getSummaryAndTags("your-mux-asset-id", {
tone: "professional" // or "neutral" or "playful"
});
console.log(result.title);
// "How to Build a Video Platform in 2025"
console.log(result.description);
// "Learn the fundamentals of building a modern video platform..."
console.log(result.tags);
// ["video streaming", "web development", "tutorial", "javascript"]You can control the style of generated content with the tone option:
// Professional tone - formal and business-appropriate
const professional = await getSummaryAndTags("your-mux-asset-id", {
tone: "professional"
});
// Neutral tone - balanced and conversational (default)
const neutral = await getSummaryAndTags("your-mux-asset-id", {
tone: "neutral"
});
// Playful tone - playful and engaging
const playful = await getSummaryAndTags("your-mux-asset-id", {
tone: "playful"
});Here's some example titles for each tone, based on the same demo video of Mux's thumbnail API:
@mux/ai supports three AI providers:
gpt-5.1 model - Fast and cost-effectiveclaude-sonnet-4-5 model - Great for nuanced understandinggemini-3-flash-preview model - Balance of speed and qualityconst result = await getSummaryAndTags("your-mux-asset-id", {
provider: "anthropic", // or "openai" or "google"
model: "claude-opus-4-5" // Optional: override default model
});By default, @mux/ai analyzes both the storyboard images and transcript. Storyboard images are always included, but you can optionally exclude the transcript:
// Exclude transcript (faster, uses only visual analysis)
const result = await getSummaryAndTags("your-mux-asset-id", {
includeTranscript: false
});You can override specific parts of the prompt to tune the output:
const result = await getSummaryAndTags("your-mux-asset-id", {
promptOverrides: {
system: "You are a video content specialist focused on technical tutorials.",
instructions: "Create a title under 60 characters and exactly 5 tags focused on technical concepts."
}
});For automated metadata generation when videos are uploaded, you should trigger the call to get the summary and tags from the video.asset.track.ready webhook:
export async function handleWebhook(req, res) {
const event = req.body;
if (event.type === 'video.asset.track.ready' &&
event.data.type === 'text' &&
event.data.language_code === 'en') {
const result = await getSummaryAndTags(event.data.asset_id, { tone: "professional" });
await db.updateVideo(event.data.asset_id, { title: result.title, description: result.description, tags: result.tags });
}
}Once you have automatically generated metadata, you can:
Under the hood, @mux/ai handles:
gpt-5.1 is cost-effective for most use cases