llm-observability

🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

AI Agent Engineering Platform built on an Open Source TypeScript AI Agent Framework

Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode. Works with any SDK or agent and adds under 200µs of overhead at 5k RPS.

Open Source LLM Gateway 🦚

Observability and enforcement for AI agent harnesses. Capture every run and runtime reliability with policy enforcement.

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.

Open-source observability for AI agents. Find where your agents fail, dispatch your coding agent to fix it, and verify the fix against real traces.

AI observability platform for production LLM and agent systems.

Agent Skills as a Memory Layer

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

OpenLIT is the open-source agent harness engineering platform: trace, evaluate, guard, and improve everything around the model in your AI agents, on OTEL.

Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

Trace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs that regress. Self-hostable.

The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.

TraceRoot - Open Source AI Agent Improvement Platform. YC S25

Self-hosted AI API and MCP gateway for organizations: SSO and RBAC, per-user identity for MCP tool calls, PII redaction and tool-call guards, rate limits, budgets, cost accounting and audit logs.

🦞 Official plugin for OpenClaw that exports agent traces to Opik. See and monitor agent behaviour, cost, tokens, errors and more.

DataBuff is an AI-native APM built on Opentelemetry，with multi-agent troubleshooting out of the box.