This repo contains example notebooks and accompanying documentation for using Fiddler.

These example notebooks aim to give you a quick start on various Fiddler capabilities using different model tasks, data types, and use cases. They can also serve as a reference guide for setting up the monitoring of your own models in Fiddler.

Use the projects in this repo to onboard models and data to illustrate ML model and LLM application monitoring, analysis, and protection. This repo contains the example notebooks listed below. You can launch them in a Google Colab environment using the Colab links.

Note: This repository uses Git Large File Storage (Git LFS) for managing large files. Please make sure you have Git LFS installed before cloning this repository. You can find installation instructions at git-lfs.github.com. You can find the file types tracked via GitLFS at the .gitattributes file (currently only .csv files)

• LLM - Comparison
• LLM - Simple Monitoring Quickstart
• ML - Simple Monitoring Quickstart
• Managing Model Versions with Fiddler
• User-defined Feature Impact Upload

• Image/Computer Vision Model Monitoring
• NLP Model Monitoring - Multiclass Classification
• Class Imbalance Drift Detection
• Ranking Model - Monitoring
• Regression Model - Monitoring

Use-case oriented notebooks that demonstrate end-to-end workflows for solving real AI evaluation and monitoring problems with Fiddler. Each cookbook has a companion guide on the Fiddler docs site.

• RAG Evaluation Fundamentals — Evaluate RAG quality with built-in evaluators
• RAG Experiments at Scale — Compare RAG pipeline configurations systematically
• Custom Judge Evaluators — Create domain-specific evaluation criteria

fiddler-utils is an admin automation library for Fiddler that provides high-level abstractions for common administrative tasks (connection management, asset export/import, FQL parsing, schema validation, model comparison, and more). It is not part of the official Fiddler SDK, but both Fiddler field engineers and customers are welcome to use and extend it.

The library now lives in its own public repository: https://github.com/fiddler-labs/fiddler-utils

Install the latest tagged release from git:

pip install "fiddler-utils @ git+https://github.com/fiddler-labs/fiddler-utils.git@v1.0.1"
# or with uv:
uv add "git+https://github.com/fiddler-labs/fiddler-utils.git@v1.0.1"

For the OpenTelemetry-based event ingestion utilities (log_pandas_traces, log_event_traces), install with the [otel] extra:

pip install "fiddler-utils[otel] @ git+https://github.com/fiddler-labs/fiddler-utils.git@v1.0.1"

Quick example:

from fiddler_utils import get_or_init, ModelComparator, SegmentManager

get_or_init(url="https://your-org.cloud.fiddler.ai", token="...")

See the fiddler-utils README for the full API reference and usage examples.

Several notebooks in the misc-utils directory demonstrate fiddler-utils features:

• export_import_models.ipynb - Complete model export/import workflows
• export_import_model_assets.ipynb - Asset-level transfers between models
• compare_models.ipynb - Comprehensive model comparison
• env_stats.ipynb - Environment reporting and analysis

The misc-utils directory contains utility notebooks for customer success engineers, field AI engineers, and solution engineers. These notebooks provide tools for various administrative tasks and solutions to common challenges when working with Fiddler deployments. See the misc-utils README for a detailed catalog of available utilities.

This project is licensed under the MIT license. See the LICENSE file for more info.

• Documentation
• Fiddler Blog
• Fiddler Resource Library