Add vLLM-style Runtime Metrics for Inference & Training (with Optional Telemetry)

This PR adds an opt-in runtime metrics system to Unsloth, inspired by vLLM’s metrics architecture, with optional Prometheus export and optional server-side telemetry forwarding.

What this enables

• Inference metrics: request counts, token counts, throughput, latency histograms (E2E, prefill, decode)
• Training metrics: steps, samples/sec, loss, LR, gradient norm, forward/backward timing
• Prometheus support (optional) with /metrics HTTP endpoint
• Programmatic access to metrics (no server required)
• Optional telemetry forwarding of aggregated metrics to Unsloth servers

How it works

• Metrics are disabled by default
• Calling enable_prometheus_metrics() automatically instruments:
  • unsloth_base_fast_generate() (inference)
  • Trainer.training_step() via a patch hook (training)
• Telemetry forwarding is opt-in and non-blocking
  • Enabled via UNSLOTH_ENABLE_METRICS_TELEMETRY=1 or enable_telemetry()
  • Can be disabled via UNSLOTH_DISABLE_METRICS_TELEMETRY=1
• No user code changes required beyond enabling metrics

Key design points

• Fully opt-in, no breaking changes
• Graceful degradation if prometheus_client is not installed
• Lightweight + low overhead
• Inspired by vLLM’s metrics model, adapted to Transformers-based pipelines
• Thread-safe singleton pattern
• Handles ModelOutput objects when return_dict_in_generate=True
• Telemetry sends aggregated stats only (counts / averages, no raw prompts or user data)

Files changed (13 files)

• New module: unsloth/metrics/ (6 files)
  • stats.py – Core statistics tracking (InferenceStats, TrainingStats, StatsCollector)
  • prometheus.py – Prometheus export with Counter / Gauge / Histogram metrics
  • server.py – Optional HTTP server for metrics scraping
  • telemetry.py – Optional background telemetry sender (aggregated stats only)
  • README.md – Documentation
• Training hook: _patch_training_metrics() in unsloth/models/_utils.py
• Inference hook: unsloth_base_fast_generate() in unsloth/models/vision.py
• Public API exports: via unsloth/__init__.py
• Tests: tests/metrics/test_metrics_standalone.py (all passing)
• Dependencies: pyproject.toml (optional prometheus_client)

Quick usage

from unsloth import enable_prometheus_metrics, get_stats_collector

enable_prometheus_metrics()

# run inference / training as usual

stats = get_stats_collector().get_all_stats()
print(stats["inference"])  # request counts, latencies, tokens/sec
print(stats["training"])   # steps, loss, samples/sec

Notes

• Telemetry is opt-in by default (can be flipped easily if preferred)
• Uses a background sender (non-blocking, silent failures)
• Endpoint is configurable via UNSLOTH_METRICS_TELEMETRY_ENDPOINT
• Current default endpoint is a placeholder pending server-side confirmation

Testing

• Kaggle smoke test confirming metrics collection + non-blocking telemetry forwarding:
  https://www.kaggle.com/code/hnxnq07/metrics-telemetry-smoketest

No breaking changes. Purely additive.