Grafana

hybridFree (self-hosted OSS)Free cloud (10k metrics)$29/mo ProCustom Enterpriseopen source

Best for

Infrastructure dashboards and alerting — best paired with Prometheus/Loki/Tempo for a fully open-source observability stack

Limitations

No native LLM tracing; requires additional tooling (Langfuse, OpenTelemetry) for AI-specific observability; steep learning curve for the full LGTM stack

Features

Llm Tracing

Trace LLM calls, tool invocations, and agent reasoning steps end-to-end

Cost Tracking

Track token usage and cost per request, per agent run, and per model

Evaluation

Score agent outputs against test datasets with automated evaluators

Prompt Management

Version, manage, and A/B test prompts in production

Real Time Monitoring

Live dashboards and alerting for agent performance metrics

Frameworks

None listed

SDK Languages

pythonjavascriptgojava

Compliance

soc2hipaagdpr

Grafana

Grafana is the dominant open-source dashboarding and visualization platform. It doesn't provide LLM-specific tracing natively, but it's the go-to choice for infrastructure observability — metrics, logs, and traces via the Prometheus/Loki/Tempo stack (often called LGTM).

For AI agent teams, Grafana is typically used alongside a dedicated LLM observability tool. It handles the infrastructure layer (container metrics, API latency, error rates) while something like Langfuse handles the LLM-specific tracing.

Last verified: 2026-04-28Verified by: editorial

docs ↗pricing ↗Spot a mistake? Open a PR →