phoenix

Accepts distributed traces from LLM applications through a dedicated gRPC server listening on port 4317, implementing the OpenTelemetry Protocol (OTLP) specification. Traces are parsed from protobuf messages, validated, and persisted to PostgreSQL or SQLite with automatic schema migrations. Supports multi-language instrumentation (Python, TypeScript, Go, etc.) without requiring application code changes when using auto-instrumentation libraries.

Exposes a Strawberry GraphQL API (on port 6006) that allows complex queries over ingested traces using a schema-driven approach. Queries support filtering by span attributes, trace duration, status codes, and custom dimensions; supports pagination, sorting, and aggregation operations. The GraphQL layer translates queries into optimized SQL against the trace database, enabling efficient retrieval of trace subsets for analysis and debugging without loading entire trace datasets into memory.

Provides a database abstraction layer supporting both PostgreSQL (production) and SQLite (development/single-instance) backends, with automatic schema migrations managed by Alembic. The abstraction uses SQLAlchemy ORM for database operations, enabling schema changes without manual SQL. Supports connection pooling, transaction management, and query optimization for both backends. Database schema includes tables for spans, traces, evaluations, datasets, and annotations with appropriate indexes for common query patterns.

Provides a command-line interface for starting the Phoenix server locally, managing database connections, and exporting trace data. CLI commands support starting the server with custom configuration (port, database URL, authentication), running database migrations, exporting traces to CSV/JSON, and importing datasets. The CLI uses Click framework for command definition and supports both interactive and scripted usage.

Provides a React-based web UI that visualizes trace execution flows as interactive diagrams showing span hierarchies, timing, and status. The UI displays spans as nodes with parent-child relationships, color-coded by status (success, error, pending), and includes timeline visualization showing span duration and overlap. Users can click spans to view detailed attributes, logs, and events; filter traces by attributes; and navigate between related traces. The frontend communicates with the backend via GraphQL API.

Implements authentication and authorization mechanisms (details in DeepWiki) supporting role-based access control (RBAC) for multi-tenant deployments. Users can be assigned roles (admin, analyst, viewer) with corresponding permissions for reading/writing traces, evaluations, and datasets. Authentication supports API keys and optional OAuth2/OIDC integration. Authorization is enforced at the API layer (GraphQL and REST) and database layer to prevent unauthorized data access.

Provides a Python-based evaluation system (arize-phoenix-evals package) that runs structured evaluators against LLM outputs to measure quality, correctness, and safety. Evaluators are composable functions that accept input/output pairs and return structured scores or classifications. The framework supports both built-in evaluators (hallucination detection, relevance scoring, toxicity detection) and custom user-defined evaluators; results are stored as annotations on spans and can be aggregated across datasets for statistical analysis.

Provides a prompt management system that stores prompt templates with version history, enabling A/B testing and experimentation. Prompts are stored in the database with metadata (model, parameters, tags) and can be retrieved by version or tag. The system tracks which prompt version was used for each LLM call via span attributes, allowing correlation between prompt changes and output quality metrics. Experiments can be defined to compare multiple prompt versions against the same dataset of inputs.

Provides auto-instrumentation libraries (arize-phoenix-otel) that automatically capture spans for popular LLM frameworks (LangChain, LlamaIndex, OpenAI SDK) without requiring manual span creation code. Uses Python decorators and context managers to wrap framework calls, extracting relevant metadata (model name, tokens, latency) and creating spans automatically. Supports both synchronous and asynchronous execution; integrates with OpenTelemetry context propagation for distributed tracing across service boundaries.

Provides a web-based playground interface (React frontend) for testing LLM prompts interactively with real-time execution. Users can write prompts, select models (OpenAI, Anthropic, local), adjust parameters (temperature, max_tokens), and execute calls with immediate feedback. Playground sessions are persisted and linked to traces, enabling correlation between playground experiments and production traces. Supports multi-turn conversations and prompt templating with variable substitution.

Enables users to attach feedback, ratings, and custom annotations to spans after execution, supporting both programmatic and UI-based annotation. Feedback can be numeric scores (0-1), categorical labels, or free-form text; annotations are stored in the database and linked to specific spans. Supports batch annotation operations for applying feedback to multiple spans matching a query. Feedback is queryable via GraphQL, enabling analysis of annotated spans and correlation with evaluation results.

Provides a dataset management system for storing input/output pairs and running experiments that execute LLM applications against datasets to measure performance. Datasets can be created from historical traces, uploaded as CSV/JSON, or defined programmatically. Experiments execute a specified LLM application (chain, agent, etc.) against each dataset row, capture outputs, run evaluations, and aggregate metrics. Results are stored with full traceability to input data and evaluation logic.

Exposes a REST API (alongside GraphQL) with auto-generated OpenAPI/Swagger documentation for programmatic access to traces, evaluations, and datasets. REST endpoints support standard CRUD operations and filtering via query parameters. The API is fully documented with interactive Swagger UI, enabling API discovery and testing without external tools. Supports both JSON request/response format and streaming responses for large result sets.

Implements a Model Context Protocol (MCP) server that exposes Phoenix capabilities to Claude and other AI assistants, enabling natural language interaction with traces, evaluations, and datasets. The MCP server translates natural language requests into Phoenix API calls, returning results in a format optimized for LLM consumption. Supports querying traces, running evaluations, creating datasets, and executing experiments through conversational interfaces.