Llm As Judge Evaluation With Plain English Assertion Syntax

via “evaluation framework with llm-as-judge and custom metrics”

Open-source LLM observability — tracing, evaluation, OpenTelemetry, span analysis.

Unique: Integrated LLM-as-judge evaluation tightly coupled with trace data (no separate evaluation dataset needed) and experiment tracking, allowing direct comparison of evaluation scores across different LLM models or prompts tested in production

vs others: More integrated than standalone evaluation frameworks (Ragas, DeepEval) because evaluations run directly on Phoenix traces without data export; more flexible than rule-based metrics because judges can reason about semantic quality