OpenLLMetry

Automatically intercepts and wraps LLM provider API calls (OpenAI, Anthropic, Bedrock, Cohere, etc.) using OpenTelemetry instrumentation hooks, capturing structured spans that include model parameters, prompt/completion content, token usage, and cost calculations without requiring manual span creation code. Uses provider-specific instrumentation packages that hook into HTTP clients or SDK methods to extract telemetry at the boundary layer.

Provides specialized instrumentation for AI orchestration frameworks (LangChain, LlamaIndex, Haystack) that automatically traces multi-step workflows including chain execution, agent reasoning loops, tool calls, and vector database queries. Captures framework-specific context like chain names, tool invocations, and retrieval steps as nested spans within a single trace, preserving the logical structure of complex AI workflows.

Emits OpenTelemetry metrics (histograms, counters, gauges) and events (structured logs) for LLM-specific KPIs including token counts, latency, cost, error rates, and model usage. Metrics are aggregated and exported separately from traces, enabling time-series analysis and alerting on LLM application health without requiring trace sampling.

Provides a prompt management system that captures prompt templates, versions, and parameters used in LLM calls, storing them as span attributes or in a separate prompt registry. Enables tracking of which prompt version was used for each LLM call, supporting reproducibility analysis and A/B testing of prompt variations.

Provides a mechanism to attach request-level context (user ID, session ID, request ID, custom tags) to all spans generated during request processing via association properties. Properties are stored in context variables and automatically added to all spans created within that context, enabling filtering and grouping of traces by request-level attributes without modifying instrumentation code.

Provides a centralized initialization API (Traceloop.init()) that configures all instrumentation, exporters, and span processors in a single call with environment variable or code-based configuration. Supports batch configuration of multiple instrumentation packages, exporter backends, and privacy controls, reducing boilerplate and enabling environment-specific configuration without code changes.

Automatically instruments vector database operations (Pinecone, Weaviate, Chroma, Milvus) to capture retrieval queries, result counts, similarity scores, and latency as spans within the broader application trace. Integrates with RAG pipelines to show which documents were retrieved and how they contributed to LLM context, enabling performance analysis of the retrieval component.

Provides Python decorators (@traceloop.span, @traceloop.workflow) that allow developers to manually create spans for custom application logic, associating them with the active trace context. Decorators automatically handle span lifecycle (start, end, exception recording) and propagate context to nested function calls, enabling developers to instrument their own code without directly using OpenTelemetry APIs.

Provides configurable privacy controls to redact or mask sensitive data in captured spans, including prompts, completions, and function arguments. Supports regex-based redaction rules, PII detection patterns, and per-span redaction policies that can be applied globally or selectively, ensuring compliance with data privacy requirements while maintaining observability.

Exports captured traces, metrics, and events to any OpenTelemetry-compatible backend (Datadog, Honeycomb, Grafana, Jaeger, Traceloop platform, etc.) using standard OTLP (OpenTelemetry Protocol) exporters. Supports multiple simultaneous exporters, batch export with configurable flush intervals, and fallback mechanisms for export failures, decoupling the instrumentation from specific observability platforms.

Handles OpenTelemetry span capture for streaming LLM responses (Server-Sent Events, token-by-token streaming) by buffering streamed tokens and updating span attributes incrementally as the stream completes. Captures final token counts and completion content after streaming finishes, avoiding span closure before response completion and ensuring accurate metrics for streaming workflows.

Maps LLM-specific telemetry data (model names, token counts, temperature, tool calls) to OpenTelemetry semantic conventions, ensuring consistent attribute naming and structure across different LLM providers and frameworks. Defines standard span attribute schemas for LLM calls, vector database queries, and framework operations, enabling downstream analysis and alerting based on standardized attribute names.

Automatically propagates OpenTelemetry context (trace ID, span ID, baggage) across Python async/await boundaries and thread pool execution, ensuring that nested async calls and background tasks maintain trace continuity. Uses context variables and thread-local storage to preserve trace context across execution contexts, enabling end-to-end tracing of complex concurrent workflows.

Provides an extensible span processor interface that allows developers to implement custom logic for transforming, filtering, or enriching spans before export. Processors are chained in a pipeline where each processor can modify span attributes, add events, filter spans, or perform custom logic, enabling use cases like dynamic sampling, cost calculation, or custom enrichment without modifying instrumentation code.