ai-cost-meter

Provides a unified API for counting tokens across 6+ LLM providers (OpenAI, Anthropic, Gemini, Mistral, Groq, DeepSeek) by wrapping each provider's native tokenization logic or implementing compatible algorithms. Uses provider-specific token encoders (e.g., tiktoken for OpenAI, claude-tokenizer for Anthropic) behind a normalized interface, allowing developers to swap providers without changing token-counting code. Handles model-specific tokenization differences (e.g., different BPE vocabularies, special token handling) transparently.

Automatically calculates monetary cost for each LLM API request by multiplying token counts (input + output) by provider-specific pricing rates. Maintains an internal pricing table for each provider and model, updated to reflect current pricing. Supports both streaming and non-streaming requests, calculating costs incrementally as tokens arrive. Returns cost breakdowns (prompt cost, completion cost, total) alongside token counts, enabling per-request cost visibility without manual billing API queries.

Tracks token usage and cost for streaming LLM responses by intercepting and counting tokens as they arrive in chunks, rather than waiting for the complete response. Maintains running totals of prompt tokens, completion tokens, and cost as the stream progresses. Works by wrapping streaming response handlers or middleware to parse token counts from provider-specific stream metadata (e.g., OpenAI's usage field in stream deltas). Enables cost visibility before streaming completes, supporting early termination or cost-aware stream handling.

Integrates with LLM client libraries (OpenAI SDK, Anthropic SDK, etc.) via middleware or wrapper patterns to automatically inject cost tracking into every API call without modifying application code. Intercepts requests before they're sent and responses after they're received, extracting token counts and calculating costs transparently. Supports both callback-based and promise-based middleware patterns, and works with async/await code. Accumulates costs across multiple requests, enabling application-level cost aggregation and reporting.

Aggregates costs across multiple requests and provides structured reports broken down by time period, model, provider, or custom categories. Maintains running totals and supports queries like 'total cost in last hour', 'cost by model', 'cost by provider'. Implements in-memory cost accumulation with optional export to JSON or CSV for external analysis. Supports custom tagging of requests (e.g., by user, feature, or endpoint) to enable cost attribution and chargeback scenarios.

Manages a versioned pricing table for all supported models across all providers, allowing developers to update rates as providers change pricing. Supports both built-in default pricing (updated with library releases) and custom pricing overrides for specific models or providers. Implements a configuration API to set custom rates programmatically, and supports loading pricing from external sources (JSON files, environment variables, or APIs). Tracks pricing version to enable cost recalculation with historical rates if needed.

Implements budget tracking and enforcement by monitoring cumulative costs against user-defined spending limits. Supports per-request budget checks (reject requests that would exceed budget), per-session limits, and per-time-period limits (e.g., daily, monthly). Provides callback hooks or event emitters to trigger alerts when costs approach or exceed thresholds. Integrates with cost tracking to enable real-time budget enforcement without external services.

Analyzes historical cost and token usage data to recommend the most cost-efficient models for specific use cases. Calculates efficiency metrics (cost per token, cost per request, tokens per dollar) for each model and provides rankings. Supports filtering by quality constraints (e.g., 'recommend models with >90% quality score') or latency constraints. Enables A/B testing by comparing costs across models for the same prompts or use cases.