@tanstack/ai

Provides a standardized API layer that abstracts over multiple LLM providers (OpenAI, Anthropic, Google, Azure, local models via Ollama) through a single `generateText()` and `streamText()` interface. Internally maps provider-specific request/response formats, handles authentication tokens, and normalizes output schemas across different model APIs, eliminating the need for developers to write provider-specific integration code.

Implements streaming text generation with built-in backpressure handling, allowing applications to consume LLM output token-by-token in real-time without buffering entire responses. Uses async iterators and event emitters to expose streaming tokens, with automatic handling of connection drops, rate limits, and provider-specific stream termination signals.

Provides React hooks (useChat, useCompletion, useObject) and Next.js server action helpers for seamless integration with frontend frameworks. Handles client-server communication, streaming responses to the UI, and state management for chat history and generation status without requiring manual fetch/WebSocket setup.

Provides utilities for building agentic loops where an LLM iteratively reasons, calls tools, receives results, and decides next steps. Handles loop control (max iterations, termination conditions), tool result injection, and state management across loop iterations without requiring manual orchestration code.

Enables LLMs to request execution of external tools or functions by defining a schema registry where each tool has a name, description, and input/output schema. The SDK automatically converts tool definitions to provider-specific function-calling formats (OpenAI functions, Anthropic tools, Google function declarations), handles the LLM's tool requests, executes the corresponding functions, and feeds results back to the model for multi-turn reasoning.

Allows developers to request LLM outputs in a specific JSON schema format, with automatic validation and parsing. The SDK sends the schema to the provider (if supported natively like OpenAI's JSON mode or Anthropic's structured output), or implements client-side validation and retry logic to ensure the LLM produces valid JSON matching the schema.

Provides a unified interface for generating embeddings from text using multiple providers (OpenAI, Cohere, Hugging Face, local models), with built-in integration points for vector databases (Pinecone, Weaviate, Supabase, etc.). Handles batching, caching, and normalization of embedding vectors across different models and dimensions.

Manages conversation history with automatic context window optimization, including token counting, message pruning, and sliding window strategies to keep conversations within provider token limits. Handles role-based message formatting (user, assistant, system) and automatically serializes/deserializes message arrays for different providers.

Provides a templating system for constructing prompts with variable substitution, conditional sections, and formatting helpers. Supports both simple string interpolation and more complex template engines, allowing developers to define reusable prompt patterns with placeholders for dynamic content like user input, context, or retrieved documents.

Calculates token counts for prompts and completions using provider-specific tokenizers (tiktoken for OpenAI, Anthropic's tokenizer, etc.), and estimates API costs based on model pricing. Supports both pre-request estimation (for budget planning) and post-request actual counts (for billing).

Implements automatic retry logic for transient failures (rate limits, timeouts, temporary service outages) with configurable exponential backoff strategies. Distinguishes between retryable errors (429, 503) and permanent failures (401, 404), and provides hooks for custom error handling and recovery strategies.

Automatically generates TypeScript types from JSON schemas used in structured output or tool definitions, providing compile-time type safety for LLM responses and tool parameters. Integrates with the SDK's structured output and tool calling to ensure type consistency between schema definitions and application code.