@forge/llm

Provides a unified TypeScript/JavaScript interface for interacting with multiple LLM providers (OpenAI, Anthropic, etc.) through a standardized SDK. Routes requests to different providers via a pluggable adapter pattern, normalizing request/response formats across incompatible APIs so developers write once and switch providers without code changes.

Manages real-time token streaming from LLM providers with granular control over chunk processing, buffering, and backpressure. Implements stream event listeners that fire on token arrival, allowing developers to process partial responses incrementally without waiting for full completion, critical for low-latency user-facing applications.

Provides a templating system for constructing LLM prompts with variable substitution, conditional sections, and optional schema validation. Developers define prompt templates with placeholders that are filled at runtime, reducing prompt engineering boilerplate and enabling reusable, testable prompt patterns across applications.

Enables LLMs to invoke external functions by defining function schemas (name, description, parameters) that the LLM can understand and call. The SDK validates LLM-generated function calls against schemas, marshals arguments to correct types, and executes registered functions, creating a bridge between LLM reasoning and deterministic code execution.

Manages conversation history by maintaining a rolling window of messages sent to the LLM, automatically truncating or summarizing older messages to stay within token limits. Tracks message roles (user, assistant, system) and implements strategies for context optimization, preventing token budget overruns while preserving conversation coherence.

Implements automatic retry mechanisms for transient LLM API failures (rate limits, timeouts, temporary outages) using configurable exponential backoff strategies. Distinguishes between retryable errors (429, 503) and permanent failures (401, 404), preventing wasted retries on unrecoverable errors while maintaining resilience for temporary issues.

Provides hooks for logging and monitoring LLM requests and responses, enabling developers to track API usage, debug issues, and measure performance. Integrates with observability systems via callback functions that fire before/after API calls, capturing request parameters, response metadata, latency, and token usage without requiring code changes.

Leverages TypeScript generics to provide compile-time type safety for LLM responses, allowing developers to define expected response shapes and automatically validate/parse responses against those types. Uses runtime validation (likely JSON Schema or Zod) to ensure LLM outputs conform to expected structures, preventing runtime errors from malformed responses.