Firebase Genkit

Genkit implements flows as strongly-typed, composable pipeline primitives that enforce input/output schemas at definition time using a unified schema system across JavaScript, Go, and Python SDKs. Flows are registered in a central action registry and support middleware injection, tracing instrumentation, and streaming responses. The schema system performs bidirectional validation (input validation before execution, output validation after) and converts between provider-specific formats (e.g., OpenAI vs Anthropic message structures) transparently.

Genkit provides a domain-specific prompt templating language (dotprompt) that supports Handlebars-style variable interpolation, conditional blocks, and declarative tool/model binding without requiring code changes. Prompts are stored as .prompt files with YAML frontmatter (metadata, model config, tools) and template body, parsed at build time or runtime, and cached in memory. The system supports multimodal prompts (text + images/media) and context caching hints for expensive prompt prefixes, with automatic model-specific prompt formatting (e.g., system messages for OpenAI vs instruction blocks for Anthropic).

Genkit integrates context caching (supported by Anthropic Claude 3.5+ and Google AI) to cache expensive prompt prefixes (system messages, long documents, examples) and reuse them across requests. The system automatically applies cache control directives to prompt parts, tracks cache hit/miss rates, and calculates cost savings. Caching is transparent — the same prompt code works with or without caching support, degrading gracefully on unsupported providers. The developer UI shows cache statistics for debugging.

Genkit provides SDKs for JavaScript/TypeScript, Go, and Python with consistent APIs and abstractions across all three languages. Each SDK implements the same core concepts (flows, actions, schemas, tools, models) using language-native idioms (async/await in JS, goroutines in Go, async generators in Python). The monorepo structure ensures feature parity and synchronized releases. Shared patterns (schema validation, tracing, middleware) are implemented in each language independently rather than through a common runtime.

Genkit provides deployment integrations for Firebase (Cloud Functions, Firestore), Google Cloud Run, and Express.js-based servers. Flows can be exported as HTTP endpoints or Cloud Functions with automatic request/response serialization. The Firebase plugin enables Firestore integration for persistence, Cloud Storage for media, and Cloud Logging for observability. Deployment configurations are defined in code or via environment variables. The system handles cold starts, scaling, and monitoring through platform-native features.

Genkit provides chat abstractions for managing conversation state and message history. Chat sessions store messages (user, assistant, tool results) with metadata (timestamps, tool calls, model used). The system supports multi-turn conversations where each turn includes user input, model response, and optional tool calls. Sessions can be persisted to Firestore or custom storage. The chat flow handles message formatting for different providers (OpenAI conversation format, Anthropic message format, etc.) and maintains context across turns.

Genkit provides safety features including content filtering (blocking unsafe content), input/output validation, and configurable guardrails. The safety plugin integrates with provider-specific safety APIs (Google AI safety settings, Anthropic safety features) and custom safety checks. Safety policies can be defined per flow or globally. The system logs safety violations for monitoring and debugging. Safety checks are applied transparently without requiring code changes.

Genkit abstracts over multiple LLM providers (Google AI, Vertex AI, OpenAI, Anthropic, Ollama, etc.) through a unified GenerateRequest/GenerateResponse interface that normalizes model inputs and outputs. The generation pipeline handles provider-specific details: message format conversion, tool calling schemas, streaming token buffering, context caching directives, and safety filter configuration. Streaming is implemented via AsyncIterable (JS), channels (Go), and generators (Python) with automatic chunk buffering and error propagation. Context caching is transparently applied when available (Anthropic, Google AI) and silently degraded on other providers.

Genkit provides a RAG pipeline with pluggable embedders (Google AI, Vertex AI, OpenAI, Ollama), vector store integrations (Chroma, Firestore, custom), and rerankers (Cohere, custom). The retrieval flow accepts a query, embeds it using the configured embedder, searches the vector store with similarity metrics, optionally reranks results, and returns chunks with metadata. Indexing is a separate flow that chunks documents, embeds chunks, and stores them with metadata. The system supports hybrid search (keyword + semantic), metadata filtering, and custom chunk strategies (fixed-size, semantic, recursive).

Genkit implements tool calling through a schema-based function registry where tools are defined with JSON schemas for inputs/outputs, then automatically converted to provider-specific function calling formats (OpenAI function definitions, Anthropic tool_use blocks, Google AI function declarations). The generation pipeline handles tool dispatch: parsing model tool calls, validating arguments against schemas, executing the function, and feeding results back to the model. Tools can be async, return streaming results, or execute side effects. The system supports tool chaining (model calls tool A, receives result, calls tool B) and parallel tool execution.

Genkit provides an evaluation system for testing AI outputs against custom metrics (BLEU, ROUGE, semantic similarity, custom functions). Evaluators are defined as flows that take a generated output and reference data, compute a score or judgment, and return structured results. The framework supports batch evaluation (running evaluators over datasets), metric aggregation (mean, median, percentiles), and comparison across model variants. Evaluation results are stored with tracing metadata for debugging. The system integrates with the developer UI for visualization of evaluation runs.

Genkit provides a local developer UI (web-based dashboard) that displays all registered flows, models, and tools with their schemas. The UI allows running flows interactively with custom inputs, viewing execution traces (including model calls, tool invocations, latency), and inspecting intermediate results. Traces are collected via a telemetry server and include structured data (inputs, outputs, errors, timing) for each step. The reflection API exposes flow/tool metadata programmatically for tooling integration. The CLI provides commands to start the dev server, run tests, and manage the local environment.

Genkit abstracts multimodal inputs (text, images, PDFs, audio, video) through a unified Part structure that represents media with MIME types and data (base64, URL, or file path). The generation pipeline automatically converts parts to provider-specific formats: OpenAI vision_content blocks, Anthropic image/document blocks, Google AI inline_data, etc. The system supports mixed-media messages (text + multiple images + PDFs) and handles large files through streaming or URL references. Embedders can process multimodal content for RAG over documents with images.

Genkit enables structured output extraction by defining output schemas as JSON Schemas, then using model-specific structured output features (OpenAI JSON mode, Anthropic structured output, Google AI schema constraints) to guarantee the model returns valid JSON matching the schema. The generation pipeline validates the output against the schema before returning it to the caller. This enables reliable extraction of entities, relationships, and structured data from unstructured text without post-processing or retries.

Genkit provides a plugin architecture where custom implementations can be registered for models, embedders, vector stores, and other components. Plugins implement standard interfaces (Model, Embedder, VectorStore) and are registered in a global registry at startup. The framework includes built-in plugins for Google AI, Vertex AI, Firebase, and Google Cloud. Custom plugins can be published as npm/PyPI packages and installed like any dependency. The plugin system uses dependency injection to wire components together without tight coupling.