genkit

Provides a consistent generate() interface across JavaScript/TypeScript, Go, and Python that abstracts away provider-specific APIs (OpenAI, Anthropic, Vertex AI, Ollama, etc.). Uses a Registry pattern to register model providers as plugins, enabling zero-code switching between LLM backends by changing configuration. Each language SDK implements the same semantic interface with native type systems (Zod for JS, native generics for Go/Python) for structured output validation.

Defines a Flow system that chains multiple AI operations (generation, retrieval, tool calls) into observable, deployable workflows using a declarative syntax. Flows are registered in the global Registry and can be invoked as HTTP endpoints, CLI commands, or from other flows. Each flow step is automatically instrumented with OpenTelemetry tracing, capturing inputs, outputs, latency, and errors for debugging and monitoring. Flows support branching, looping, and error handling through native language constructs (async/await in JS, goroutines in Go).

Enables LLMs to call external tools (functions, APIs, custom actions) through a schema-based function calling mechanism. Developers define tool schemas (input/output types) and register them as actions in the Registry. When a model supports function calling, Genkit automatically converts action schemas to the model's function calling format (OpenAI functions, Anthropic tools, Vertex AI function calling). The framework handles tool invocation, result parsing, and re-prompting the model with results. Supports both single-turn tool calls and multi-turn agentic loops.

Genkit flows can be deployed as HTTP endpoints to serverless platforms (Google Cloud Functions, AWS Lambda, Firebase Functions) or containerized services (Docker, Kubernetes). The framework provides deployment helpers and examples for each platform. Flows are automatically exposed as REST endpoints with OpenAPI documentation. Environment-specific configuration (API keys, model selection) is handled through environment variables or configuration files. Observability (tracing, metrics) is integrated with cloud provider observability services (Google Cloud Trace, CloudWatch, etc.).

Enables flows and actions defined in one language (e.g., Go) to be called from another language (e.g., JavaScript) through HTTP or gRPC bridges. Flows are exposed as HTTP endpoints with JSON request/response bodies, and schemas are shared via JSON schema format. gRPC support (in development) will provide typed, efficient cross-language calls. This enables polyglot architectures where different services use different languages but share AI workflows.

Enforces strict typing and validation on LLM outputs using language-native schema systems: Zod for JavaScript/TypeScript, Go structs with reflection, and Python dataclasses. Schemas are registered in the Registry and used to validate model responses before returning to the caller. Supports JSON schema generation for OpenAI/Anthropic function calling, enabling models to produce structured outputs that are automatically parsed and validated. Schemas are shared across language boundaries via JSON schema interchange format.

Implements a global Registry that acts as a service locator for models, embedders, retrievers, evaluators, and custom actions. Plugins register implementations at startup, and the framework resolves them by name at runtime. Plugins can be first-party (Google AI, Vertex AI, Firebase) or third-party (OpenAI, Anthropic, Ollama, Pinecone, Chroma). Each plugin exports a standard interface (e.g., ModelProvider, EmbedderProvider) that the core framework calls. Plugins can depend on other plugins (e.g., a RAG plugin depends on embedders and retrievers).

Provides a complete RAG (Retrieval-Augmented Generation) system with pluggable components: embedders (convert text to vectors), retrievers (query vector stores), and rerankers (re-score retrieved documents). Embedders are registered plugins that support multiple providers (Google Vertex AI, OpenAI, Ollama). Retrievers query vector stores (Pinecone, Chroma, Firebase Vector Store, custom implementations) and return ranked documents. Rerankers use cross-encoder models to improve retrieval quality. The framework handles chunking, embedding, storage, and retrieval orchestration; developers compose these into RAG flows.

Introduces a .prompt file format (YAML-like syntax) for defining prompts with metadata, input schemas, and output schemas in a version-controllable text format. Dotprompt files are compiled into Flow-like objects that can be invoked via the CLI or SDK. Supports prompt templating with variable substitution, conditional sections, and multi-turn conversation templates. Prompts are registered in the Registry and can be referenced by name, enabling prompt reuse across applications and easy A/B testing by swapping prompt files.

Automatically instruments all Genkit operations (generation, retrieval, flow execution) with OpenTelemetry spans, capturing inputs, outputs, latency, token counts, and errors. Traces are sent to a local Developer UI (included in Genkit CLI) or external backends (Jaeger, Datadog, Google Cloud Trace). Metrics include token usage, latency percentiles, and error rates. No manual instrumentation required; tracing is transparent to application code. Traces are queryable and filterable in the Developer UI, enabling debugging and performance analysis.

Provides a framework for evaluating AI workflows using built-in metrics (BLEU, ROUGE, exact match, semantic similarity) and custom evaluators. Evaluators are registered plugins that take a flow output and return a score or judgment. Supports batch evaluation of flows against test datasets, with results aggregated and visualized. Evaluation runs are traced and stored, enabling comparison across prompt/model/parameter changes. Custom evaluators can use LLMs (e.g., 'does this response answer the question?') or deterministic logic.

Provides a command-line interface (genkit CLI) for local development, including: starting a local telemetry server that captures traces and metrics, running flows from the command line with JSON input, testing prompts with different models, and generating boilerplate code. The telemetry server exposes a web UI for visualizing traces, metrics, and flow execution history. CLI commands are auto-generated from registered flows and actions, enabling zero-code testing of AI workflows.

Supports multimodal inputs and outputs including text, images, and video. The generation API accepts image inputs (base64, URLs, or file paths) for vision models (GPT-4V, Gemini, Claude 3 Vision). Responses can include generated images where supported by the model. Content is abstracted through a unified Content type that handles serialization across language boundaries. Image processing utilities (resizing, format conversion) are available through plugins.