harbor

Harbor abstracts Docker Compose through a CLI system that dynamically resolves and merges compose files based on requested services, hardware capabilities (GPU detection via has_nvidia()), and user profiles. The orchestration engine uses a 'Lego-like' modular approach where each service is a pluggable module, with the core harbor.sh script handling service lifecycle management through functions like run_up() for starting services with flags like --tail or --open. Configuration is merged via compose_with_options() which combines base compose files with service-specific overrides.

Harbor provides a dedicated env_manager() function in harbor.sh (lines 1257-1350) that handles get, set, and list operations for the .env file, enabling users to configure services through environment variables without editing files directly. The system supports profile-based configuration through profiles/default.env, allowing users to switch between different hardware profiles, model selections, and service configurations. Configuration changes are persisted to the .env file and automatically loaded on subsequent service starts.

Harbor implements automatic service dependency resolution through its compose file merging system (compose_with_options function in harbor.sh lines 402-520). When a user requests a service, Harbor analyzes service metadata to identify required dependencies, then merges the appropriate compose files in dependency order. This ensures that if a user enables a RAG service, the required vector database and embedding model services are automatically started. The system prevents circular dependencies and validates that all required services are available before starting the stack.

Harbor includes version synchronization logic (routines/models/hf.ts, routines/models/llamacpp.ts) that manages model versions across different inference backends. The system tracks which models are available in each backend (Ollama, llama.cpp, HuggingFace), handles model downloads and caching, and ensures version consistency when switching backends. Users can specify model versions through environment variables, and Harbor automatically downloads the correct version for the selected backend. The system supports model quantization variants (e.g., 4-bit, 8-bit) and automatically selects the appropriate variant based on available hardware.

Harbor includes observability and evaluation services that enable monitoring of LLM inference (latency, throughput, token usage) and evaluation of model outputs (quality metrics, safety checks). These services integrate with Harbor Boost to collect metrics from every LLM request, and provide dashboards and APIs for analyzing performance. The system supports custom evaluation modules that can be plugged into the request/response pipeline to assess output quality, detect hallucinations, or check for safety violations.

Harbor provides a framework for creating custom services and Harbor Boost modules that extend the platform's capabilities. Custom services are defined as Docker Compose services with metadata declarations, while Boost modules are Python classes that hook into the LLM request/response pipeline. The framework includes templates, documentation, and integration testing utilities to help developers build and test custom extensions. Custom services are automatically discovered and integrated into the service catalog, and Boost modules can be enabled through configuration without modifying Harbor core.

Harbor maintains a curated service catalog (app/src/serviceMetadata.ts lines 8-103) with over 50 AI-related services organized by Harbor Service Tags (HST). Each service has associated metadata including category (LLM backends, frontends, satellite services, RAG tools), dependencies, port mappings, and integration patterns. The catalog enables users to discover available services, understand their purpose, and understand how they integrate with other services in the stack. Service metadata drives the dynamic composition of Docker Compose files and the Harbor Desktop App's UI.

Harbor Boost is an optimizing LLM proxy layer (services/boost/pyproject.toml) built with a Python-based module system that intercepts LLM requests and applies transformations such as prompt optimization, response caching, cost tracking, and multi-provider routing. The module system allows users to create custom Boost modules that hook into the request/response pipeline. Boost acts as a middleware between client applications and inference backends (Ollama, llama.cpp, OpenAI), enabling advanced features like artifact generation and visualization without modifying the underlying models.

Harbor includes hardware detection logic (has_nvidia() function in harbor.sh lines 262-264) that automatically detects NVIDIA GPUs and applies GPU-accelerated compose files when available. The system maintains separate compose file variants for CPU-only and GPU-accelerated deployments, allowing the same service configuration to run optimally on different hardware without user intervention. If GPU detection fails or GPUs are unavailable, Harbor automatically falls back to CPU-only compose files, ensuring services remain functional across heterogeneous hardware.

Harbor integrates with the Model Context Protocol (MCP) to enable LLM agents to call external tools and services as function calls. Services in Harbor expose MCP-compatible tool schemas that agents can discover and invoke. The integration allows agents to use tools like web search (SearXNG), code execution, file operations, and custom service APIs without hardcoding tool logic into the agent. Tool schemas are defined in service metadata and automatically registered with the MCP server when services start.

Harbor includes a Tauri-based desktop application (app/src-tauri/tauri.conf.json) that provides a graphical interface for managing services, viewing service status, configuring settings, and monitoring logs. The app is built with Tauri (Rust backend, TypeScript/React frontend) for cross-platform compatibility and minimal resource overhead. The app communicates with the Harbor CLI and Docker daemon to provide real-time service status, enable one-click service start/stop, and visualize service dependencies and health.

Harbor supports multiple LLM inference backends including Ollama (local model serving), llama.cpp (CPU-optimized inference), and cloud providers (OpenAI, Anthropic) through a unified interface. Each backend is a pluggable service with its own compose file and configuration. Harbor's LiteLLM Gateway (satellite service) provides a unified API endpoint that routes requests to the selected backend, allowing applications to switch backends without code changes. Model selection and backend routing are configured through environment variables and Harbor Boost modules.

Harbor includes satellite services for RAG workflows, including vector databases, document indexers, and retrieval engines. Services like SearXNG provide web search integration for RAG, while knowledge base services enable semantic search over local documents. Harbor's service composition automatically wires RAG services together (e.g., connecting a document indexer to a vector database), and Harbor Boost modules can intercept LLM requests to augment prompts with retrieved context. The system supports multiple RAG patterns including semantic search, hybrid search, and multi-hop retrieval.

Harbor includes pre-configured web UI frontends including Open WebUI (chat interface for LLMs) and ComfyUI (node-based workflow builder for image generation). These frontends are automatically wired to the appropriate inference backends through Harbor's service composition. Open WebUI connects to LLM backends (Ollama, llama.cpp, cloud providers) through the LiteLLM Gateway, while ComfyUI connects to image generation models. The frontends are accessible through a unified landing page service that provides navigation and service discovery.