Harbor

Orchestrates multiple LLM backend services (e.g., Ollama, vLLM, LocalAI) within isolated Docker containers, exposing unified API endpoints through a single CLI invocation. Uses Docker Compose under the hood to manage container lifecycle, networking, and service dependencies, eliminating manual container configuration and port mapping complexity.

Exposes a standardized HTTP API interface across heterogeneous LLM backends (Ollama, vLLM, LocalAI, etc.) by implementing adapter patterns that normalize request/response schemas. Routes incoming requests to the appropriate backend container based on model name or explicit routing rules, abstracting away backend-specific API differences.

Bundles and containerizes web UI applications (e.g., Open WebUI, Gradio interfaces) alongside LLM backends, exposing them on standard ports (typically 3000, 8000) with automatic service discovery. Manages UI container lifecycle and networking configuration so developers access the UI immediately after running the CLI command without additional setup.

Provisions a complete local LLM development environment (backends, APIs, UIs, supporting services) with a single CLI command that reads a declarative configuration file. Internally composes Docker Compose manifests, manages container startup order via dependency declarations, and handles port allocation and volume mounting for model persistence.

Manages Docker volumes for LLM model storage, ensuring downloaded models persist across container restarts and are shared between multiple backend instances. Handles volume mounting configuration automatically so developers don't manually specify mount paths, and supports model caching strategies to avoid re-downloading large model files.

Monitors containerized service health by checking endpoint availability and response times, providing real-time status feedback via CLI output or dashboard. Implements health check patterns (HTTP probes, port availability checks) to detect when services are ready to accept requests, preventing premature client connections to initializing backends.

Provides declarative YAML configuration files that specify which LLM backends, UIs, and supporting services to run, with options for customizing ports, environment variables, resource limits, and service dependencies. Parses configuration files and generates corresponding Docker Compose manifests, allowing developers to version-control infrastructure as code without writing Docker directly.

Manages model downloads and caching across multiple LLM backends (Ollama, vLLM, LocalAI) with different model formats and storage conventions. Handles backend-specific model pulling logic (e.g., Ollama's model registry vs vLLM's HuggingFace integration) transparently, allowing developers to specify models declaratively without understanding each backend's model management system.