Yi (6B, 9B, 34B)

Generates coherent, contextually relevant text in English and Chinese using a transformer-based architecture trained on 3 trillion tokens of high-quality bilingual corpus. The model processes input text through attention mechanisms and produces token-by-token output via standard language modeling, with support for both single-turn and multi-turn conversation patterns through message-based API interfaces.

Exposes a REST API endpoint (http://localhost:11434/api/chat) accepting JSON payloads with message arrays in OpenAI-compatible format, enabling stateless HTTP-based inference without SDK dependencies. Requests are processed through Ollama's inference engine which manages model loading, tokenization, and streaming response delivery back to clients.

Provides `ollama run yi` command-line interface that automatically downloads, caches, and loads the specified model variant, then enters an interactive REPL-style chat loop where user input is tokenized, processed through the model, and streamed to stdout. Model lifecycle (loading, unloading, memory management) is handled transparently by Ollama.

Offers three pre-quantized model variants (6B, 9B, 34B parameters) distributed as separate GGUF artifacts, allowing users to select based on available hardware and latency requirements. Larger variants provide better quality/reasoning at cost of increased VRAM and inference latency; smaller variants enable deployment on resource-constrained devices. Selection is made via model tag (e.g., `ollama run yi:6b`).

Provides official Python and JavaScript client libraries (`ollama` package) that wrap the REST API with language-native abstractions, handling JSON serialization, streaming response parsing, and error handling. Developers call `ollama.chat()` with message arrays, receiving structured responses without manual HTTP handling.

Models are available through Ollama's cloud service (Ollama Pro/Max tiers) which provisions GPU infrastructure, manages model serving, and enforces concurrent model limits (1 for free, 3 for Pro, 10 for Max). Inference is billed on GPU compute time rather than tokens, with the same REST API and SDK interfaces as local deployment.

Processes input text through tokenization (converting text to token IDs), then generates output within a hard 4,096 token context window that includes both input and output tokens. The model maintains positional embeddings and attention mechanisms across this window, enabling coherent multi-turn conversations up to the token limit.

Ollama automatically downloads and caches model artifacts (GGUF files) on first use, storing them in a local directory (~/.ollama/models by default). Subsequent invocations load from cache without re-downloading. Model loading into VRAM is deferred until first inference request, enabling multiple models to coexist on disk with only active models consuming VRAM.