Weights & Biases

Logs training metrics, validation scores, and custom KPIs to a centralized cloud dashboard via the Python SDK's `run.log()` API, which batches metrics and syncs asynchronously to W&B servers. Supports scalar values, histograms, confusion matrices, and media (images, audio, video). Real-time visualization updates as training progresses, enabling live monitoring without polling or manual refresh.

Automates hyperparameter search by defining a sweep configuration (parameter ranges, search strategy) and launching parallel training jobs across local or cloud workers. Supports grid search, random search, and Bayesian optimization via the W&B Sweeps API. The platform manages job scheduling, monitors metrics, and suggests next hyperparameters based on prior runs, reducing manual tuning effort.

Enables on-premise deployment of W&B using Docker, allowing organizations to run the full W&B platform on their own infrastructure. Supports air-gapped environments and provides options for customer-managed encryption keys. Includes local server startup via `wandb server start` command and supports scaling to multiple nodes for high availability.

Provides serverless infrastructure for fine-tuning models using reinforcement learning, abstracting away compute provisioning and scaling. Users define a fine-tuning job with a base model, reward function, and dataset, and W&B handles training on managed hardware. Integrates with W&B's experiment tracking to log RL metrics (rewards, policy loss, value loss) and model checkpoints.

Logs and visualizes multi-modal artifacts (images, audio, video, 3D point clouds) alongside metrics and configs. Supports automatic media gallery rendering in the dashboard, enabling visual inspection of model outputs (e.g., generated images, segmentation masks, audio spectrograms). Integrates with metric logging to correlate media with performance metrics.

Enables team collaboration through shared projects with granular permission controls (view, edit, admin). Team members can view shared runs, compare experiments, and comment on results. Supports role-based access control (RBAC) for enterprise teams, with options to restrict access by project or workspace. Integrates with SSO (SAML, OAuth) for enterprise authentication.

Captures trained models as versioned artifacts in the W&B Registry using `run.log_artifact()`, storing model files (PyTorch `.pt`, TensorFlow SavedModel, ONNX, etc.) alongside metadata (training config, metrics, timestamp). Tracks lineage — which dataset, code version, and hyperparameters produced each model — enabling reproducibility and rollback. Models are immutable once logged and can be retrieved by version alias (e.g., 'production', 'latest').

Logs datasets as versioned artifacts in the W&B Registry, capturing data snapshots alongside metadata (row count, schema, statistics). Tracks which datasets were used in each training run, enabling reproducibility and data lineage analysis. Supports large datasets via chunked uploads and provides a dataset browser for exploring versions and statistics without downloading full files.

Traces LLM API calls, document retrieval, and agent steps using the Weave SDK (`@weave.op()` decorator). Captures prompts, completions, latency, token counts, and costs for each LLM call. Automatically instruments popular LLM libraries (OpenAI, Anthropic, Ollama) and provides a trace browser for debugging multi-step LLM applications. Traces are stored in W&B and queryable via SQL-like interface.

Evaluates LLM application outputs using custom scorer functions defined in Python. Scorers can be deterministic (e.g., exact match, BLEU score) or LLM-based (e.g., using GPT-4 to judge quality). Runs evaluations across datasets and logs results alongside traces, enabling systematic quality assessment. Supports batch evaluation and integrates with W&B's experiment tracking for comparing evaluation metrics across runs.

Provides a web-based dashboard for comparing metrics, configs, and artifacts across multiple training runs. Supports filtering by hyperparameters, metrics ranges, and tags; grouping by config values; and exporting results as tables or plots. Enables side-by-side comparison of run details (config, metrics, artifacts) and identification of best-performing configurations without manual spreadsheet work.

Manages model lifecycle through a centralized registry with semantic versioning and aliases (e.g., 'production', 'staging', 'best'). Models can be promoted between stages by updating aliases without re-uploading files. Supports model cards with documentation, links to training runs, and evaluation results. Enables teams to coordinate model deployments and track which version is currently in production.

Logs LLM prompts as versioned artifacts in the W&B Registry, capturing prompt text, variables, and metadata (model, temperature, max_tokens). Enables teams to version prompts alongside experiments and track which prompt version was used in each run. Supports prompt templates with variable substitution and provides a prompt browser for exploring versions and comparing changes.

Integrates with CI/CD pipelines to trigger training jobs on code commits, log results to W&B, and send alerts (Slack, email) when metrics exceed thresholds or runs fail. Supports webhook-based triggers and can be integrated with GitHub Actions, GitLab CI, or custom CI systems. Enables automated model retraining and quality gates without manual intervention.