Weights & Biases API

Logs and visualizes ML experiment metrics in real-time by instrumenting training loops with the Python SDK, storing timestamped metric data in W&B's cloud backend, and rendering interactive dashboards with filtering, grouping, and comparison views. Supports custom charts, parameter sweeps, and historical run comparison to identify optimal hyperparameters and model configurations across training iterations.

Defines and executes automated hyperparameter search using Bayesian optimization, grid search, or random search by specifying parameter ranges and objectives in a YAML config file, then launching W&B Sweep agents that spawn parallel training jobs, evaluate results, and iteratively suggest new parameter combinations. Integrates with experiment tracking to automatically log each trial's metrics and select the best-performing configuration.

Allows users to define custom metrics and visualizations by combining logged data (scalars, histograms, images) into interactive charts without code. Supports metric aggregation (e.g., rolling averages), filtering by hyperparameters, and custom chart types (scatter, heatmap, parallel coordinates). Charts are embedded in reports and shared with teams.

Generates shareable reports combining experiment results, charts, and analysis into a single document that can be embedded in web pages or shared via link. Reports are interactive (viewers can filter and zoom charts) and automatically update when underlying experiment data changes. Supports markdown formatting, custom sections, and team-level sharing with granular permissions.

Stores and versions model checkpoints, datasets, and training artifacts as immutable objects in W&B's artifact registry with automatic lineage tracking, enabling reproducible model retrieval by version tag or commit hash. Supports model promotion workflows (e.g., 'staging' → 'production'), dependency tracking across artifacts, and integration with CI/CD pipelines to gate deployments based on model performance metrics.

Traces execution of LLM applications (prompts, model calls, tool invocations, outputs) through W&B Weave by instrumenting code with trace decorators, capturing full call stacks with latency and token counts, and evaluating outputs against custom scoring functions. Supports side-by-side comparison of different prompts or models on the same inputs, cost estimation per request, and integration with LLM evaluation frameworks.

Provides an interactive web-based playground for testing and comparing multiple LLM models (via W&B Inference or external APIs) on identical prompts, displaying side-by-side outputs, latency, token counts, and costs. Supports prompt templating, parameter variation (temperature, top-p), and batch evaluation across datasets to identify which model performs best for specific use cases.

Executes serverless reinforcement learning and fine-tuning jobs for LLM post-training via W&B Training, supporting multi-turn agentic tasks and automatic GPU scaling. Integrates with frameworks like ART and RULER for reward modeling and policy optimization, handles job orchestration without manual infrastructure management, and tracks training progress with automatic metric logging.

Provides an OpenAI-compatible API endpoint for running inference on foundation models via W&B Inference, supporting standard OpenAI request/response formats (chat completions, embeddings) with automatic usage tracking and integration with W&B Weave for tracing. Enables drop-in replacement of OpenAI API calls with W&B-hosted models while maintaining compatibility with existing client libraries.

Integrates with CI/CD pipelines to automatically trigger model training, evaluation, and deployment based on code commits or schedule, with conditional gating that blocks deployment if model metrics fall below thresholds. Supports custom automation rules (e.g., 'deploy only if accuracy > 95%'), Slack/email alerts on job completion, and integration with GitHub Actions or other CI/CD platforms.

Enables multi-user team access to experiments, models, and reports with role-based access control (RBAC) and granular permissions. Supports shared workspaces, team projects, and audit logs tracking who accessed or modified what and when. Available on Pro tier and above with features like team invitations, permission management, and activity history.

Versions datasets as immutable artifacts with automatic lineage tracking showing which dataset versions were used in which training runs and produced which models. Supports dataset comparison (schema changes, row counts), integration with data validation frameworks, and programmatic dataset retrieval by version tag. Enables reproducibility by capturing the exact data used for each model.