label-studio

Provides a declarative XML-based labeling interface system that dynamically generates annotation UIs for images, text, audio, video, and time-series data without code changes. The frontend architecture uses React components that parse Label Studio's custom XML schema to render task-specific controls (bounding boxes, classifications, relations, etc.), enabling teams to define complex annotation workflows through configuration rather than custom development.

Implements a pluggable next-task selection algorithm (documented in label_studio/projects/functions/next_task.py) that determines which task to present to annotators based on project configuration, annotation progress, and optional ML model predictions. The system supports sequential ordering, random sampling, and active learning strategies that prioritize uncertain predictions from integrated ML models, reducing annotation effort for model-in-the-loop workflows.

Uses Celery task queue (documented in Advanced Topics: Background Jobs and Tasks) to handle long-running operations asynchronously, including batch exports, model predictions, and data syncs. Jobs are queued with status tracking, allowing users to monitor progress and retrieve results without blocking the web interface. Supports job retry logic and failure notifications.

Implements feature flag system (documented in Advanced Topics: Managing Feature Flags) allowing teams to enable/disable features per-organization or per-user without code deployment. Flags are stored in database and evaluated at runtime, supporting gradual rollouts, A/B testing, and quick rollback if issues are detected. Integrates with frontend and backend to control feature visibility.

Exposes comprehensive REST API (documented in API Reference section) covering Projects, Tasks, Annotations, Users, Organizations, Storage, and Data Manager endpoints. API uses standard HTTP methods (GET, POST, PATCH, DELETE) with JSON request/response bodies, supporting filtering, pagination, and bulk operations. Authentication via API tokens enables external tools and scripts to automate Label Studio workflows.

Provides Docker image and Kubernetes manifests (documented in Build and Deployment section) for containerized deployment with environment-based configuration. Supports PostgreSQL backend, Redis for caching, and Celery workers, with Helm charts for simplified Kubernetes deployment. Configuration is managed via environment variables, enabling teams to deploy Label Studio across development, staging, and production environments with minimal code changes.

Provides abstraction layer (label_studio/io_storages/) supporting S3, Google Cloud Storage, Azure Blob Storage, and local filesystem for bidirectional data sync. Tasks are imported from cloud buckets on-demand, and completed annotations are exported back to configured storage with automatic format conversion, enabling seamless integration with ML training pipelines without manual file transfers.

Implements organization and user management (label_studio/organizations/, label_studio/users/) with role-based access control (RBAC) supporting Admin, Manager, Annotator, and Reviewer roles at both organization and project levels. Uses Django's permission system with custom mixins to enforce access policies, enabling teams to isolate projects by department, control who can export data, and audit annotation activity across organizational boundaries.

Provides ML Integration subsystem (label_studio/ml/) that accepts predictions from external models via REST API, stores them as pre-annotations, and feeds uncertainty scores back to the next-task algorithm for active learning. Supports both batch prediction (pre-annotate entire project) and online prediction (score tasks as they're created), with automatic format conversion between Label Studio's internal representation and model-specific output formats.

Exports completed annotations in multiple formats (JSON, COCO, Pascal VOC, YOLO, IOB/BIO, VTT, CSV) via configurable export pipelines (label_studio/tasks/serializers.py). Each format has a dedicated serializer that transforms Label Studio's internal annotation representation into domain-specific schemas, with support for filtering by annotator, agreement score, or annotation status before export.

Calculates inter-annotator agreement metrics (Kappa, F1, Precision/Recall) when multiple annotators label the same task, storing agreement scores in the database for filtering and quality assessment. The Data Manager subsystem (label_studio/data_manager/) provides UI for visualizing agreement distributions and identifying low-agreement tasks for review or re-annotation.

Provides Data Manager subsystem (label_studio/data_manager/api.py) with SQL-based filtering, full-text search, and faceted navigation across tasks and annotations. Supports complex queries combining multiple filters (annotator, agreement score, prediction confidence, task metadata) with efficient database indexing, enabling teams to quickly locate specific subsets of data for review or re-annotation.

Allows teams to define project-level settings including label taxonomy, annotation interface (via XML schema), task sampling strategy, and quality control rules. Projects are stored as database records with serialized configuration (label_studio/projects/serializers.py), enabling teams to create reusable templates and clone projects with identical settings, reducing setup time for similar annotation tasks.

Supports bulk import of tasks from multiple sources (CSV, JSON, cloud storage) with automatic format detection and validation against project schema. The import pipeline (label_studio/tasks/api.py) parses input files, validates data types, and creates task records in batch, with error reporting for malformed entries. Supports resumable imports for large datasets, allowing interrupted uploads to continue without re-processing.