mmdet

MMDetection decomposes object detection into pluggable components (backbone, neck, head, loss) registered in a centralized registry pattern, enabling users to construct custom detectors by combining pre-built modules without modifying core framework code. The registry system maps string identifiers to component classes, allowing configuration-driven model instantiation where backbone (ResNet, Swin), neck (FPN, PAFPN), and head (detection, mask, ROI) modules are swapped declaratively.

MMDetection abstracts the entire training workflow (data loading, augmentation, optimization, checkpointing) into declarative Python configuration files that specify dataset paths, model architecture, learning rates, schedules, and distributed training parameters. The framework parses these configs and orchestrates multi-GPU/multi-node training via PyTorch DistributedDataParallel, handling gradient synchronization, checkpoint saving, and metric logging automatically without requiring manual distributed training code.

MMDetection supports semi-supervised detection where unlabeled data is leveraged via pseudo-labeling (generating predictions on unlabeled data and using high-confidence predictions as training targets) and consistency regularization (enforcing consistent predictions under different augmentations). The framework implements teacher-student models where a teacher network generates pseudo-labels for unlabeled data, and a student network is trained on both labeled and pseudo-labeled data with consistency losses.

MMDetection provides analysis tools for visualizing model predictions, attention maps, and feature activations to aid debugging and interpretation. The framework includes visualization utilities for drawing bounding boxes, segmentation masks, and attention heatmaps on images, as well as analysis tools for computing prediction confidence distributions, false positive/negative analysis, and per-class performance breakdown. These tools help practitioners understand model behavior and identify failure modes.

MMDetection provides a composable data augmentation pipeline that applies geometric transforms (resize, crop, rotate, flip) and photometric transforms (color jitter, normalization) in sequence, with bounding box and segmentation mask updates automatically propagated through each transform. The pipeline is defined declaratively in config files and supports both online augmentation (applied during training) and test-time augmentation (TTA) where multiple augmented versions of test images are inferred and results are aggregated.

MMDetection provides implementations of single-stage detectors that predict bounding boxes and class scores directly from feature maps without region proposal generation. These detectors use dense prediction heads that output predictions at multiple scales (via FPN), with focal loss to handle class imbalance and IoU-based loss functions for box regression. The architecture supports anchor-based (YOLO, SSD, RetinaNet) and anchor-free (FCOS, ATSS) variants with configurable backbone and neck modules.

MMDetection implements two-stage detectors that first generate region proposals (via RPN) and then refine them with classification and bounding box regression heads. The framework supports cascade refinement (Cascade R-CNN) where proposals are progressively refined through multiple stages with increasing IoU thresholds, and instance segmentation (Mask R-CNN) where a mask head predicts per-pixel segmentation masks for each detected instance. ROI pooling/alignment extracts fixed-size features from proposals for downstream processing.

MMDetection provides implementations of transformer-based detectors (DETR, Deformable DETR, DINO) that replace hand-crafted detection heads with learned transformer encoders/decoders. These detectors treat object detection as a set prediction problem where a fixed number of learnable query embeddings are refined through transformer layers to predict bounding boxes and class scores. Deformable attention mechanisms enable efficient processing of high-resolution feature maps by attending only to relevant spatial regions.

MMDetection supports multi-task learning where detection, instance segmentation, and panoptic segmentation are trained jointly with shared backbones and necks. The framework provides separate heads for each task (detection head, mask head, semantic segmentation head) that operate on shared feature maps, with task-specific losses combined via weighted summation. Panoptic segmentation unifies instance and semantic segmentation by assigning each pixel to either an instance or semantic class.

MMDetection provides comprehensive evaluation metrics for object detection including COCO Average Precision (AP), LVIS metrics (with long-tail class weighting), and custom metrics. The evaluation pipeline computes metrics at multiple IoU thresholds (0.5:0.95), object sizes (small, medium, large), and supports both standard evaluation and class-wise breakdown. Metrics are computed on validation sets during training and on test sets for final model evaluation.

MMDetection provides inference APIs that support single-image and batch inference with automatic preprocessing (resizing, normalization) and postprocessing (NMS, score thresholding). The framework supports test-time augmentation (TTA) where multiple augmented versions of input images are inferred and predictions are aggregated via NMS or weighted averaging. Inference can be executed on CPU or GPU with configurable batch sizes for throughput optimization.

MMDetection implements grounded object detection models (GLIP, Grounding DINO) that align image regions with natural language descriptions, enabling detection of arbitrary object classes without training-time class labels. These models use vision-language pre-training where image patches are aligned with text embeddings, allowing zero-shot detection by matching image features to arbitrary text queries. The framework supports both phrase-level grounding (detecting specific noun phrases) and image-level grounding (detecting all objects matching a description).