rtdetr_v2_r18vd

Performs object detection on images using a deformable transformer backbone (ResNet-18 variant) combined with deformable attention mechanisms that dynamically focus on relevant spatial regions. The model uses a two-stage detection head with anchor-free predictions, enabling real-time inference (~30 FPS on standard hardware) while maintaining competitive accuracy on COCO-scale datasets. Deformable attention reduces computational overhead by sampling only task-relevant spatial locations rather than processing full feature maps.

Provides pre-trained weights initialized on COCO dataset (80 object classes: person, car, dog, bicycle, etc.) enabling zero-shot or few-shot transfer to custom detection tasks. The model outputs class predictions across all 80 COCO categories with per-class confidence scores, allowing downstream filtering or class-specific post-processing. Weights are stored in safetensors format for secure, reproducible model loading without arbitrary code execution.

Processes multiple images in parallel with automatic resolution padding/resizing to handle variable input dimensions without recompilation. The model uses dynamic shape handling in the transformer backbone, allowing batch processing of images with different aspect ratios by padding to a common size and tracking valid regions. This enables efficient GPU utilization for batched inference while maintaining per-image detection accuracy.

Applies non-maximum suppression (NMS) to raw model outputs to eliminate duplicate detections of the same object, then filters results by confidence threshold. The model outputs raw class logits and box coordinates; post-processing applies softmax normalization, confidence thresholding (default 0.5), and NMS with IoU threshold (default 0.6) to produce final detections. This two-stage filtering reduces false positives and overlapping boxes typical of raw transformer outputs.

Supports conversion to quantized formats (INT8, FP16) and export to ONNX, TensorRT, or CoreML for deployment on edge devices, mobile phones, and embedded systems. The model can be quantized post-training using PyTorch quantization APIs or exported to optimized inference runtimes that reduce model size by 4-8x and latency by 2-3x compared to full-precision inference. Safetensors format enables secure, reproducible quantization without code execution risks.

Predicts bounding boxes directly from image features without predefined anchor templates, using IoU-aware loss functions (e.g., GIoU, DIoU) that optimize box overlap with ground truth rather than L1/L2 distance. The model regresses box coordinates (x1, y1, x2, y2 or cx, cy, w, h) end-to-end, with loss functions that account for box geometry and overlap quality. This approach eliminates manual anchor design and improves convergence compared to anchor-based methods.

Extracts features at multiple scales (e.g., 1/8, 1/16, 1/32 of input resolution) using a feature pyramid network (FPN) that combines high-resolution semantic features with low-resolution spatial context. The ResNet-18 backbone produces features at multiple levels; FPN applies top-down pathways and lateral connections to create a pyramid of feature maps suitable for detecting objects at different scales. This architecture enables detection of both small objects (using high-resolution features) and large objects (using low-resolution features with larger receptive fields).

Uses transformer self-attention to aggregate contextual information across spatial regions of the image, allowing each detected object to incorporate features from distant regions. Unlike CNNs with limited receptive fields, transformer attention enables long-range spatial relationships (e.g., detecting a person holding a phone by attending to both person and phone regions). Deformable attention makes this efficient by sampling only task-relevant regions rather than all spatial locations.