A ConvNet for the 2020s (ConvNeXt)

Pure convolutional neural network architecture that systematically incorporates Vision Transformer design principles (larger kernels, layer normalization, inverted bottlenecks, reduced activation functions) into ResNet-style convolutions without attention mechanisms. Achieves 87.8% ImageNet top-1 accuracy by applying incremental architectural modifications that bridge the performance gap between standard ConvNets and ViTs while maintaining convolutional simplicity and computational efficiency.

Generates multi-resolution feature pyramids across network depth through staged downsampling blocks that progressively reduce spatial dimensions while increasing channel capacity. Enables downstream tasks (object detection, semantic segmentation) to operate on features at multiple semantic scales by maintaining hierarchical feature maps that capture both low-level details and high-level semantic information.

Increases convolutional kernel sizes from standard 3x3 to 7x7 receptive fields, expanding the local context window that each convolution operates on. This design choice directly mirrors Vision Transformer patch embedding behavior by increasing the spatial context captured in a single convolution operation, enabling the model to learn longer-range spatial dependencies without explicit attention mechanisms.

Implements inverted bottleneck blocks (expand-then-contract channel flow) instead of standard residual bottlenecks, where channels are first expanded to a larger intermediate dimension before being contracted back. This design pattern, borrowed from MobileNet and Vision Transformers' MLP blocks, allows the model to learn richer feature transformations in the expanded space while maintaining parameter efficiency through the contraction phase.

Replaces batch normalization with layer normalization across the network, normalizing feature statistics per sample and channel rather than across the batch dimension. This design choice, inspired by Vision Transformers, decouples normalization from batch size, improving training stability and enabling more flexible batch size configurations during inference and fine-tuning.

Replaces ReLU activations with GELU (Gaussian Error Linear Unit) and reduces the number of activation functions per block, using activations more selectively. GELU provides smoother gradient flow and better approximates the cumulative distribution function, while reducing activation frequency decreases computational overhead and aligns with Vision Transformer design patterns that use fewer non-linearities.

Serves as a feature extraction backbone for object detection tasks on the COCO dataset, producing hierarchical multi-scale features that integrate with standard detection heads (Faster R-CNN, RetinaNet, etc.). The model outperforms Swin Transformer on COCO benchmarks, demonstrating that pure ConvNet architectures can match or exceed transformer-based detection performance when properly modernized.

Serves as a feature extraction backbone for semantic segmentation on the ADE20K dataset, producing dense multi-scale features that integrate with segmentation decoders (FPN, DeepLab, etc.). The model outperforms Swin Transformer on ADE20K benchmarks, showing that pure ConvNets can match transformer performance on dense prediction tasks requiring pixel-level accuracy.

Provides ImageNet pre-trained weights (87.8% top-1 accuracy) that serve as initialization for downstream vision tasks (detection, segmentation, classification). The model achieves competitive ImageNet accuracy with modern ConvNet design principles, enabling transfer learning to specialized vision tasks without training from random initialization.