Semi Supervised And Self Supervised Learning With Pseudo Labeling

via “semi-supervised and self-supervised learning with pseudo-labeling”

OpenMMLab detection toolbox with 300+ models.

Unique: Implements semi-supervised detection with pseudo-labeling where a teacher model generates labels on unlabeled data, and a student model is trained with both labeled and pseudo-labeled data; uses exponential moving average (EMA) teacher updates for stability and consistency regularization for improved robustness

vs others: More practical than fully self-supervised approaches because it leverages labeled data when available; more stable than naive pseudo-labeling because EMA teacher updates reduce label noise; better integrated than external semi-supervised frameworks because it's built into the training pipeline

via “semi-supervised object detection with pseudo-labeling and consistency regularization”

OpenMMLab Detection Toolbox and Benchmark

Unique: Implements semi-supervised detection via teacher-student models where the teacher generates pseudo-labels on unlabeled data and the student is trained with consistency regularization, enabling leveraging of unlabeled data without manual annotation

vs others: More integrated than standalone pseudo-labeling implementations because it provides teacher-student infrastructure and consistency loss computation; more flexible than FixMatch (which is image-classification focused) because it handles bounding box pseudo-labels with confidence thresholding

via “self-training with pseudo-labeling for unlabeled audio”

* ⭐ 08/2022: [MuLan: A Joint Embedding of Music Audio and Natural Language (MuLan)](https://arxiv.org/abs/2208.12415)

Unique: Integrates pseudo-labeling as middle stage between SSL pre-training and supervised fine-tuning in three-stage pipeline; specific pseudo-label generation and filtering mechanisms not disclosed, but represents systematic approach to leveraging unlabeled data in semi-supervised ASR

vs others: More systematic than ad-hoc pseudo-labeling by grounding in pre-trained representations; effectiveness vs alternatives depends on undisclosed pseudo-label quality control mechanisms

via “self-supervised-model-training”

via “weak-supervision-label-aggregation”