Type

Conference Proceedings

Authors

Kevin McGuinness
Noel E. O'Connor
Paul Albert
Diego Ortego
Eric Arazo Sanchez

Subjects

Computer Science

Topics
deep learning bias image processing machine learning state of the art artificial intelligence semi supervised learning image classification

Pseudo-labeling and confirmation bias in deep semi-supervised learning (2020)

Abstract Semi-supervised learning, i.e. jointly learning from labeled and unlabeled samples, is an active research topic due to its key role on relaxing human supervision. In the context of image classification, recent advances to learn from unlabeled samples are mainly focused on consistency regularization methods that encourage invariant predictions for different perturbations of unlabeled samples. We, conversely, propose to learn from unlabeled data by generating soft pseudo-labels using the network predictions. We show that a naive pseudo-labeling overfits to incorrect pseudo-labels due to the so-called confirmation bias and demonstrate that mixup augmentation and setting a minimum number of labeled samples per mini-batch are effective regularization techniques for reducing it. The proposed approach achieves state-of-the-art results in CIFAR-10/100, SVHN, and Mini-ImageNet despite being much simpler than other methods. These results demonstrate that pseudo-labeling alone can outperform consistency regularization methods, while the opposite was supposed in previous work. https://git.io/fjQsC
Collections Ireland -> Dublin City University -> Publication Type = Conference or Workshop Item
Ireland -> Dublin City University -> Subject = Computer Science: Artificial intelligence
Ireland -> Dublin City University -> Subject = Computer Science: Image processing
Ireland -> Dublin City University -> Status = Published
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools: Faculty of Engineering and Computing: School of Electronic Engineering
Ireland -> Dublin City University -> DCU Faculties and Centres = Research Initiatives and Centres: INSIGHT Centre for Data Analytics
Ireland -> Dublin City University -> Subject = Computer Science: Machine learning

Full list of authors on original publication

Kevin McGuinness, Noel E. O'Connor, Paul Albert, Diego Ortego, Eric Arazo Sanchez

Experts in our system

1
Kevin McGuinness
Dublin City University
Total Publications: 93
 
2
Noel E. O'Connor
Dublin City University
Total Publications: 474
 
3
Diego Ortego
Dublin City University
Total Publications: 3
 
4
Eric Arazo Sanchez
Dublin City University
Total Publications: 3