Semi-Supervised Learning for Image Classification using Compact Networks in the BioMedical Context

Resum

Background and objectives: The development of mobile and on the edge appli-cations that embed deep convolutional neural models has the potential to revolutionisebiomedicine. However, most deep learning models require computational resourcesthat are not available in smartphones or edge devices; an issue that can be faced bymeans of compact models. The problem with such models is that they are, at leastusually, less accurate than bigger models. In this work, we study how this limitationcan be addressed with the application of semi-supervised learning techniques.Methods: We conduct several statistical analyses to compare performance of deepcompact architectures when trained using semi-supervised learning methods for tack-ling image classification tasks in the biomedical context. In particular, we explore threefamilies of compact networks, and two families of semi-supervised learning techniquesfor 10 biomedical tasks.Results: By combining semi-supervised learning methods with compact net-works, it is possible to obtain a similar performance to standard size networks. Ingeneral, the best results are obtained when combining data distillation with MixNet,and plain distillation with ResNet-18. Also, in general, NAS networks obtain betterresults than manually designed networks and quantized networks.Conclusions: The work presented in this paper shows the benefits of apply semi-supervised methods to compact networks; this allow us to create compact models thatare not only as accurate as standard size models, but also faster and lighter. Finally,we have developed a library that simplifies the construction of compact models usingsemi-supervised learning methods