Date : July 11, 2022, 2 p.m. - XIE Jiarui - Amphi Bruno Garcia
Dealing with confusing samples in image classification based model
Uncertainty estimation is a straight approach adopted to detect confusing samples. The uncertainty estimation method can calculate a scalar value representing the uncertainty, then detect confusing samples by comparing to a pre-defined threshold. In the first part of this thesis, a Subjective Logic based Uncertainty Estimation method (SLUE) is proposed by extending the existing method named Evidential Deep Learning (EDL). Compared with the EDL that overlooked the advantages of the base rate, the SLUE method takes the base rates explicitly into account. The base rate in SLUE is used to guide the training process in the desired direction.
Through uncertainty estimation, we can decide whether to reject or accept a prediction. However, it is more reasonable to predict a subset for cautious decision-making. Consequently, a Partial Classification derived from Model Output (PCMO) is proposed in the second part. The PCMO method can assign beliefs to nested subsets under the Dempster-Shafer Theory (DST). That can alleviate the exponential increase in the subset number brought by the increase of the class number. Compared with the existing methods, the PCMO method shows a simple and scalable framework. On the one hand, the PCMO method is fulfilled only based on the model output that can be applied to any pre-trained convolutional neural networks, without any demand to retrain the model or conduct any further modifications. On the other hand, by considering good features of the log function and analyzing the regular pattern of model output, a novel and reasonable transformation from model output to possibility distribution is proposed. One weakness of the PCMO method is that it cannot provide beliefs for the empty set and the entire set. Consequently, we rethink the possibility calculation process in the PCMO method and proposed a new method named PCBS that combines the DST and the bell-shaped function (BSF). In which, the BSF is responsible for the possibility calculation by taking the information provided by the training output. By doing that the model is able to assign beliefs to the empty set and the entire set leading to the ability to distinguish between the out-of-domain samples and imprecise samples.
Partial classification, Uncertainty estimation, Dempster-Shafer theory, Subjective logic, Confusing sample, Dirichlet distribution, Bell-shaped function
The jury is composed of :
Pr., David Mercier, Université d'Artois, Reviewer
Pr., Didier Coquin, Université Savoie Mont Blanc, Reviewer
Pr., Mephu Nguifo Engelbert, Université Clermont Auvergne, Examiner
Pr., Nicolas Sutton-Charani, IMT Mines-Alès, Examiner
Pr., Thierry Chateau, Université Clermont Auvergne, Director of thesis
Pr., Violaine Antoine, Université Clermont Auvergne, Supervisor.