Guaranteeing the model trained by in-domain samples to make correct and reliable predictions when encountering confusing samples in the open world is an inevitable component of image classification based models. Consequently, three novelty methods based on uncertainty estimation and partial classification, respectively, are proposed. The thesis scope is limited to the image classification based model, and the confusing sample contains out-of-domain, and imprecise samples.
Uncertainty estimation is a straight approach adopted to detect confusing samples. The uncertainty estimation method can calculate a scalar value representing the uncertainty, then detect confusing samples by comparing to a pre-defined threshold. In the first part of this thesis, a Subjective Logic based Uncertainty Estimation method (SLUE) is proposed by extending the existing method named Evidential Deep Learning (EDL). Compared with the EDL that overlooked the advantages of the base rate, the SLUE method takes the base rates explicitly into account. The base rate in SLUE is used to guide the training process in the desired direction.
Through uncertainty estimation, we can decide whether to reject or accept a prediction. However, it is more reasonable to predict a subset for cautious decision-making. Consequently, a Partial Classification derived from Model Output (PCMO) is proposed in the second part. The PCMO method can assign beliefs to nested subsets under the Dempster-Shafer Theory (DST). That can alleviate the exponential increase in the subset number brought by the increase of the class number. Compared with the existing methods, the PCMO method shows a simple and scalable framework. On the one hand, the PCMO method is fulfilled only based on the model output that can be applied to any pre-trained convolutional neural networks, without any demand to retrain the model or conduct any further modifications. On the other hand, by considering good features of the log function and analyzing the regular pattern of model output, a novel and reasonable transformation from model output to possibility distribution is proposed. One weakness of the PCMO method is that it cannot provide beliefs for the empty set and the entire set. Consequently, we rethink the possibility calculation process in the PCMO method and proposed a new method named PCBS that combines the DST and the bell-shaped function (BSF). In which, the BSF is responsible for the possibility calculation by taking the information provided by the training output. By doing that the model is able to assign beliefs to the empty set and the entire set leading to the ability to distinguish between the out-of-domain samples and imprecise samples.

 

Keywords: