You’ve Got the Wrong Number: Evaluating Deep Learning Training Paradigms Using Handwritten Digit Recognition Data

Abstract: To build more accurate and trustworthy artificial intelligence algorithms in deep learning, it is essential to understand the mechanisms driving classification systems to identify their targets. Typically, post hoc methods provide insights into this process. In this preliminary work, we shift the reconstruction of the class activation map to the training phase to evaluate how the model’s performance changes compared to standard classification approaches. The Modified National Institute of Standards and Technology dataset and its variants, such as Fashion Modified National Institute of Standards and Technology, consist of well-defined images that facilitate testing this type of training process. Specifically, the classification targets are the only significant content in the images, excluding the background, allowing for a direct comparison of the reconstruction against the input images. To enhance the guidance of the network, we introduce a contrastive loss term to complement the standard classification function, which often uses categorical cross-entropy. By comparing the accuracy and the extracted pattern of the standard approach with the proposed method, we can gain valuable insights into the network’s learning process. This approach aims to improve the interpretability and effectiveness of the model during training, ultimately leading to higher classification accuracy and reliability.