Using contrastive learning to inject domain-knowledge into neural networks for recognizing emotions

With application contexts ranging from psychophysiology to neuromarketing, electroencephalography (EEG)-based emotion recognition is a fundamental technology for affective computing. In this context, EEG signals can be processed via artificial neural networks (NNs) to achieve accurate recognition of users’ emotions. Still, NNs are rarely employed in realworld decision-making processes, since their internal model works as a hardly trustable black box. A NN’s reasoning can be explained in a human-comprehensible manner by exploring its latent space to understand if some domain knowledge is actually represented and exploited for the classification. Those approaches assume that a trained NN autonomously organizes its latent space according to some domain concepts to process the data via human-like reasoning. However, there is no guarantee that such an assumption holds, since the latent space is not built for this aim. On the other hand, forcing the organization of the latent space (e.g. via contrastive learning) can result in poor recognition performances due to information loss. To guarantee great recognition performances and provide a domainknowledge-driven organization of NNs’ latent space, we combine the well-known training procedure based on a categorical crossentropy loss with a supervised contrastive learning approach for continuous values labels. The proposed approach (i) enables the explanation of NN’s reasoning in terms of the importance of high-level domain concepts in the final classification, and (ii) results in a recognition performance comparable to or better than the one achieved via an approach based solely on maximizing recognition. The proposed approach is tested on the publicly available MAHNOB dataset