Detecting happiness from 14-channel binary-valued EEG charts via deep learning

Electroencephalography (EEG) is widely used for emotion recognition, but it typically requires cumbersome equipment used in lab settings. Modern EEG sensors are light and handy, and can be worn in daily life. However, these sensors have fewer electrodes than those used in labs which means less information available and the need for more sophisticated techniques for emotion recognition. This paper presents a system that combines Convolutional Neural Networks (CNNs) and Support Vector Machines (SVMs) to determine whether or not a person feels happiness based on a 14-channel EEG chart. The system first combines three pre-trained CNNs, namely, Inception V3, VGG-16, and ResNet-50 to extract features from an EEG chart represented as a binary matrix (binary-valued EEG chart). One module made up of three one-class SVMs is then used for each CNN to detect happiness (or non-happiness). The final class is that detected by the majority of the SVM modules. A dataset was collected involving 30 participants who watched 26 video clips as their EEG signals were being recorded by an EMOTIV EPOCX device. Participants self-rated their emotions using the 9-point Self-Assessment Manikin (SAM), and emojis. The system achieved an accuracy of 86.4%.