Online Knowledge Distillation and Deep Supervision in HRNet: Green AI for Preterm Infants’ Pose Estimation

The current approach to deep-learning research is exemplified by the pursuit of Red AI models—designs that show increasingly higher performance but with increasingly higher costs, in terms of economical requirements and environmental footprint. This approach is particularly detrimental in sectors like healthcare, which typically have limited resources. Meanwhile, Green AI prioritizes efficiency and sustainability, by reducing the environmental footprint and making advanced technologies accessible. Following the Green AI principles, this study focuses on the combined use of two techniques, namely Knowledge Distillation (KD) and Deep Supervision (DS), to reduce the costs of HRNet, a convolutional neural network designed for human pose estimation, here applied to support the diagnosis of neurological impairments in preterm infants. All the experiments are carried out on the BabyPose dataset, a collection of videos from a depth camera showing hospitalized preterm infants. By combining KD and DS, we can use a sub-network of HRNet, which needs 27.5% of parameters and 61.7% of FLOPs in HRNet, without affecting performance (-0.59 percentage points in average precision). This achievement can have deep implications in the actual clinical practice, as it fosters democratization of high-quality technologies. Our codes are available at https://github.com/geronimaw/OnlineKD-HRNet-Human-Pose-Estimation.git.