Hardware-in-the-loop validation of a sense and avoid system leveraging data fusion between radar and optical sensors for a mini UAV

The present work illustrates the results obtained at the conclusion of the three-year project TERSA (Tecnologie Elettriche e Radar per Sapr Autonomi), involving the aerospace section of the Dept. of Civil and Industrial Engineering (DICI) of the University of Pisa and its industrial partners. The project aimed at the design and development of a fully autonomous Sense and Avoid (SAA) prototype system, based on data fusion between optical and radar sensors data, for a tactical lightweight surveillance UAV (MTOW<25Kg). The problem of non-cooperative collision avoidance is well known in literature and is currently a central theme of investigation within the aeronautical industry, considering the growing UAV traffic and the consequent need to employ autonomous self-separation technologies in the market. Several past works have investigated the most varied solutions for the Sense problem utilizing optical, acoustic, electromagnetic signals or a combination of the previous. Likewise, the Avoidance problem has been successfully tackled in literature by means of a wide variety of different approaches ranging from rule-based methods, strategies based on game theory, force field methods, optimization frameworks leveraging genetic algorithms and nonlinear programming techniques and geometric methods. Yet, to the knowledge of the authors, no previous work found in literature has successfully demonstrated and validated the real-time simultaneous interaction of both sense and avoid functionalities within a highly integrated simulation environment. The present work describes the implementation of a complex, nonlinear, simulation environment conceived in order to perform real-time, Hardware-in-the-Loop (HIL), testing of the effective cooperation between sense and the avoid algorithms constituting the core of the SAA system developed within the context of the project. The system effectiveness has been validated by means of complex dynamic simulations, comprising an accurate, fully nonlinear, flight mechanic model of the aircraft, a graphic rendering engine of the scene, proper video capture and transmission pipelines, computer vision algorithms and collision avoidance logics running on the target hardware (Nvidia Jetson Nano) and tailored noise resilient data fusion algorithms. Results show the effectiveness of the system in detecting impending collisions and performing last-resort resolution manoeuvres with high computational efficiency and update frequencies compatible with real world applications in Unmanned Aircraft Systems (UAS).