Semi-automatic classification of digital heritage on the Aïoli open source 2D/3D annotation platform via machine learning and deep learning

Today, digital tools represent a privileged means of describing and understanding of cultural heritage. However, existing hybrid methods for semantic annotation on digital models are mostly manual and require considerable human intervention. This may cause data dispersion or data alteration issues, preventing the digital collection of multidisciplinary heritage information in collaborative valorization and conservation projects. Tackling this problem, this work presents three different methods for more automated 2D/3D data annotation, making the most of Machine Learning or Deep Learning and of the collaborative, web-based annotation platform Aïoli. The proposed approaches are designed to assist heritage experts in the hybrid annotation of architectural objects, e.g., to identify building components, degradation patterns, mapping of alterations, materials etc. and to share and access 2D/3D information via the web. The work aims at providing public and private stakeholders in charge of restoration and conservation activities with a set of more automated annotation tools, allowing to: i) preserve data in the transition to different representation types; ii) propagate analytically relevant information via digital models and/or images; iii) share heritage-related data on the web with an open source strategy. The validated workflows sketch the outlines of a dedicated and user-friendly toolbar for semi-automatic annotation, directly integrated into the Aïoli user interface, where automated annotation tools are readily available to the community of heritage professionals to properly support documentation and restoration activities.