Implicit Neural Decision Trees

Representation learning is a central topic in machine learning, with significant efforts dedicated to encoding structured data such as sequences, trees, and graphs for various downstream tasks. A branch of these studies focuses on functional data analysis, which views data not as discrete arrays but as continuous functions. When these functions are parameterized using neural networks, they are called Implicit Neural Representations (INR). INRs have been successfully applied to represent diverse data types but, to the best of our knowledge, have not been used for encoding decision models. This work addresses the novel challenge of using INRs to represent decision trees. We introduce a tailored coordinate system and train INRs to reconstruct decision trees with a loss function to minimize node reconstruction errors. We benchmark implicit neural decision trees on several datasets, showing that they can effectively represent individual trees, and show potential extensions to tree forests through meta-learning.