Too Many Butterflies from One Chrysalis. Continual Learning, Continual Forgetting and the Harmonic Flow of Information

Despite addressing dynamic learning scenarios, the Continual Learning paradigm is still an evolving field, with no consensus on a definitive methodology among the numerous approaches proposed. In this study, we reflect upon possible novel perspectives about the learning process itself, posing a few questions: how does information get structured in models’ parameters? what if memory and oblivion were two faces of the same coin? therefore, could we conceive a network capable of both learning and forgetting continuously? We put forward that information be distributed as a harmony, meaning that there should be some degree of consonance in the data for the continuous learning process to succeed. Provided that, Continual Learning might be possible, say, as a variation on the theme, possibly deeming optimization as a kind of orchestration, even among various agents. We encourage the enhancement of this framework, where current brute-force monolithic models would be surpassed in favor of more efficient agents, capable of evolving dynamically from their interactions.