Contributo in atti di convegno, 2024, ENG

GloNets: Globally Connected Neural Networks

Antonio Di Cecco, Carlo Metta, Marco Fantozzi, Francesco Morandin, Maurizio Parton

Università Chieti-Pescara, Isti-CNR, Università Parma, Università di Parma, Università Chieti-Pescara,

Deep learning architectures suffer from depth-related performance degradation, limiting the effective depth of neural networks. Approaches like ResNet are able to mitigate this, but they do not completely eliminate the problem. We introduce Globally Connected Neural Networks (GloNet), a novel architecture overcoming depth-related issues, designed to be superimposed on any model, enhancing its depth without increasing complexity or reducing performance. With GloNet, the network's head uniformly receives information from all parts of the network, regardless of their level of abstraction. This enables GloNet to self-regulate information flow during training, reducing the influence of less effective deeper layers, and allowing for stable training irrespective of network depth. This paper details GloNet's design, its theoretical basis, and a comparison with existing similar architectures. Experiments show GloNet's self-regulation ability and resilience to depth-related learning challenges, like performance degradation. Our findings suggest GloNet as a strong alternative to traditional architectures like ResNets.

IDA 2024, 24/04/2024, 26/04/2024

Keywords

Machine Learning, Deep Learning, Neural Network

CNR authors

Metta Carlo

CNR institutes