Contributo in atti di convegno, 2021, ENG, 10.1109/ICTAI52525.2021.00073
Da Silva Neto J.S.; Coelho Da Silva T.L.; Cruz L.A.; Monteiro de Lira V.; José Antônio F. de Macêdo José A.F.; Magalh R.P.
Federal University of Ceara, Fortaleza, Brazil; Federal University of Ceara, Fortaleza, Brazil; Federal University of Ceara, Fortaleza, Brazil; CNR-ISTI, Pisa, Italy; Federal University of Ceara, Fortaleza, Brazil; Federal University of Ceara, Fortaleza, Brazil; Federal University of Ceara, Fortaleza, Brazil
Neural Networks (NN), although successfully applied to several Artificial Intelligence tasks, are often unnecessarily over-parametrized. In edge/fog computing, this might make their training prohibitive on resource-constrained devices, contrasting with the current trend of decentralizing intelligence from remote data centres to local constrained devices. Therefore, we investigate the problem of training effective NN models on constrained devices having a fixed, potentially small, memory budget. We target techniques that are both resource-efficient and performance effective while enabling significant network compression. Our Dynamic Hard Pruning (DynHP) technique incrementally prunes the network during training, identifying neurons that marginally contribute to the model accuracy. DynHP enables a tunable size reduction of the final neural network and reduces the NN memory occupancy during training. Freed memory is reused by a dynamic batch sizing approach to counterbalance the accuracy degradation caused by the hard pruning strategy, improving its convergence and effectiveness. We assess the performance of DynHP through reproducible experiments on three public datasets, comparing them against reference competitors. Results show that DynHP compresses a NN up to 10 times without significant performance drops (up to 3.5% additional error w.r.t. the competitors), reducing up to 80% the training memory occupancy.
ICTAI 2021 - IEEE 33rd International Conference on Tools with Artificial Intelligence, Washington, DC, USA, 1-3/11/2021
Trajactory data, Stolen vehicles, Next location prediction
Monteiro De Lira Vinicius Cezar
ISTI – Istituto di scienza e tecnologie dell'informazione "Alessandro Faedo"
ID: 465630
Year: 2021
Type: Contributo in atti di convegno
Creation: 2022-03-25 18:35:09.000
Last update: 2022-03-29 13:42:45.000
CNR authors
External IDs
CNR OAI-PMH: oai:it.cnr:prodotti:465630
DOI: 10.1109/ICTAI52525.2021.00073
Scopus: 2-s2.0-85123911686
ISI Web of Science (WOS): 000747482300065