Doktorandské kolokvium KAI - Filip Kerák (17.3.2025)
Prednášajúci: Filip Kerák
Názov: Training Sparse Neural Networks
Termín: 17.3.2025, 13:10 hod., I/9
Abstrakt:
Modern neural networks easily contain billions of parameters, which leads to high computation requirements during both the training and inference phases. An astronomical amount of money and resources is used to create models that, on one hand, achieve state-of-the-art results in all areas but are not widely feasible. One way to tackle this problem is to reduce the number of parameters in those networks. Best improvement can be achieved by introducing sparsity as the key part of the training. Different sparse training approaches for pruning and growing networks are being used, each trading performance for accuracy and vice versa. What is the "golden mean"?