A connection between sparse and low rank matrices. Let S be a sparse similarity matrix, for example the distances of the 3 nearest neighbours in a low dimensional manifold. Can you recover S if you have a low rank (dense) matrix L from in a high dimensional space? This paper provides a geometric interpretation for S = max(0,L). It proposes a decomposition algorithm, that can be modelled as a ReLU neural network layer.

#MachineLearning #SparseDecomposition #LowRank #TMLR
https://openreview.net/forum?id=p8gncJbMit

Steffen Schotthöfer & Emanuele Zangrando from our lab are attending #NeurIPS next week in person and will present our work on #lowrank #training & #pruning of #NNs

Meet both at the poster session in HallJ#604 on Wed 30 Nov 9:30amPST

What is it? We developed a framework to perform stable and efficient training on low-rank manifolds, resulting in an order of magnitude less memory cost & training time! Tested successfully on #imagenet1k #transformers +other benchmarks

https://openreview.net/forum?id=IILJ0KWZMy9&referrer=%5Bthe%20profile%20of%20Francesco%20Tudisco%5D(%2Fprofile%3Fid%3D~Francesco_Tudisco1)