Blake Richards · @tyrell_turing
1007 followers · 349 posts · Server fediscience.org
8/10) We find that α correlates more strongly to downstream task performance than the #BarlowTwins loss itself! 🤯
Thus, we propose a model selection #algorithm based on this result to reduce the number of #readout evals required to identify the best #hyperparamaters. 🤓
#neuroscience #deeplearning #ml #ai #hyperparamaters #readout #algorithm #barlowtwins
Blake Richards · @tyrell_turing
1006 followers · 348 posts · Server fediscience.org
7/10) This finding led to our #proposal: Can we use α for #modelSelection in an #SSL pipeline?
Two key +s of α:
1. α doesn’t require labels
2. α is quick to #compute (compared to training a readout)
We study hyperparam selection in #BarlowTwins (Zbontar et al.) as a case study!
#neuroscience #deeplearning #ml #ai #barlowtwins #compute #ssl #modelselection #proposal