Leshem Choshen · @LChoshen
753 followers · 95 posts · Server sigmoid.socialWe want to pretrain🤞
Instead we finetune🚮😔
Could we collaborate?🤗
ColD Fusion:
🔄Recycle finetuning to multitask
➡️evolve pretrained models forever
On 35 datasets
+2% improvement over RoBERTa
+7% in few shot settings
🧵
#NLProc #MachinLearning #NLP #ML #modelRecyclying #collaborativeAI #scientivism #pretrain
#nlproc #machinlearning #nlp #ml #modelrecyclying #CollaborativeAI #scientivism #pretrain
Leshem Choshen · @LChoshen
753 followers · 94 posts · Server sigmoid.socialWe want to pretrain🤞
Instead we finetune🚮😔
Could we collaborate?🤗
ColD Fusion:
🔄Recycle finetuning to multitask
➡️evolve pretrained models forever
On 35 datasets
+2% improvement over RoBERTa
+7% in few shot settings
🧵
#nlproc #machinlearning #nlp #ml #modelrecyclying