The lottery ticket hypothesis suggests that by training DNNs from “lucky” initializations, we can train networks which are 10-100x smaller with minimal performance losses. In new work, we extend our understanding of this phenomenon in several ways... https://ai.facebook.com/blog/understanding-the-generalization-of-lottery-tickets-in-neural-networks https://twitter.com/facebookai/status/1199042155743862784/video/1
Do lottery tickets contain generic inductive biases or are they overfit to the particular dataset and optimizer used to find them? Encouragingly, we found that lottery tickets generalize across related, but distinct datasets and across optimizers: https://arxiv.org/abs/1906.02773
Is the lottery ticket phenomenon a general property of DNNs or merely an artifact of supervised image classification? We show that the lottery ticket phenomenon is a general property which is present in both #reinforcementlearning and #NLP
Can we begin to explain lottery tickets theoretically? We introduce a new theoretical framework on the formation of lottery tickets to help researchers advance toward a better understanding of lucky initializations
Via twitter.com/facebookai/status/1199042159334154241
Do lottery tickets contain generic inductive biases or are they overfit to the particular dataset and optimizer used to find them? Encouragingly, we found that lottery tickets generalize across related, but distinct datasets and across optimizers: https://arxiv.org/abs/1906.02773
Is the lottery ticket phenomenon a general property of DNNs or merely an artifact of supervised image classification? We show that the lottery ticket phenomenon is a general property which is present in both #reinforcementlearning and #NLP
Can we begin to explain lottery tickets theoretically? We introduce a new theoretical framework on the formation of lottery tickets to help researchers advance toward a better understanding of lucky initializations
Via twitter.com/facebookai/status/1199042159334154241
Facebook
Understanding the generalization of ‘lottery tickets’ in neural networks
The lottery ticket hypothesis suggests that by training DNNs from “lucky” initializations, we can train networks which are 10-100x smaller with minimal performance losses. In new work, we extend our understanding of this phenomenon in several ways.