Go offline with the Player FM app!
Learning Syntax Without Planting Trees: Understanding When and Why Transformers Generalize Hierarchically
Manage episode 414652797 series 3524393
The paper explores inductive bias in transformer models, showing language modeling training leads to hierarchical generalization, supported by pruning experiments and Bayesian analysis.
https://arxiv.org/abs//2404.16367
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
--- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/support
1049 episodes
Manage episode 414652797 series 3524393
The paper explores inductive bias in transformer models, showing language modeling training leads to hierarchical generalization, supported by pruning experiments and Bayesian analysis.
https://arxiv.org/abs//2404.16367
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
--- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/support
1049 episodes
All episodes
×Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.