Go offline with the Player FM app!
Characterizing stable regions in the residual stream of LLMs
Manage episode 441899042 series 3524393
The paper identifies stable regions in Transformers' residual streams, showing insensitivity to small changes but high sensitivity at boundaries, aligning with semantic distinctions and clustering similar prompts.
https://arxiv.org/abs//2409.17113
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
--- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/support
1605 episodes
Manage episode 441899042 series 3524393
The paper identifies stable regions in Transformers' residual streams, showing insensitivity to small changes but high sensitivity at boundaries, aligning with semantic distinctions and clustering similar prompts.
https://arxiv.org/abs//2409.17113
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
--- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/support
1605 episodes
All episodes
×Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.