Go offline with the Player FM app!
[QA] LoCa: Logit Calibration for Knowledge Distillation
Manage episode 439051503 series 3524393
This paper introduces Logit Calibration (LoCa) to enhance knowledge distillation by correcting teacher model predictions while preserving valuable information, improving student model performance without extra parameters.
https://arxiv.org/abs//2409.04778
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
--- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/support
1577 episodes
Manage episode 439051503 series 3524393
This paper introduces Logit Calibration (LoCa) to enhance knowledge distillation by correcting teacher model predictions while preserving valuable information, improving student model performance without extra parameters.
https://arxiv.org/abs//2409.04778
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
--- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/support
1577 episodes
All episodes
×Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.