Artwork

Content provided by PurePerformance. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by PurePerformance or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.
Player FM - Podcast App
Go offline with the Player FM app!

OpenLLMetry - Observing the Quality of LLMs with Nir Gazit

50:32
 
Share
 

Manage episode 395473545 series 1897294
Content provided by PurePerformance. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by PurePerformance or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.
Its only been a year since ChatGPT was introduced. Since then we see LLMs (Large Language Models) and Generative AIs being integrated into every days life software applications. Developers have the hard choice to pick the right model for their use case to produce the quality of output their end users demand.
Tune in to this session where we have Nir Gazit, CEO and Co-founder of Traceloop, educating us about how to observe and quantify the quality of LLMs. Besides performance and costs engineers need to look into quality attributes such as accuracy, readability or grammatical correctness.
Nir introduces us to OpenLLMetry - a set of Open Source extensions built on top of OpenTelemetry providing automated observability into the usage of LLMs for developers to better understand how to optimize the usage of LLMs. His advice to every developer is to start measuring the quality of your LLMs on Day 1 and continuously evaluate as you change your model, the prompt and the way you interact with your LLM stack!
If you have more questions about LLM Observability check out the following links:
OpenLLMetry GitHub Page: https://github.com/traceloop/openllmetry
Traceloop Website: https://www.traceloop.com/
OpenLLMetry Documentation: https://traceloop.com/docs/openllmetry
  continue reading

288 episodes

Artwork
iconShare
 
Manage episode 395473545 series 1897294
Content provided by PurePerformance. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by PurePerformance or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.
Its only been a year since ChatGPT was introduced. Since then we see LLMs (Large Language Models) and Generative AIs being integrated into every days life software applications. Developers have the hard choice to pick the right model for their use case to produce the quality of output their end users demand.
Tune in to this session where we have Nir Gazit, CEO and Co-founder of Traceloop, educating us about how to observe and quantify the quality of LLMs. Besides performance and costs engineers need to look into quality attributes such as accuracy, readability or grammatical correctness.
Nir introduces us to OpenLLMetry - a set of Open Source extensions built on top of OpenTelemetry providing automated observability into the usage of LLMs for developers to better understand how to optimize the usage of LLMs. His advice to every developer is to start measuring the quality of your LLMs on Day 1 and continuously evaluate as you change your model, the prompt and the way you interact with your LLM stack!
If you have more questions about LLM Observability check out the following links:
OpenLLMetry GitHub Page: https://github.com/traceloop/openllmetry
Traceloop Website: https://www.traceloop.com/
OpenLLMetry Documentation: https://traceloop.com/docs/openllmetry
  continue reading

288 episodes

All episodes

×
 
Loading …

Welcome to Player FM!

Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.

 

Quick Reference Guide