Artwork

Content provided by Carl Franklin. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Carl Franklin or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.
Player FM - Podcast App
Go offline with the Player FM app!

Azure API Management's GenAI Gateway with Andrei Kamenev

50:49
 
Share
 

Manage episode 433149544 series 1414674
Content provided by Carl Franklin. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Carl Franklin or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.
How do you manage APIs to GenAI, and how can GenAI help with API management? Carl and Richard chat with Andrei Kamenev about the latest features coming to Azure API Management. On the one hand, there are Copilot tools to help craft and understand APIM policies, which can get very complex. Then, there is the provisioning of access to GenAI-related APIs like the Azure OpenAI service, which utilize tokens - and those tokens mean money, so they need to be controlled. The GenAI Gateway provides the ability to rate-limit token issuing and all the other capabilities you expect from APIM. Prompt caching is in preview and can decrease the cost of repeated use of the same prompts. Many of the features are new, and more are coming!
  continue reading

1015 episodes

Artwork
iconShare
 
Manage episode 433149544 series 1414674
Content provided by Carl Franklin. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Carl Franklin or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.
How do you manage APIs to GenAI, and how can GenAI help with API management? Carl and Richard chat with Andrei Kamenev about the latest features coming to Azure API Management. On the one hand, there are Copilot tools to help craft and understand APIM policies, which can get very complex. Then, there is the provisioning of access to GenAI-related APIs like the Azure OpenAI service, which utilize tokens - and those tokens mean money, so they need to be controlled. The GenAI Gateway provides the ability to rate-limit token issuing and all the other capabilities you expect from APIM. Prompt caching is in preview and can decrease the cost of repeated use of the same prompts. Many of the features are new, and more are coming!
  continue reading

1015 episodes

All episodes

×
 
Loading …

Welcome to Player FM!

Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.

 

Quick Reference Guide