Embark on a captivating exploration of Large Language Models (LLMs), prompt engineering, and generative AI with hosts Bradley Arsenault and Justin Macorin. With 25 years of combined machine learning and product engineering experience, they are delving deep into the world of LLMs to uncover best practices and stay at the forefront of AI innovation. Join them in shaping the future of technology and software development through their discoveries in LLMs and generative AI. Podcast website: https ...
…
continue reading
In this episode, AI experts Bradley Arsenault and Justin Macon dive deep into the challenges and best practices for safely executing code generated by large language models in a production environment. They discuss key security considerations, containerization techniques, static/dynamic code analysis, and error handling - providing valuable insight…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
How to Rescue AI Innovation at Big Companies
19:20
19:20
Play later
Play later
Lists
Like
Liked
19:20
In this episode, we dive into how companies can compete and innovate in the age of AI. We explore the limitations of simple "Generate with AI" buttons and discuss the need for more radical changes in AI implementation. Drawing from Clayton Christensen's work on disruptive innovation, we examine why large organizations struggle with adopting transfo…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
How UX Will Change With Integrated Advice
17:22
17:22
Play later
Play later
Lists
Like
Liked
17:22
Are you tired of hearing about agents? Chat-bot this, agent that. It's the only way to apply AI that anyone talks about. But are agents the only way to use LLM technology? Is everything just converging towards conversational interface? In this episode of The Prompt Desk, your show hosts discuss a different way to use cutting edge AI in user-interfa…
…
continue reading
If you are using systems prompts, chat completions, and tool completions with OpenAI, you might find it challenging to get the model to follow your prompts. If you are like the show hosts, you often find that there are certain instructions that your bot just refuses to listen to! In the latest episode of The Prompt Desk, show host Bradley Arsenault…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Can custom chips save AI's power problem?
37:26
37:26
Play later
Play later
Lists
Like
Liked
37:26
Cutting-edge AI models use a lot of power. It is not uncommon to hear of data centers under construction that require the power output of the largest nuclear reactors all by themselves. Large tech companies used to tout their green credentials - now they quietly buried that message to pursue fortunes in generative AI. In this environmentally consci…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Towards an Inter-Agent Communication Standard
21:21
21:21
Play later
Play later
Lists
Like
Liked
21:21
In this future-looking episode, hosts Bradley Arsenault and Justin Macorin discuss how agents might communicate with each other in the future. Will agents be making HTTP requests to each other? Will agents dial each other up on the phone? Might it be possible that one-hundred years from now, all our AI systems communicate to each-other over plain-t…
…
continue reading
In this thought-provoking episode, hosts Bradley and Justin explore the intriguing concept of using AI to write prompts for other AI systems. They delve into the paradoxical nature of this idea and discuss potential approaches to make it feasible. The hosts emphasize the importance of having a structured system and well-defined schemas when attempt…
…
continue reading
Brad and Justin discuss the gap between AI's perceived capabilities and limitations. We explore common misconceptions about chatbots, discussing pre-defined tasks, integration dependencies, and the importance of clearly defining both what AI can and cannot do. Join us for insights on creating more realistic AI systems and managing expectations. ---…
…
continue reading
In this episode of The Prompt Desk Podcast, hosts Bradley Arsenault and Justin Macorin dive into the fascinating world of AI-powered networking bots. Bradley shares his experience creating and refining a bot that makes phone calls to schedule lunches and happy hours, using his own voice. The hosts discuss the technical challenges of building such a…
…
continue reading
In this cutting-edge episode of The Prompt Desk, your hosts Justin Macorin and Bradley Arsenault explore innovative techniques for testing AI agents using synthetic data and AI-generated test cases. Discover how to create testing agents that validate the performance of your main AI agent, handle edge cases, ensure proper conversation realignment, a…
…
continue reading
Explore the current landscape and future possibilities of AI agents with machine learning experts Justin and Brad. They discuss the trade-offs between specialized and general-purpose agents, the importance of domain expertise, and the potential for agent aggregators. The conversation also covers the standardization of agent-to-agent communication a…
…
continue reading
As AI agents powered by large language models become more complex, developers often encounter erratic and unexpected behaviors during testing. From agents falling into infinite loops to models struggling with certain data formats, these issues can be tricky to diagnose and resolve. In this episode, Bradley Arsenault and Justin Macorin explore real-…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
[Bonus] LLMs making Web-Browsing Decisions
22:50
22:50
Play later
Play later
Lists
Like
Liked
22:50
[Bonus] This is a bonus episode. We had too many unreleased episodes in our backlog, so we decided to give you an extra treat this week! Hope you enjoy it! Join the discussion on how AI agents can intelligently browse websites to retrieve specific information across links and pages. The hosts dive deep into techniques like embedding link text, URL …
…
continue reading
Gain a better understanding of completion vs. chat completion methods for large language models. Get the scoop on why chat completions are great for building chatbots quick, but can sometimes go off the rails. Learn how to keep your model on track with slick moves like tool completions and throwing a "governor" into the mix. --- Continue listening …
…
continue reading
[Bonus] This is a bonus episode. We had too many unreleased episodes in our backlog, so we decided to give you an extra treat this week! Hope you enjoy it! Dive into the world of prompt engineering for large language models like GPT. This episode explores the important role that non-technical subject matter experts play in developing effective AI p…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Measuring Business Results of LLMs with Abi Aryan
43:09
43:09
Play later
Play later
Lists
Like
Liked
43:09
In this insightful episode, hosts Justin Macorin and Bradley Arsenault chat with AI expert Abi Aryan to dive deep into the challenges of measuring the business impact of large language model applications. Abby draws from her 8 years of experience deploying and productizing LLMs to provide a comprehensive framework for defining relevant metrics. She…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
[Bonus] Next iteration of Chat Interfaces
20:07
20:07
Play later
Play later
Lists
Like
Liked
20:07
[Bonus] This is a bonus episode. We had too many unreleased episodes in our backlog, so we decided to give you an extra treat this week! Hope you enjoy! This episode explores the idea of making large language model interactions more transparent and immersive for users. The hosts discuss the potential of showing intermediate steps and outputs instea…
…
continue reading
Did you immediately try out the GPT-4o model as soon as it came splashing across your news feed? If you were anything like your show hosts, you quickly realized that it's just one step in the ever-evolving world of AI. This realization might have sparked your curiosity about the future potential of language models and their impact on various indust…
…
continue reading
Note! This is an episode that we found in our archive unreleased. It is one of the earliest episodes we ever recorded, but has so far gone not been listened to. It's not quite the same style as our current episodes, but we thought we'd release it anyway as a special treat for interested listeners! The podcast episode explores the realm of large lan…
…
continue reading
In this insightful episode, hosts Bradley and Justin tackle the challenge of managing expectations when using large language models (LLMs) and AI agents for tasks like content creation and website interactions. They dispel the common misconception that these systems can produce perfect outputs right out of the gate without any human guidance. Inste…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
How to protect your LLM against Prompt Injections
22:00
22:00
Play later
Play later
Lists
Like
Liked
22:00
In this episode, we discuss, how we might protect prompt-based applications and LLMs from prompt injection. We discuss how data validation was done in the 1960s and modern libraries and techniques that can successfully act as a first line of defense against prompt injection. We touch on the idea that using other types of models, such as decision tr…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
How do we know if Llama 3 is actually any better with Maxime Labonne
34:07
34:07
Play later
Play later
Lists
Like
Liked
34:07
If you are anything like us, for 36 hours your Twitter and LinkedIn feeds were set ablaze by claims that Llama 3 is the greatest LLM since sliced bread. But how do we know that is actually the case? How do we separate the hype and noise from the reality? On what basis can we credibly claim that one LLM is better than another? On this episode of The…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
The Importance of Decomposition in Prompt Engineering
16:00
16:00
Play later
Play later
Lists
Like
Liked
16:00
As AI language models become increasingly powerful, more and more people are discovering the potential of well-crafted prompts to boost productivity and solve complex problems. Justin and Brad share their expertise on designing effective prompts that adhere to rigorous engineering principles. They highlight real-world examples of prompts that have …
…
continue reading
![Artwork](/static/images/128pixel.png)
1
How LLM’s have changed MLOps with Andy McMahon
44:09
44:09
Play later
Play later
Lists
Like
Liked
44:09
What is LLM and ML Ops? How has the field of MLOps changed with the introduction of LLM models? What is the emerging field of Prompt Ops? How should we measure the effectiveness of our models in production? To answer these questions, we invited MLOps expert Andrew McMahon to chat. Check out Andrew's LinkedIn, Twitter, and book using these links: Li…
…
continue reading
Explore the current landscape and future possibilities of AI agents with machine learning experts Justin and Brad. They discuss the trade-offs between specialized and general-purpose agents, the importance of domain expertise, and the potential for agent aggregators. The conversation also covers the standardization of agent-to-agent communication a…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Prompt Injections to Coerce any LLM to do Anything You Want with Jonas Geiping
42:52
42:52
Play later
Play later
Lists
Like
Liked
42:52
Prompt Injections might be the single biggest risk to the deployment of LLM systems. In this episode of The Prompt Desk, your host Bradley Arsenault sits down with researcher Jonas Geiping to talk about his research on LLM security. Jonas and his colleagues have found optimization techniques that allow them to find prompts that they can put into an…
…
continue reading
Explore the six essential components that define an AI agent in 2024, including input/output, memory, loops and recursion, third-party actions, and comprehension. Join Brad and Justin as they discuss the criteria needed for a sophisticated AI system to be considered an agent, and how these requirements contribute to realizing the sci-fi vision of A…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
AI Agents in the Enterprise with Alan Wunsche
45:11
45:11
Play later
Play later
Lists
Like
Liked
45:11
Are you wondering how LLM Agents can be applied in large enterprises? Are we all going to become managers to teams of AI systems, cobbled together like a rag-tag group of robot butlers? In the latest episode of The Prompt Desk, hosts Justin Macorin and Bradley Arsenault sit down with Alan Wunsche to talk about the deployment and rollout of AI agent…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Programming 10x Faster with AI Augmentation
26:22
26:22
Play later
Play later
Lists
Like
Liked
26:22
In this episode of The Prompt Desk, hosts Justin Macorin and Bradley Arsenault discuss the various ways they have been using coding assistants, such as Github Copilot, in order to boost their productivity as software developers. The two exchange ideas, thoughts and techniques on how to better use these tools. — Continue listening to The Prompt Desk…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Prompt Design for AI Art with Sachin Kamath
42:35
42:35
Play later
Play later
Lists
Like
Liked
42:35
Join us with Sachin Kamath as we explore how to use AI in creative projects effectively and how we may effectively integrate AI into existing workflows, significantly enhancing productivity and creative output. We discuss frameworks for advanced prompting, ethical considerations, and best practices in AI. Consistent Character GPT: https://chat.open…
…
continue reading
Does Voodo in prompt engineering exist? The evolution from manual prompt crafting to automated refinement and optimization in AI interactions marks a transition from art to science. Let's discuss how initial natural language inputs transform, improving communication with large language models for more precise and effective outputs. Join us as we ex…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Using LLMs to Generate Source Code: Part 2
17:31
17:31
Play later
Play later
Lists
Like
Liked
17:31
This episode discusses improving code generation with LLMs by dynamically extracting functions and context, using techniques like Python's inspect modules and JavaScript's toString. The goal is to produce code that integrates well with existing projects, enhancing customization and practicality. — Continue listening to The Prompt Desk Podcast for e…
…
continue reading
Join us as we explore the advancements and challenges in implementing Retriever-Augmented Generation (RAG) systems for document retrieval and summarization, examining selective information inclusion, re-ranking strategies, and the importance of document parsing and query pre-processing. We discuss with Denys Linkov the practical applications, benef…
…
continue reading
Don't miss out on our discussion about the critical factors of expert domain knowledge, intellectual property, and the impact of first-mover advantage. These factors remain the most important competitive advantages, even in today's rapidly evolving technological landscape. — Continue listening to The Prompt Desk Podcast for everything LLM & GPT, Pr…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Prompt injection and LLM Security a Deep Dive with Peter Gostev
41:49
41:49
Play later
Play later
Lists
Like
Liked
41:49
In today's world, application security is more important than ever before. One of the most significant risks to application security is prompt injection. It can cause irreparable damage to your business. That's why it's critical to understand the various injection techniques, their impact on your business, and how you can mitigate the risks. — Cont…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Why Small Language Models are Going to Eat the World
20:06
20:06
Play later
Play later
Lists
Like
Liked
20:06
Let's discover the rise of local Small Language Models (SLMs), contrasting them with their larger, internet-dependent counterparts - LLMs. We discuss real-world applications like Google Maps, in-browser neural networks, and Apple Photos, examining the benefits of local installation for speed, reliability, privacy, and cost, and address the challeng…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Data-free Quality Analysis of Deep Neural Nets with Charles H. Martin
53:42
53:42
Play later
Play later
Lists
Like
Liked
53:42
In this episode, we interview Charles H Martin about his open-source Weight Watcher project ( found here https://weightwatcher.ai/ ), which provides ways to test the quality and fit of deep neural networks without having to rely upon a validation dataset. Given the scarcity of high-quality data and the complexity of modern multi-stage ML training a…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Why AI Systems are More Complex than they Seem
21:16
21:16
Play later
Play later
Lists
Like
Liked
21:16
Artificial Intelligence (AI) systems are often considered magical, but they are actually composed of several components, with LLMs being only one of them. It is essential to understand that AI systems utilize a range of components, including classification models, transformers, code execution, API calls, and IO read/write file calls, among others, …
…
continue reading
![Artwork](/static/images/128pixel.png)
1
How LLM-based Logic can Revolutionize Product Design
18:24
18:24
Play later
Play later
Lists
Like
Liked
18:24
Conditional logic powered by Large Language Models (LLMs) is transforming the traditional if/else statements in software applications, enabling quicker and more efficient integration of AI-based conditional logic. This advancement significantly enhances non-technical specialists' ability to customize complex applications easily. — Continue listenin…
…
continue reading
Should computers talk English to eachother? This podcasts discusses how computers can use natural language queries to execute API and database calls. Join us as we discuss the challenges, opportunities, and future potential of natural language processing in computer communication. — Continue listening to The Prompt Desk Podcast for everything LLM &…
…
continue reading
Dive deep into the world of prompt development where we break down the different steps of developing prompt-based applications. From your first prompt to a fine-tuned version, we unravel the intricacies of prompt engineering, exploring how to effectively craft, test, and refine prompts to optimize interaction and functionality in AI-driven applicat…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Using LLMs to Generate Source Code: The First Step
17:05
17:05
Play later
Play later
Lists
Like
Liked
17:05
Dive deep into the world of using Large Language Models (LLMs) to generate source code as we explore the importance of types and schema definitions. Join us as we discuss the challenges of generating accurate and reliable code, the role of types and schema in improving code quality, and how LLMs can be leveraged to enhance the development process. …
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Did OpenAI really just kill 1,000 Startups?
24:47
24:47
Play later
Play later
Lists
Like
Liked
24:47
Dive deep into OpenAI's newest press release as we discuss and examine the implications of their newest features, and draw parallels with past tech giants like AOL, Netscape, and Apple's Flashlight application. Join us for a thought-provoking journey through the evolution of AI and technology. — Continue listening to The Prompt Desk Podcast for eve…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Sandwich Prompts: Instructions, Data, and Everything in Between
22:22
22:22
Play later
Play later
Lists
Like
Liked
22:22
Dive deep into how instruction and data placement can have a significant impact on prompt and output quality. Join us as we explore how we use statistics to measure prompt quality and what data and instruction placements yield the best success. — Continue listening to The Prompt Desk Podcast for everything LLM & GPT, Prompt Engineering, Generative …
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Prompt Size: Best Practices From Zero-Shot to Few-Shot Prompts
13:14
13:14
Play later
Play later
Lists
Like
Liked
13:14
Dive deep into the art of prompt engineering as Brad and Justin share their personal experiences with machine learning models, unravel the mysteries of why shorter prompts often yield better results and the nuances of zero-shot vs. few-shot prompts. Whether you're seeking to understand the neural network mechanics or looking for best practices in c…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Decoupling for Clarity: The Future of Prompt-Based Development
17:22
17:22
Play later
Play later
Lists
Like
Liked
17:22
Dive into the world of prompt deleveopment best practices as we explore the importance of decoupling prompts from source code. Learn about the challenges of hard-coding, the rising trends in the realm of prompts engineering, and how proper decoupling can foster collaboration, clarity, and future-proof your projects. — Continue listening to The Prom…
…
continue reading
![Artwork](/static/images/128pixel.png)
1
Decoding LLM Quality: From Unit Testing to User Feedback
18:20
18:20
Play later
Play later
Lists
Like
Liked
18:20
Dive deep into the nuances of measuring the quality of Large Language Model (LLM) prompts, as we explore past methodologies, and evaluate both qualitative assessments and large-scale testing techniques. Join us as we discuss the challenges of traditional metrics, the role of user feedback, and brainstorm new ways to gauge generative model performan…
…
continue reading