Best Serverless Kafka Podcasts (2024)

1
Optimizing Cloud Costs for SaaS Startups 29:35

3M ago29:35

29:35

You can't manage what you don't measure, and this includes your cloud costs. But how detailed should this measurement be? And how will the data translate into impact?I sat down with Adam Shugar, co-founder and CTO of Dashdive, to discuss his approach to cloud costs. He shared his advice, not only on cost cutting but also technology, growing a start…

1
SaaS: More than just a business model 52:04

6M ago52:04

52:04

Bill Tarr has the most interesting job in the world. He and his team of SaaS Evangelists at the AWS SaaS Factory work with companies large and small to help them build SaaS on AWS. In this conversation, we discuss the different ways technology and business interact when building SaaS - from cool technical options available only to SaaS companies to…

1
Building Streaming on S3 58:11

6M ago58:11

58:11

Ryan Worl, co-founder and CTO of WarpStream, is on a mission to re-engineer fundamental infrastructure on top of S3. Starting with metrics at Uber, continuing with Husky - DataDog's platform for events, logs and everything except metrics and then... Ryan thought "it will be cool to re-build Kafka on S3", reached out to other developers to hear thei…

1
Kora: Cloud Native Platform for Kafka 42:49

8M ago42:49

42:49

We invited Anna Povzner, director of engineering at Confluent, to the show to discuss Kora. Kora is a Cloud Native platform based on Apache Kafka, which her team at Confluent built. Anna and her team recently published a paper about Kora in VLDB 2023, and it won the industry's best paper award.Kora VLDB paper: https://www.vldb.org/pvldb/vol16/p3822…

1
Cell Based Architecture for Early Stage SaaS 55:12

8M ago55:12

55:12

In the last year or two I started hearing a lot about cell-based architectures. Usually in the form of “We had a lot of issues scaling our infrastructure, but then we moved to cell-based architectures” and “I wish I’ve learned about cell based architectures earlier, it would have saved me a lot of pain”. As a result, I’ve wanted to share knowledge …

1
Building a Serverless Streaming Platform 42:23

8M ago42:23

42:23

Krishna Raman has many years of experience building platforms for developers. Now he's applying this experience at Delta Streams to build a serverless platform for stream processing in SQL. In our conversation, we discussed the serverless developer experience, some of the secret sauce behind Delta Stream's Flink Operator, how to deliver a bring-you…

1
Never Rewrite! And other advice for SaaS Developers 36:31

11M ago36:31

36:31

You are a founding engineer at a SaaS startup. You built the MVP, and to everyone's great delight - usage is picking up. What's next? In this episode, Jeffrey shares how a messy MVP can gradually evolve into a scalable SaaS product. We discuss the critical design decisions engineers have to make in the early days of building the product: Tabs or sp…

1
The Wonders of Postgres Logical Decoding Messages 40:38

11M ago40:38

40:38

Did you know that Postgres that lets you write data that you can’t query? Events that will show up in the write-ahead log (WAL) of the database but not in any table. Gunnar Morling, senior staff engineer at Decodable and world expert on change data capture, walks us through data capture basics, this not-new but little-known feature, and dives into …

1
Postgres, Performance and Rails 57:58

11M ago57:58

57:58

Andrew Atkinson took a Rails web application that was struggling with load, and optimized it to handle over 9000 HTTP requests per second with an average latency of 35ms end to end. Handling a much higher load, on a smaller RDS instance, with lower latencies. He then shared his expertise by writing a book: "High-Performance Postgres with Rails."And…

1
Trends in Observability and Alerting 36:44

11M ago36:44

36:44

Ensuring your production system is well-behaved is table stakes for any SaaS. The space around monitoring and alerting is complex and moving fast, so it is too easy to end up with results that are worse than useless. Alert fatigue is real, and developers must learn to avoid this. Shahar and Tal, founders of Keep and active SaaS Developer community …

1
Cloudflare: Performance isolation in multi-tenant DB 36:14

11M ago36:14

36:14

Cloudflare is no longer "just" a CDN serving 55M HTTP requests/sec; they now offer a wide range of cloud services on the edge. These services run on a data layer with 15 Postgres clusters running hundreds of databases. Vignesh Ravichandran, engineering manager of Cloudflare's database team, joined us to discuss the challenges of running this large-…

1
Real-time Data Infrastructure - At Uber and Beyond 38:10

11M ago38:10

38:10

Pinot team at Uber wrote an excellent paper about the real-time analytics platform they built. Chinmay, formerly a principal engineer at Uber and now head of product at StarTree, joined me for a conversation.We discussed the challenges they encountered at Uber, the solutions they came up with, the platform they built, and how to best apply their ex…

1
Scalable Multi-tenant Platforms at Loom and at Times 47:03

11M ago47:03

47:03

Shayon wrote a great blog post on the guiding principles he and his team at Loom used to guide them as they evolved Loom's data platform through a period of hypergrowth. I invited Shayon to the show to discuss the challenges he encountered and how he solved them - and I learned that he is now at Tines - solving an entirely new set of challenges wit…

1
Building SaaS on Kafka Streams 53:48

12M ago53:48

53:48

Colt McNealy is re-imagining the future of microservices orchestration and he decided to build it entirely on Kafka Streams. In this conversation we discuss how Kafka Streams provides the low latency, reliability, availability and elasticity that is needed for the next generation of microservices orchestration. Colt also shares the most exciting up…

1
Transaction Isolation - Demystified! 31:11

12M ago31:11

31:11

If you used a relational database at all, you probably heard of transaction isolation levels. Transaction isolation levels have a massive impact on the behavior of your application - correctness, performance, and error rates. Your database may be distributed these days, so you may have to reason about distributed transactions too.In this video, I e…

1
Giving and Receiving Actually Useful Advice 17:16

12M ago17:16

17:16

YouTube and Twitter are full of “things developers should never do”. There's an endless demand for simple advice that applies in all situations.And that's not a bad thing. If there's a simple solution that works 80% of the time, this is valuable information. More practical than just "it depends." But advice-givers and advice-getters can do better. …

1
The Promise of Serverless 1:06:30

12M ago1:06:30

1:06:30

When developers talk about Serverless, they often focus on FaaS. But the best Serverless experience, by far, is delivered by a data store. S3. Why? Because it "just works" and lets developers focus on their code. Serverless databases help you focus on your queries and workload. They abstract the compute. Which also means - usage based pricing. In t…

1
Apache Kafka 3.5 - Kafka Core, Connect, Streams, & Client Updates 11:25

1y ago11:25

11:25

Apache Kafka® 3.5 is here with the capability of previewing migrations between ZooKeeper clusters to KRaft mode. Follow along as Danica Fine highlights key release updates. Kafka Core: KIP-833 provides an updated timeline for KRaft. KIP-866 now is preview and allows migration from an existing ZooKeeper cluster to KRaft mode. KIP-900 introduces a wa…

1
A Special Announcement from Streaming Audio 1:18

1+ y ago1:18

1:18

After recording 64 episodes and featuring 58 amazing guests, the Streaming Audio podcast series has amassed over 130,000 plays on YouTube in the last year. We're extremely proud of these achievements and feel that it's time to take a well-deserved break. Streaming Audio will be taking a vacation! We want to express our gratitude to you, our valued …

1
Airtable - Migrating a Multitenant Architecture to MySQL 8.0 38:34

1+ y ago38:34

38:34

The storage team at Airtable published a blog post describing, in detail, the migration of their petabyte-scale storage layer from MySQL 5.6 to MySQL 8.0. Andrew Wang, the lead of Airtable's storage team, joined us to discuss the migration, Airtable's storage architecture, data isolation levels, engineering culture, and more. The blog: https://medi…

1
How to use Data Contracts for Long-Term Schema Management 57:28

1+ y ago57:28

57:28

Have you ever struggled with managing data long term, especially as the schema changes over time? In order to manage and leverage data across an organization, it’s essential to have well-defined guidelines and standards in place around data quality, enforcement, and data transfer. To get started, Abraham Leal (Customer Success Technical Architect, …

1
How to use Python with Apache Kafka 31:57

1+ y ago31:57

31:57

Can you use Apache Kafka® and Python together? What’s the current state of Python support? And what are the best options to get started? In this episode, Dave Klein joins Kris to talk about all things Kafka and Python: the libraries, the tools, and the pros & cons. He also talks about the new course he just launched to support Python programmers en…

1
Next-Gen Data Modeling, Integrity, and Governance with YODA 55:55

1+ y ago55:55

55:55

In this episode, Kris interviews Doron Porat, Director of Infrastructure at Yotpo, and Liran Yogev, Director of Engineering at ZipRecruiter (formerly at Yotpo), about their experiences and strategies in dealing with data modeling at scale. Yotpo has a vast and active data lake, comprising thousands of datasets that are processed by different engine…

1
Migrate Your Kafka Cluster with Minimal Downtime 1:01:30

1+ y ago1:01:30

1:01:30

Migrating Apache Kafka® clusters can be challenging, especially when moving large amounts of data while minimizing downtime. Michael Dunn (Solutions Architect, Confluent) has worked in the data space for many years, designing and managing systems to support high-volume applications. He has helped many organizations strategize, design, and implement…

1
The Multitenant journey - From 0 to 500M ARR 55:16

1+ y ago55:16

55:16

SaaS applications are multi-tenant, so whether you are writing the first line of code in a new app or worried about scaling your successful SaaS fast enough - you need to be aware of multi-tenant requirements. Isolation, access control, perfornance, operations, scale and compliance In this video, Ram Subramanian, Nile CEO and SaaS Community founder…

1
Real-Time Data Transformation and Analytics with dbt Labs 43:41

1+ y ago43:41

43:41

dbt is known as being part of the Modern Data Stack for ELT processes. Being in the MDS, dbt Labs believes in having the best of breed for every part of the stack. Oftentimes folks are using an EL tool like Fivetran to pull data from the database into the warehouse, then using dbt to manage the transformations in the warehouse. Analysts can then bu…

1
What is the Future of Streaming Data? 41:29

1+ y ago41:29

41:29

What’s the next big thing in the future of streaming data? In this episode, Greg DeMichillie (VP of Product and Solutions Marketing, Confluent) talks to Kris about the future of stream processing in environments where the value of data lies in their ability to intercept and interpret data. Greg explains that organizations typically focus on the inf…

1
What can Apache Kafka Developers learn from Online Gaming? 55:32

1+ y ago55:32

55:32

What can online gaming teach us about making large-scale event management more collaborative in real-time? Ben Gamble (Developer Relations Manager, Aiven) has come to the world of real-time event streaming from an usual source: the video games industry. And if you stop to think about it, modern online games are complex, distributed real-time data s…

1
Apache Kafka 3.4 - New Features & Improvements 5:13

1+ y ago5:13

5:13

Apache Kafka® 3.4 is released! In this special episode, Danica Fine (Senior Developer Advocate, Confluent), shares highlights of the Apache Kafka 3.4 release. This release introduces new KIPs in Kafka Core, Kafka Streams, and Kafka Connect. In Kafka Core: KIP-792 expands the metadata each group member passes to the group leader in its JoinGroup sub…

1
How to use OpenTelemetry to Trace and Monitor Apache Kafka Systems 50:01

1+ y ago50:01

50:01

How can you use OpenTelemetry to gain insight into your Apache Kafka® event systems? Roman Kolesnev, Staff Customer Innovation Engineer at Confluent, is a member of the Customer Solutions & Innovation Division Labs team working to build business-critical OpenTelemetry applications so companies can see what’s happening inside their data pipelines. I…

1
What is Data Democratization and Why is it Important? 47:27

1+ y ago47:27

47:27

Data democratization allows everyone in an organization to have access to the data they need, and the necessary tools needed to use this data effectively. In short, data democratization enables better business decisions. In this episode, Rama Ryali, a Senior IT and Data Executive, chats with Kris Jenkins about the importance of data democratization…

1
Git for Data: Managing Data like Code with lakeFS 30:42

1+ y ago30:42

30:42

Is it possible to manage and test data like code? lakeFS is an open-source data version control tool that transforms object storage into Git-like repositories, offering teams a way to use the same workflows for code and data. In this episode, Kris sits down with guest Adi Polak, VP of DevX at Treeverse, to discuss how lakeFS can be used to facilita…

1
Compute-Storage Separation Explained 18:01

1+ y ago18:01

18:01

Gunnar Morling asked a great querstion on Twitter: ""Separating storage and compute" vs. "Predicate push-down" -- I can't quite square these two with each other. Is there a world where they co-exist, or is it just two opposing patterns/trends in DB tech. ?" Those are complimentary patterns and you definitely want them together. They appear contradi…

1
Using Kafka-Leader-Election to Improve Scalability and Performance 51:06

1+ y ago51:06

51:06

How does leader election work in Apache Kafka®? For the past 2 ½ years, Adithya Chandra, Staff Software Engineer at Confluent, has been working on Kafka scalability and performance, specifically partition leader election. In this episode, he gives Kris Jenkins a deep dive into the power of leader election in Kafka replication, why we need it, how i…

1
Real-Time Machine Learning and Smarter AI with Data Streaming 38:56

1+ y ago38:56

38:56

Are bad customer experiences really just data integration problems? Can real-time data streaming and machine learning be democratized in order to deliver a better customer experience? Airy, an open-source data-streaming platform, uses Apache Kafka® to help business teams deliver better results to their customers. In this episode, Airy CEO and co-fo…

Podcasts Worth a Listen

Serverless Kafka Podcasts

Podcasts Worth a Listen

Quick Reference Guide