Beyond Embeddings: The Power of Rerankers in Modern Search | S2 E6

How AI Is Built

Today, we're talking to Aamir Shakir, the founder and baker at mixedbread.ai, where he's building some of the best embedding and re-ranking models out there. We go into the world of rerankers, looking at how they can classify, deduplicate documents, prioritize LLM outputs, and delve into models like ColBERT.

We discuss:

• The role of rerankers in retrieval pipelines
• Advantages of late interaction models like ColBERT for interpretability
• Training rerankers vs. embedding models and their impact on performance
• Incorporating metadata and context into rerankers for enhanced relevance
• Creative applications of rerankers beyond traditional search
• Challenges and future directions in the retrieval space

Still not sure whether to listen? Here are some teasers:

• Rerankers can significantly boost your retrieval system's performance without overhauling your existing setup.
• Late interaction models like ColBERT offer greater explainability by allowing token-level comparisons between queries and documents.
• Training a reranker often yields a higher impact on retrieval performance than training an embedding model.
• Incorporating metadata directly into rerankers enables nuanced search results based on factors like recency and pricing.
• Rerankers aren't just for search—they can be used for zero-shot classification, deduplication, and prioritizing outputs from large language models.
• The future of retrieval may involve compound models capable of handling multiple modalities, offering a more unified approach to search.

Aamir Shakir:

• LinkedIn
• X (Twitter)
• Mixedbread.ai

Nicolay Gerold:

• ⁠LinkedIn⁠
• ⁠X (Twitter)

00:00 Introduction and Overview 00:25 Understanding Rerankers 01:46 Maxsim and Token-Level Embeddings 02:40 Setting Thresholds and Similarity 03:19 Guest Introduction: Aamir Shakir 03:50 Training and Using Rerankers (Episode Start) 04:50 Challenges and Solutions in Reranking 08:03 Future of Retrieval and Recommendation 26:05 Multimodal Retrieval and Reranking 38:04 Conclusion and Takeaways