Aligning AI, before it's too late, with Rebecca Gorman

London Futurists

Our guest in this episode is Rebecca Gorman, the co-founder and CEO of Aligned AI, a start-up in Oxford which describes itself rather nicely as working to get AI to do more of the things it should do and fewer of the things it shouldn’t.
Rebecca built her first AI system 20 years ago and has been calling for responsible AI development since 2010. With her co-founder Stuart Armstrong, she has co-developed several advanced methods for AI alignment, and she has advised the EU, UN, OECD and the UK Parliament on the governance and regulation of AI.
The conversation highlights the tools faAIr, EquitAI, and ACE, developed by Aligned AI. It also covers the significance of recent performance by Aligned AI software in the CoinRun test environment, which demonstrates the important principle of "overcoming goal misgeneralisation".
Selected follow-ups:

• buildaligned.ai
• Article: "Using faAIr to measure gender bias in LLMs"
• Article: "EquitAI: A gender bias mitigation tool for generative AI"
• Article: "ACE for goal generalisation"
• "CoinRun: Solving Goal Misgeneralisation" - a publication on arXiv
• Aligned AI repositories on GitHub
• "Specification gaming examples in AI" - article by Victoria Krakovna
• Rebecca Gorman speaking at the Cambridge Union on "This House Believes Artificial Intelligence Is An Existential Threat" (YouTube)

Music: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

Chapters

1. [Ad] What If? So What? (00:12:59)

2. (Cont.) Aligning AI, before it's too late, with Rebecca Gorman (00:13:00)