Best Xrisk Podcasts (2024)

1
37 - Jaime Sevilla on AI Forecasting 1:44:25

26d ago1:44:25

1:44:25

Epoch AI is the premier organization that tracks the trajectory of AI - how much compute is used, the role of algorithmic improvements, the growth in data used, and when the above trends might hit an end. In this episode, I speak with the director of Epoch AI, Jaime Sevilla, about how compute, data, and algorithmic improvements are impacting AI, an…

1
36 - Adam Shai and Paul Riechers on Computational Mechanics 1:48:27

1M ago1:48:27

1:48:27

Sometimes, people talk about transformers as having "world models" as a result of being trained to predict text data on the internet. But what does this even mean? In this episode, I talk with Adam Shai and Paul Riechers about their work applying computational mechanics, a sub-field of physics studying how to predict random processes, to neural net…

1
New Patreon tiers + MATS applications 5:32

1M ago5:32

5:32

Patreon: https://www.patreon.com/axrpodcast MATS: https://www.matsprogram.org Note: I'm employed by MATS, but they're not paying me to make this video.

1
35 - Peter Hase on LLM Beliefs and Easy-to-Hard Generalization 2:17:24

3M ago2:17:24

2:17:24

How do we figure out what large language models believe? In fact, do they even have beliefs? Do those beliefs have locations, and if so, can we edit those locations to change the beliefs? Also, how are we going to get AI to perform tasks so hard that we can't figure out if they succeeded at them? In this episode, I chat with Peter Hase about his re…

1
34 - AI Evaluations with Beth Barnes 2:14:02

3M ago2:14:02

2:14:02

How can we figure out if AIs are capable enough to pose a threat to humans? When should we make a big effort to mitigate risks of catastrophic AI misbehaviour? In this episode, I chat with Beth Barnes, founder of and head of research at METR, about these questions and more. Patreon: patreon.com/axrpodcast Ko-fi: ko-fi.com/axrpodcast The transcript:…

1
33 - RLHF Problems with Scott Emmons 1:41:24

5M ago1:41:24

1:41:24

Reinforcement Learning from Human Feedback, or RLHF, is one of the main ways that makers of large language models make them 'aligned'. But people have long noted that there are difficulties with this approach when the models are smarter than the humans providing feedback. In this episode, I talk with Scott Emmons about his work categorizing the pro…

1
32 - Understanding Agency with Jan Kulveit 2:22:29

5M ago2:22:29

2:22:29

What's the difference between a large language model and the human brain? And what's wrong with our theories of agency? In this episode, I chat about these questions with Jan Kulveit, who leads the Alignment of Complex Systems research group. Patreon: patreon.com/axrpodcast Ko-fi: ko-fi.com/axrpodcast The transcript: axrp.net/episode/2024/05/30/epi…

1
31 - Singular Learning Theory with Daniel Murfet 2:32:07

6M ago2:32:07

2:32:07

What's going on with deep learning? What sorts of models get learned, and what are the learning dynamics? Singular learning theory is a theory of Bayesian statistics broad enough in scope to encompass deep neural networks that may help answer these questions. In this episode, I speak with Daniel Murfet about this research program and what it tells …

1
30 - AI Security with Jeffrey Ladish 2:15:44

6M ago2:15:44

2:15:44

Top labs use various forms of "safety training" on models before their release to make sure they don't do nasty stuff - but how robust is that? How can we ensure that the weights of powerful AIs don't get leaked or stolen? And what can AI even do these days? In this episode, I speak with Jeffrey Ladish about security and AI. Patreon: patreon.com/ax…

1
29 - Science of Deep Learning with Vikrant Varma 2:13:46

7M ago2:13:46

2:13:46

In 2022, it was announced that a fairly simple method can be used to extract the true beliefs of a language model on any given topic, without having to actually understand the topic at hand. Earlier, in 2021, it was announced that neural networks sometimes 'grok': that is, when training them on certain tasks, they initially memorize their training …

1
28 - Suing Labs for AI Risk with Gabriel Weil 1:57:30

7M ago1:57:30

1:57:30

How should the law govern AI? Those concerned about existential risks often push either for bans or for regulations meant to ensure that AI is developed safely - but another approach is possible. In this episode, Gabriel Weil talks about his proposal to modify tort law to enable people to sue AI companies for disasters that are "nearly catastrophic…

1
27 - AI Control with Buck Shlegeris and Ryan Greenblatt 2:56:05

7M ago2:56:05

2:56:05

A lot of work to prevent AI existential risk takes the form of ensuring that AIs don't want to cause harm or take over the world---or in other words, ensuring that they're aligned. In this episode, I talk with Buck Shlegeris and Ryan Greenblatt about a different approach, called "AI control": ensuring that AI systems couldn't take over the world, e…

1
Ep 57: India's Statistical Story 56:52

10M ago56:52

56:52

I spoke to Pramit Bhattacharya an independent data journalist about India's statistical system. We talked about How did Indian statisticians adapt their statistical methods to account for India's informal sector? How are India's GDP numbers constructed? Why it's so hard to outright manipulate GDP numbers The case for optimism…

1
Ep 56: Talking AI Regulation with Harry Law 45:59

10M ago45:59

45:59

I talked with Harry Law of the University of Cambridge and Google DeepMind about AI regulation We talked about Me vs Harry on open source AI regulation Being humble when regulating AI Why a "Manhattan Project for X" is overrated

1
Ep 55: Adaptive markets and crypto with Derek Wong 48:58

11M ago48:58

48:58

I spoke with Derek Wong about the adaptive markets hypothesis, macro investing and investing in cryptocurrency markets We talk about Why investors should consider the financial markets as a complex adaptive system How investing in China is very different from investing in the West How he invests in crypto without fundamental anchors…

1
Ep 54: High Skilled Immigration in America 44:45

11M ago44:45

44:45

I spoke to Adam Ozimek and Connor O'Brien from the Economic Innovation Group about the policy and politics of high skilled immigration in America. We talked about Why reforming high skilled immigration in the US is so difficult The lump of labour fallacy in immigration Reforming immigration through place-based visas…

1
26 - AI Governance with Elizabeth Seger 1:57:13

12M ago1:57:13

1:57:13

The events of this year have highlighted important questions about the governance of artificial intelligence. For instance, what does it mean to democratize AI? And how should we balance benefits and dangers of open-sourcing powerful AI systems such as large language models? In this episode, I speak with Elizabeth Seger about her research on these …

1
25 - Cooperative AI with Caspar Oesterheld 3:02:09

1y ago3:02:09

3:02:09

Imagine a world where there are many powerful AI systems, working at cross purposes. You could suppose that different governments use AIs to manage their militaries, or simply that many powerful AIs have their own wills. At any rate, it seems valuable for them to be able to cooperatively work together and minimize pointless conflict. How do we ensu…

1
Ep 53: The State of Indian Cities ft. Devashish Dhar 56:10

1y ago56:10

56:10

I spoke with Devashish Dhar, the author of the excellent book India's Blind Spot which talks about India's urbanisation crisis and solutions to it. We talk about Why does India have a much lower reported rate of urbanisation than the rest of the world? Explaining the global bias against cities “Extremely high levels of traffic is caused by poor lan…

1
Ep 52: Tyler Cowen on Singapore, AI and Economic Growth 1:07:01

1y ago1:07:01

1:07:01

I interviewed one of the most interesting thinkers today, Tyler Cowen. We talked about Why there are such few Singaporean famous people What Singapore can do to get more weird Why he's sceptical of an AI-driven singularity What happens to kids in a post-GPT world What happens to public intellectuals in a post-GPT world Why he's optimistic on Kenyan…

1
Ep 51: Why AI won't kill us all ft. Rohit Krishnan 1:05:52

1y ago1:05:52

1:05:52

I spoke to Rohit Krishnan the author of the blog Strange Loop Canon about why he is sceptical about the idea that AI will kill us all We talked about Why he’s sceptical of AI regulation proposals Why AI “timelines” are not as meaningful as you think AI deployment is harder than you think! The value of incrementalism in AI policy Why he thinks instr…

1
Ep. 50: mRNA vaccines in India ft. Soham Sankaran 1:01:43

1y ago1:01:43

1:01:43

I spoke to Soham Sankaran who runs PopVax, an Indian mRNA vaccine company. Their goal is to build low-cost broadly-protective vaccines to protect against the entire sarbecovirus species. Read Soham's experience here (https://chronicles.popvax.com/p/three-meetings-and-six-million-funerals) as a complement to this episode. Also check out their jobs p…

1
24 - Superalignment with Jan Leike 2:08:29

1+ y ago2:08:29

2:08:29

Recently, OpenAI made a splash by announcing a new "Superalignment" team. Lead by Jan Leike and Ilya Sutskever, the team would consist of top researchers, attempting to solve alignment for superintelligent AIs in four years by figuring out how to build a trustworthy human-level AI alignment researcher, and then using it to solve the rest of the pro…

1
23 - Mechanistic Anomaly Detection with Mark Xu 2:05:52

1+ y ago2:05:52

2:05:52

Is there some way we can detect bad behaviour in our AI system without having to know exactly what it looks like? In this episode, I speak with Mark Xu about mechanistic anomaly detection: a research direction based on the idea of detecting strange things happening in neural networks, in the hope that that will alert us of potential treacherous tur…

1
Survey, store closing, Patreon 4:26

1+ y ago4:26

4:26

Very brief survey: bit.ly/axrpsurvey2023 Store is closing in a week! Link: store.axrp.net/ Patreon: patreon.com/axrpodcast Ko-fi: ko-fi.com/axrpodcast

1
Ep 49: Dwarkesh Patel - podcasting, Robert Moses, Effective Altruism and AI xrisk 51:24

1+ y ago51:24

51:24

I talked to Dwarkesh Patel of the Lunar Society Podcast about many topics. We talked about: Why do AI researchers and rationalists disagree about existential risk? What would happen if Robert Moses ran San Francisco? Is localism overrated? What does Effective Altruism get right and wrong? Which politicians would he like to interview…

1
22 - Shard Theory with Quintin Pope 3:28:21

1+ y ago3:28:21

3:28:21

What can we learn about advanced deep learning systems by understanding how humans learn and form values over their lifetimes? Will superhuman AI look like ruthless coherent utility optimization, or more like a mishmash of contextually activated desires? This episode's guest, Quintin Pope, has been thinking about these questions as a leading resear…

1
Ep. 47: The World's Biggest Invisible Country: Indonesia 43:47

1+ y ago43:47

43:47

I talked to Faris Abdurrachman about Indonesia and it's nickel boom. We talked specifically about Why Indonesia banned exports of raw nickel Who gets the value from nickel exports Indonesia's new sovereign wealth fund

1
Ep 46: Growth teams with Kartik Akileswaran 1:23:27

1+ y ago1:23:27

1:23:27

I spoke to Kartik Akileswaran who runs Growth Teams - an initiative which helps build state capacity for economic growth in developing countries. We talked about - Why implementation is a binding constraint for economic policy - How industrial policy helps reduce information constraints for investors - Underrated growth reforms…

1
21 - Interpretability for Engineers with Stephen Casper 1:56:02

1+ y ago1:56:02

1:56:02

Lots of people in the field of machine learning study 'interpretability', developing tools that they say give us useful information about neural networks. But how do we know if meaningful progress is actually being made? What should we want out of these tools? In this episode, I speak to Stephen Casper about these questions, as well as about a benc…

1
20 - 'Reform' AI Alignment with Scott Aaronson 2:27:35

1+ y ago2:27:35

2:27:35

How should we scientifically think about the impact of AI on human civilization, and whether or not it will doom us all? In this episode, I speak with Scott Aaronson about his views on how to make progress in AI alignment, as well as his work on watermarking the output of language models, and how he moved from a background in quantum complexity the…

1
Store, Patreon, Video 2:39

2y ago2:39

2:39

Store: https://store.axrp.net/ Patreon: https://www.patreon.com/axrpodcast Ko-fi: https://ko-fi.com/axrpodcast Video: https://www.youtube.com/watch?v=kmPFjpEibu0

1
19 - Mechanistic Interpretability with Neel Nanda 3:52:47

2y ago3:52:47

3:52:47

How good are we at understanding the internal computation of advanced machine learning models, and do we have a hope at getting better? In this episode, Neel Nanda talks about the sub-field of mechanistic interpretability research, as well as papers he's contributed to that explore the basics of transformer circuits, induction heads, and grokking. …

1
Ep 45: What's going on with nuclear weapons? 37:21

2y ago37:21

37:21

I spoke to Matt Korda who works at the Federation of American Scientists on nuclear weapon policy. We have an exciting discussion about the role of nuclear weapons, their growth and the dangerous arms races that are starting Some highlights of the show: The advent of “exotic” nuclear weapon systems China’s nuclear strategy has changed dramatically!…

1
New podcast - The Filan Cabinet 1:18

2y ago1:18

1:18

I have a new podcast, where I interview whoever I want about whatever I want. It's called "The Filan Cabinet", and you can find it wherever you listen to podcasts. The first three episodes are about pandemic preparedness, God, and cryptocurrency. For more details, check out the podcast website (thefilancabinet.com), or search "The Filan Cabinet" in…

1
18 - Concept Extrapolation with Stuart Armstrong 1:46:19

2y ago1:46:19

1:46:19

Concept extrapolation is the idea of taking concepts an AI has about the world - say, "mass" or "does this picture contain a hot dog" - and extending them sensibly to situations where things are different - like learning that the world works via special relativity, or seeing a picture of a novel sausage-bread combination. For a while, Stuart Armstr…

1
17 - Training for Very High Reliability with Daniel Ziegler 1:00:59

2y ago1:00:59

1:00:59

Sometimes, people talk about making AI systems safe by taking examples where they fail and training them to do well on those. But how can we actually do this well, especially when we can't use a computer program to say what a 'failure' is? In this episode, I speak with Daniel Ziegler about his research group's efforts to try doing this with present…

1
Ep 44: Trade Policy Tragedy in India 50:30

2+ y ago50:30

50:30

I talked to Anupam Manur, a professor of economics about India's trade policy before 1991. We talked about: The scarcity mindset about foreign exchange reserves The controversial 1966 devaluation How did the pre-1991 import licensing system work? “The financial account was almost non existent” “Hindustan Motors and Toyota were set up at the same ti…

1
Ep 43: Funding Science and Innovation in the UK 54:06

2+ y ago54:06

54:06

I spoke to Professor Richard Jones, about how science funding in the UK could improve. Some interesting questions we talked about are “Penny wise, pound foolish” in science funding Creating markets for technological advances How he’d invest a billion £ to accelerate scientific innovation?

1
Ep 42: Parliamentarism 50:02

2+ y ago50:02

50:02

I talked to Tiago Santos, a diplomat, about his book Why Not Parliamentarism. Tiago and I explore some questions here What makes parliamentary democracies superior to presidential ones? The creeping presidentialisation of parliamentary democracies The optimal rate of constitutional amendments

1
16 - Preparing for Debate AI with Geoffrey Irving 1:04:49

2+ y ago1:04:49

1:04:49

Many people in the AI alignment space have heard of AI safety via debate - check out AXRP episode 6 (axrp.net/episode/2021/04/08/episode-6-debate-beth-barnes.html) if you need a primer. But how do we get language models to the stage where they can usefully implement debate? In this episode, I talk to Geoffrey Irving about the role of language model…

1
Ep 41: Libertarianism and podcasting with Amit Varma 1:13:41

2+ y ago1:13:41

1:13:41

I talked to Amit Varma who runs one of my favourite podcasts - The Seen and the Unseen about politics, economics and public policy. We talked about Libertarianism within the Indian canon Cultivating your audience Being a public intellectual The differences between generations

1
Ep 40: Progress Studies 48:24

2+ y ago48:24

48:24

I spoke to Jason Crawford of The Roots of Progress about the new movement of Progress Studies. We talked about Building a culture of economic progress Why are developed countries more averse to progress? Is there a tradeoff between economic progress and existential risk? What is the main constraint for the movement today?…

1
Ep 39: Macro Investing 53:41

2+ y ago53:41

53:41

I talked with Mayank Seksaria of Liberty Mutual Investments about investing on macroeconomic views. We talked about Translating macro views into investing allocations A bottom up view of the macroeconomy Evaluating macro talent Why does institutional research cost so much?

1
15 - Natural Abstractions with John Wentworth 1:36:30

2+ y ago1:36:30

1:36:30

Why does anybody care about natural abstractions? Do they somehow relate to math, or value learning? How do E. coli bacteria find sources of sugar? All these questions and more will be answered in this interview with John Wentworth, where we talk about his research plan of understanding agency via natural abstractions. Topics we discuss, and timest…

1
Ep 38: The Bond King 42:24

2+ y ago42:24

42:24

I spoke to Mary Childs who is the author of the exceptional book The Bond King. We talked about How finance became an interesting profession How do you build institutions that succeed at investing? Can we automate the Fed? Financial history being undervalued

1
Ep 37: Australia: A Mine with a Parliament? 57:46

2+ y ago57:46

57:46

I spoke to Steven Hamilton professor of Economics at George Washington University about Australian economic policy, and their upcoming elections. We talk about Why was Australian COVID policy so strict? Australia as a nation of prison guards Economic issues of the Australian election “Australia is a mine with a parliament” Dutch disease in Australi…

1
Ep 36: Labour Economics Versus the World 1:13:58

2+ y ago1:13:58

1:13:58

What is the labour market like? What are the largest barriers in the labour market? Nathan Young and I spoke to economist Bryan Caplan about his new book Labor Econ Versus The World. We also talk about Censorship and dictatorships Bets he is willing to take Malengo and international migration DALLE-2 and writing graphic novels The literature on edu…

1
Working on Existential Risks with David Manheim 39:35

2+ y ago39:35

39:35

I spoke to David Manehim who works on reducing existential threats to humanity at the Technion. We talked about The biggest threats to humanity Preventing all future pandemics Is working on X-risk even tractable? How you can work on reducing existential risk Very fun on an underrated topic!

1
Ep 34: Long Run Growth with Trevor Chow 1:09:59

2+ y ago1:09:59

1:09:59

I spoke to the very very talented Trevor Chow about the history of long run growth. Topics include Wishlist of economic history topics Getting over vetocracy Qualities of the best economic blogs Why more economics graduates should join a VC firm! Meme theory of money Highly highly recommended

Podcasts Worth a Listen

Xrisk Podcasts

Podcasts Worth a Listen

Quick Reference Guide