Jesse Hoogland on Developmental Interpretability and Singular Learning Theory

The Inside View

Jesse Hoogland is a research assistant at David Krueger's lab in Cambridge studying AI Safety. More recently, Jesse has been thinking about Singular Learning Theory and Developmental Interpretability, which we discuss in this episode. Before he came to grips with existential risk from AI, he co-founded a health-tech startup automating bariatric surgery patient journeys.

(00:00) Intro

(03:57) Jesse’s Story And Probability Of Doom

(06:21) How Jesse Got Into Singular Learning Theory

(08:50) Intuition behind SLT: the loss landscape

(12:23) Does SLT actually predict anything? Phase Transitions

(14:37) Why care about phase transition, grokking, etc

(15:56) Detecting dangerous capabilities like deception in the (devel)opment

(17:24) A concrete example: magnets

(20:06) Why Jesse Is Bullish On Interpretability

(23:57) Developmental Interpretability

(28:06) What Happens Next? Jesse’s Vision

(31:56) Toy Models of Superposition

(32:47) Singular Learning Theory Part 2

(36:22) Are Current Models Creative? Reasoning?

(38:19) Building Bridges Between Alignment And Other Disciplines

(41:08) Where To Learn More About Singular Learning Theory

Make sure I upload regularly: https://patreon.com/theinsideview

Youtube: https://youtu.be/713KyknwShA

Transcript: https://theinsideview.ai/jesse

Jesse: https://twitter.com/jesse_hoogland

Host: https://twitter.com/MichaelTrazzi

Patreon supporters:

- Vincent Weisser

- Gunnar Höglund

- Ryan Coppolo

- Edward Huff

- Emil Wallner

- Jesse Hoogland

- William Freire

- Cameron Holmes

- Jacques Thibodeau

- Max Chiswick

- Jack Seroy

- JJ Hepburn