LW - We ran an AI safety conference in Tokyo. It went really well. Come next year! by Blaine

The Nonlinear Library: LessWrong

Link to original article
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: We ran an AI safety conference in Tokyo. It went really well. Come next year!, published by Blaine on July 18, 2024 on LessWrong. Abstract Technical AI Safety 2024 (TAIS 2024) was a conference organised by AI Safety 東京 and Noeon Research, in collaboration with Reaktor Japan, AI Alignment Network and AI Industry Foundation. You may have heard of us through ACX. The goals of the conference were 1. demonstrate the practice of technical safety research to Japanese researchers new to the field 2. share ideas among established technical safety researchers 3. establish a good international reputation for AI Safety 東京 and Noeon Research 4. establish a Schelling conference for people working in technical safety We sent out a survey after the conference to get feedback from attendees on whether or not we achieved those goals. We certainly achieved goals 1, 2 and 3; goal 4 remains to be seen. In this post we give more details about the conference, share results from the feedback survey, and announce our intentions to run another conference next year. Okay but like, what was TAIS 2024? Technical AI Safety 2024 (TAIS 2024) was a small non-archival open academic conference structured as a lecture series. It ran over the course of 2 days from April 5th-6th 2024 at the International Conference Hall of the Plaza Heisei in Odaiba, Tokyo. We had 18 talks covering 6 research agendas in technical AI safety: Mechanistic Interpretability Developmental Interpretability Scaleable Oversight Agent Foundations Causal Incentives ALIFE …including talks from Hoagy Cunningham (Anthropic), Noah Y. Siegel (DeepMind), Manuel Baltieri (Araya), Dan Hendrycks (CAIS), Scott Emmons (CHAI), Ryan Kidd (MATS), James Fox (LISA), and Jesse Hoogland and Stan van Wingerden (Timaeus). In addition to our invited talks, we had 25 submissions, of which 19 were deemed relevant for presentation. 5 were offered talk slots, and we arranged a poster session to accommodate the remaining 14. In the end, 7 people presented posters, 5 in person and 2 in absentia. Our best poster award was won jointly by Fazl Berez for Large Language Models Relearn Removed Concepts and Alex Spies for Structured Representations in Maze-Solving Transformers. We had 105 in-person attendees (including the speakers). Our live streams had around 400 unique viewers, and maxed out at 18 concurrent viewers. Recordings of the conference talks are hosted on our youtube channel. How did it go? Very well, thanks for asking! We sent out a feedback survey after the event, and got 68 responses from in-person attendees (58% response rate). With the usual caveats that survey respondents are not necessarily a representative sample of the population: Looking good! Let's dig deeper. How useful was TAIS 2024 for those new to the field? Event satisfaction was high across the board, which makes it hard to tell how relatively satisfied population subgroups were. Only those who identified themselves as "new to AI safety" were neutrally satisfied, but the newbies were also the most likely to be highly satisfied. It seems that people new to AI safety had no more or less trouble understanding the talks than those who work for AI safety organisations or have published AI safety research: They were also no more or less likely to make new research collaborations: Note that there is substantial overlap between some of these categories, especially for categories that imply a strong existing relationship to AI safety, so take the above charts with a pinch of salt: Total New to AI safety Part of the AI safety community Employed by an AI safety org Has published AI safety research New to AI safety 26 100% 19% 12% 4% Part of the AI safety community 28 18% 100% 36% 32% Employed by an AI safety org 20 15% 50% 100% 35% Has published AIS research 13 8% 69% 54% 100% Subjectively, it fe...