The Anatomy of AI: The Tech Behind the Intelligence

Chaos Lever Podcast

AI's infrastructure, from the coding languages that power the models, to the networking that connects it all. Plus, a look into what effects this has on our planet.

AI's Infrastructure: How It's Built and Powered

Ned and Chris embark on a journey through the world of AI infrastructure, touching on key software and hardware components that make AI tick. From the basics of Python and TensorFlow to the power-hungry NVIDIA DGX servers, this episode covers everything you need to know about the backbone of AI. They also explore the rapid world of InfiniBand networking, highlighting its importance as well as the challenges it faces against Ethernet advancements. Finally, Ned and Chris reflect on the environmental impact of AI's power consumption, humorously suggesting we might need to find a new planet or a better power source sooner than later.

Links:

• I bet OpenAI has published something about their stack… sorta
• According to a post from 2016, OpenAI was using Python, TensorFlow, Numpy, Keras, and Anaconda
• TensorFlow is a machine learning library with APIs available for Python and C++
• Numpy is a Python package developed to support scientific computing
• Flyte, a graduated project from the Linux Foundation that leverages Kubernetes
• Nvidia H100 Spec Sheet
• DGX SuperPod
• Tensor Cores and CUDA cores, what is the difference?
• Nvidia helpfully posted some example code using the cuDNN library
• The Infiniband standard is a different protocol than Ethernet
• Ultra Ethernet Consortium is focused on developing an open standard that meets or exceeds what Infiniband does today
• Remember when Crypto was using as much power as the country of Argentina
• AI is already on pace to match that by 2027
• Microsoft’s AI for Beginners site to get down some of the terminology and lab time
• You may also want to take a beginner course on Python