Go offline with the Player FM app!
[EN] Jean Zay Supercomputer, Large Language Models - Nathan Cassereau, Hatim Bourfoune
Manage episode 384868151 series 2909423
I met with Nathan Cassereau and Hatim Bourfoune from IDRIS, a national computing centre for the CNRS (the national research centre in France). Nathan and Hatim work on the Bloom project, an open source large language model, which was created using the Jean-Zay supercomputer.
Thanks to Nathan and Hatim I had the chance to take a look at the machine after our interview.
LLMs and AI/ML in general have created a lot of excitement. Hatim said he got into AI/ML himself, and he highlighted a Coursera course run by Andrew Ng.
Here are a few links:
- https://arxiv.org/abs/2211.05100 a paper on BLOOM on ArXiv
- https://github.com/ncassereau-idris/lm-evaluation-harness Evaluation of LM
- https://github.com/dptrsa-300/start_with_bloom Getting started with BLOOM on GitHub
- https://huggingface.co/bigscience/bloom Summary on BLOOM from Huggingface
- https://www.technologyreview.com/2022/07/12/1055817/inside-a-radical-new-project-to-democratize-ai/ a technology review on BLOOM by MIT
- https://towardsdatascience.com/run-bloom-the-largest-open-access-ai-model-on-your-desktop-computer-f48e1e2a9a32 another BLOOM article
- https://www.youtube.com/@CNRS-FIDLE YouTube channel by CNRS
- https://github.com/NVIDIA/Megatron-LM Megatron LM library used in the project
- https://github.com/microsoft/DeepSpeed DeepSpeed library used in the project
- https://pytorch.org PyTorch library
- https://www.genci.fr/en a national infrastructure to provide access to HPC (Grand Equipement National de Calcul Intensif) in France
- https://en.wikipedia.org/wiki/Jean_Zay brief summary of Jean Zay's life
- http://www.idris.fr/eng/jean-zay/jean-zay-presentation-eng.html The Jean Zay supercomputer at IDRIS/Paris-Saclay
Thank you for listening and your ongoing support. It means the world to us!
Support the show on Patreon https://www.patreon.com/codeforthought
Get in touch:
- Email mailto:code4thought@proton.me
- UK RSE Slack (ukrse.slack.com): @code4thought or @piddie
- US RSE Slack (usrse.slack.com): @Peter Schmidt
- Mastodon: https://fosstodon.org/@code4thought or @code4thought@fosstodon.org
- LinkedIn: https://www.linkedin.com/in/pweschmidt/ (personal Profile)
- LinkedIn: https://www.linkedin.com/company/codeforthought/ (Code for Thought Profile)
This podcast is licensed under the Creative Commons Licence: https://creativecommons.org/licenses/by-sa/4.0/
135 episodes
Manage episode 384868151 series 2909423
I met with Nathan Cassereau and Hatim Bourfoune from IDRIS, a national computing centre for the CNRS (the national research centre in France). Nathan and Hatim work on the Bloom project, an open source large language model, which was created using the Jean-Zay supercomputer.
Thanks to Nathan and Hatim I had the chance to take a look at the machine after our interview.
LLMs and AI/ML in general have created a lot of excitement. Hatim said he got into AI/ML himself, and he highlighted a Coursera course run by Andrew Ng.
Here are a few links:
- https://arxiv.org/abs/2211.05100 a paper on BLOOM on ArXiv
- https://github.com/ncassereau-idris/lm-evaluation-harness Evaluation of LM
- https://github.com/dptrsa-300/start_with_bloom Getting started with BLOOM on GitHub
- https://huggingface.co/bigscience/bloom Summary on BLOOM from Huggingface
- https://www.technologyreview.com/2022/07/12/1055817/inside-a-radical-new-project-to-democratize-ai/ a technology review on BLOOM by MIT
- https://towardsdatascience.com/run-bloom-the-largest-open-access-ai-model-on-your-desktop-computer-f48e1e2a9a32 another BLOOM article
- https://www.youtube.com/@CNRS-FIDLE YouTube channel by CNRS
- https://github.com/NVIDIA/Megatron-LM Megatron LM library used in the project
- https://github.com/microsoft/DeepSpeed DeepSpeed library used in the project
- https://pytorch.org PyTorch library
- https://www.genci.fr/en a national infrastructure to provide access to HPC (Grand Equipement National de Calcul Intensif) in France
- https://en.wikipedia.org/wiki/Jean_Zay brief summary of Jean Zay's life
- http://www.idris.fr/eng/jean-zay/jean-zay-presentation-eng.html The Jean Zay supercomputer at IDRIS/Paris-Saclay
Thank you for listening and your ongoing support. It means the world to us!
Support the show on Patreon https://www.patreon.com/codeforthought
Get in touch:
- Email mailto:code4thought@proton.me
- UK RSE Slack (ukrse.slack.com): @code4thought or @piddie
- US RSE Slack (usrse.slack.com): @Peter Schmidt
- Mastodon: https://fosstodon.org/@code4thought or @code4thought@fosstodon.org
- LinkedIn: https://www.linkedin.com/in/pweschmidt/ (personal Profile)
- LinkedIn: https://www.linkedin.com/company/codeforthought/ (Code for Thought Profile)
This podcast is licensed under the Creative Commons Licence: https://creativecommons.org/licenses/by-sa/4.0/
135 episodes
All episodes
×Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.