Artwork

Content provided by Emily M. Bender and Alex Hanna, Emily M. Bender, and Alex Hanna. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Emily M. Bender and Alex Hanna, Emily M. Bender, and Alex Hanna or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.
Player FM - Podcast App
Go offline with the Player FM app!

Episode 28: LLMs Are Not Human Subjects, March 4 2024

1:00:57
 
Share
 

Manage episode 406215352 series 3483641
Content provided by Emily M. Bender and Alex Hanna, Emily M. Bender, and Alex Hanna. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Emily M. Bender and Alex Hanna, Emily M. Bender, and Alex Hanna or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.

Alex and Emily put on their social scientist hats and take on the churn of research papers suggesting that LLMs could be used to replace human labor in social science research -- or even human subjects. Why these writings are essentially calls to fabricate data.

References:

PNAS: ChatGPT outperforms crowd workers for text-annotation tasks

Political Analysis: Out of One, Many: Using Language Models to Simulate Human Samples

Behavioral Research Methods: Can large language models help augment English psycholinguistic datasets?

Information Systems Journal: Editorial: The ethics of using generative AI for qualitative data analysis

Fresh AI Hell:

Advertising vs. reality, synthetic Willy Wonka edition

A news outlet used an LLM to generate a story...and it falsely quoted Emily

Trump supporters target Black voters with faked AI images

You can check out future livestreams at https://twitch.tv/DAIR_Institute.

Follow us!
Emily

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

  continue reading

30 episodes

Artwork
iconShare
 
Manage episode 406215352 series 3483641
Content provided by Emily M. Bender and Alex Hanna, Emily M. Bender, and Alex Hanna. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Emily M. Bender and Alex Hanna, Emily M. Bender, and Alex Hanna or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.

Alex and Emily put on their social scientist hats and take on the churn of research papers suggesting that LLMs could be used to replace human labor in social science research -- or even human subjects. Why these writings are essentially calls to fabricate data.

References:

PNAS: ChatGPT outperforms crowd workers for text-annotation tasks

Political Analysis: Out of One, Many: Using Language Models to Simulate Human Samples

Behavioral Research Methods: Can large language models help augment English psycholinguistic datasets?

Information Systems Journal: Editorial: The ethics of using generative AI for qualitative data analysis

Fresh AI Hell:

Advertising vs. reality, synthetic Willy Wonka edition

A news outlet used an LLM to generate a story...and it falsely quoted Emily

Trump supporters target Black voters with faked AI images

You can check out future livestreams at https://twitch.tv/DAIR_Institute.

Follow us!
Emily

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

  continue reading

30 episodes

All episodes

×
 
Loading …

Welcome to Player FM!

Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.

 

Quick Reference Guide