Auxiliary Turtles All the Way Down


Manage episode 253266734 series 2520036
By Sanjay Srivastava, Alexa Tullett, and Simine Vazire, Sanjay Srivastava, Alexa Tullett, and Simine Vazire. Discovered by Player FM and our community — copyright is owned by the publisher, not Player FM, and audio is streamed directly from their servers. Hit the Subscribe button to track updates in Player FM, or paste the feed URL into other podcast apps.

The path from theory to study consists of a thousand decisions, big and small. How and how much do these decisions matter? We discuss a recent crowdsourced meta-study that tried to find out. Fifteen teams of researchers were given 5 different hypotheses and told to design a study to test them, then they ran all the studies and got widely varying results. What are the implications of this study for how we should think about the role of theory in study design? What does it say about the different functions of direct and conceptual replications? Is this evidence of hidden moderators? How predictable were the differences in results, and were they predictable because of differences in the study designers' expertise, biases, or something else? Plus: We answer a letter about getting scooped on a systematic review.


The Black Goat is hosted by Sanjay Srivastava, Alexa Tullett, and Simine Vazire. Find us on the web at, on Twitter at @blackgoatpod, on Facebook at, and on instagram at @blackgoatpod. You can email us at You can subscribe to us on iTunes or Stitcher.

Our theme music is Peak Beak by Doctor Turtle, available on under a Creative Commons noncommercial attribution license. Our logo was created by Jude Weaver.

This is episode 75. It was recorded on January 30, 2020.

86 episodes