Episode 42: Aestimatio Scriptorum

Nullius in Verba

#Science #Smriti Mehta and Daniël Lakens #Daniël Lakens #Smriti Mehta

22:08

Host Francesca Amiker sits down with directors Joe and Anthony Russo, producer Angela Russo-Otstot, stars Millie Bobby Brown and Chris Pratt, and more to uncover how family was the key to building the emotional core of The Electric State . From the Russos’ own experiences growing up in a large Italian family to the film’s central relationship between Michelle and her robot brother Kid Cosmo, family relationships both on and off of the set were the key to bringing The Electric State to life. Listen to more from Netflix Podcasts . State Secrets: Inside the Making of The Electric State is produced by Netflix and Treefort Media.…

about a year ago 1:05:34

MP3•Episode home

In today’s episode, we discuss critically reading and appraising scientific articles. How do we select which articles to read carefully? Which heuristics are useful for assessing paper quality? And do open science practices actually lead to better quality papers? Enjoy.

Shownotes

Bacon, F. (1625). Of Studies.
PNAS Submissions contributed by NAS members "The contributing member submits the manuscript to PNAS along with the names of at least two experts in the field of the paper who have agreed to review the work and brief comments about why each of those reviewers was chosen." https://www.pnas.org/pb-assets/authors/ifora-1720190309383.pdf
How many p-values just below 0.05 should we expect across multiple tests? https://daniellakens.blogspot.com/2014/05/the-probability-of-p-values-as-function.html
Lakens, D. (2024). When and How to Deviate From a Preregistration. Collabra: Psychology, 10(1), 117094. https://doi.org/10.1525/collabra.117094
TIER protocol: https://www.projecttier.org/tier-protocol/protocol-4-0/
Gino fraud investigation and excel meta-data: https://datacolada.org/109
REAPPRAISED checklist: https://www.nature.com/articles/d41586-019-03959-6
Yang, Z., & Hung, I. W. (2021). Creative thinking facilitates perspective taking. Journal of Personality and Social Psychology, 120(2), 278.

74 episodes

Episode 42: Aestimatio Scriptorum

Nullius in Verba

19 subscribers

published about a year ago

MP3•Episode home

Shownotes

Bacon, F. (1625). Of Studies.
PNAS Submissions contributed by NAS members "The contributing member submits the manuscript to PNAS along with the names of at least two experts in the field of the paper who have agreed to review the work and brief comments about why each of those reviewers was chosen." https://www.pnas.org/pb-assets/authors/ifora-1720190309383.pdf
How many p-values just below 0.05 should we expect across multiple tests? https://daniellakens.blogspot.com/2014/05/the-probability-of-p-values-as-function.html
Lakens, D. (2024). When and How to Deviate From a Preregistration. Collabra: Psychology, 10(1), 117094. https://doi.org/10.1525/collabra.117094
TIER protocol: https://www.projecttier.org/tier-protocol/protocol-4-0/
Gino fraud investigation and excel meta-data: https://datacolada.org/109
REAPPRAISED checklist: https://www.nature.com/articles/d41586-019-03959-6
Yang, Z., & Hung, I. W. (2021). Creative thinking facilitates perspective taking. Journal of Personality and Social Psychology, 120(2), 278.

74 episodes

#Science #Smriti Mehta and Daniël Lakens #Daniël Lakens #Smriti Mehta

All episodes

1
Prologus 56: Probability Pyramiding (A. Neher) 19:47

1 day ago19:47

19:47

In preparation for our discussion of "Why Most Published Research Findings Are False" by John Ioannidis from 2005, we read a very similar paper published 40 years earlier: Neher, A. (1967). Probability Pyramiding, Research Error and the Need for Independent Replication. The Psychological Record, 17(2), 257–262. https://doi.org/10.1007/BF03393713…

1
Episode 55: Pseudoscientia 1:04:02

8 days ago1:04:02

1:04:02

In this episode, we discuss what separates science from pseudoscience and touch upon the demarcation problem, the recent controversial podcast called the Telepathy Tapes, and the movie Ghostbusters. Enjoy. Shownotes McLean v. Arkansas Pigliucci, M., & Boudry, M. (Eds.). (2019). Philosophy of pseudoscience: Reconsidering the demarcation problem . University of Chicago Press. Report of the Royal Commission to Investigate Animal Magnetism The Telepathy Tapes Frankfurt, H. G. (2009). On bullshit . Moberger, V. (2020). Bullshit, pseudoscience and pseudophilosophy. Theoria, 86 (5), 595–611. Ghostbusters (1984) - Venkman's ESP Test Scene…

1
Episode 54: Fabulae Coniurationis 1:04:17

22 days ago1:04:17

1:04:17

Conspiracy Stories Show Notes: Zeitgeist documentary: https://en.wikipedia.org/wiki/Zeitgeist_(film_series) Podcast Drang naar Samenhang: https://podcasts.apple.com/nl/podcast/drang-naar-samenhang/id1584797552 This is not a conspiracy theory documentary. https://www.everythingisaremix.info/tinact Parker, M. (2000). Human Science as Conspiracy Theory. The Sociological Review, 48(2_suppl), 191-207. https://doi.org/10.1111/j.1467-954X.2000.tb03527.x Douglas, K. M., Sutton, R. M., & Cichocka, A. (2017). The Psychology of Conspiracy Theories. Current Directions in Psychological Science, 26(6), 538-542. https://doi.org/10.1177/0963721417718261 Tage-gate episode of More of a Comment than a Question: https://moreofacomment.buzzsprout.com/1207223/episodes/5511751-tage-gate…

1
Episode 53: Fraus - II 51:11

5 weeks ago51:11

51:11

Broad, W. J., & Wade, N. (1983). Betrayers of the truth. New York : Simon and Schuster. http://archive.org/details/betrayersoftruth00broa Wolfgang Stroebe, Tom Postmes, & Russell Spears. (2012). Scientific Misconduct and the Myth of Self-Correction in Science. Perspectives on Psychological Science, 7(6), 670–688. https://doi.org/10.1177/1745691612460687 Zotero can track if you are citing retractions: https://retractionwatch.com/2019/06/12/want-to-check-for-retractions-in-your-personal-library-and-get-alerts-for-free-now-you-can/ 100% CI blog: The Untold Mystery of Rogue RA https://www.the100.ci/2024/12/18/rogue-ra/ Merton, R. K. (1957). Priorities in Scientific Discovery: A Chapter in the Sociology of Science. American Sociological Review, 22(6), 635–659. https://doi.org/10.2307/2089193 Senior RIKEN scientist involved in stem cell scandal commits suicide https://www.science.org/content/article/senior-riken-scientist-involved-stem-cell-scandal-commits-suicide Kis, A., Tur, E. M., Lakens, D., Vaesen, K., & Houkes, W. (2022). Leaving academia: PhD attrition and unhealthy research environments. PLOS ONE, 17(10), e0274976. https://doi.org/10.1371/journal.pone.0274976…

1
Episode 52: Fraus - I 1:05:58

7 weeks ago1:05:58

1:05:58

Babbage, C. (1830). Reflections on the Decline of Science in England: And on Some of Its Causes. B. Fellowes. Sokal, A. D. (1996). Transgressing the Boundaries: Toward a Transformative Hermeneutics of Quantum Gravity. Social Text, 46/47, 217. https://doi.org/10.2307/466856 Grievance studies: https://en.wikipedia.org/wiki/Grievance_studies_affair It is legal to own and/or read Mein Kampf in The Netherlands (and Germany). Hand, D. (2007). Deception and dishonesty with data: Fraud in science. Significance, 4(1), 22–25. https://doi.org/10.1111/j.1740-9713.2007.00215.x Gross, C. (2016). Scientific Misconduct. Annual Review of Psychology, 67(Volume 67, 2016), 693–711. https://doi.org/10.1146/annurev-psych-122414-033437 Paolo Macchiarini: https://www.science.org/content/article/macchiarini-guilty-misconduct-whistleblowers-share-blame-new-karolinska-institute The Truth about China’s Cash-for-Publication Policy: https://www.technologyreview.com/2017/07/12/150506/the-truth-about-chinas-cash-for-publication-policy/ Claudine Gay plagiarism: https://www.plagiarismtoday.com/2024/01/22/harvard-releases-details-of-claudine-gay-investigation/ Many Co-Authors: https://manycoauthors.org/ Paper describing a replication study where students make up data: Azrin, N. H., Holz, W., Ulrich, R., & Goldiamond, I. (1961). The control of the content of conversation through reinforcement. Journal of the Experimental Analysis of Behavior, 4, 25–30. Francesca Gino defamation case dismissed: https://www.thecrimson.com/article/2024/9/12/judge-dismisses-gino-lawsuit-defamation-charges/ Retractions in Social Influence of the work of Guéguen: https://www.tandfonline.com/doi/full/10.1080/15534510.2024.2431408 , https://www.tandfonline.com/doi/full/10.1080/15534510.2024.2431415 , https://www.tandfonline.com/doi/full/10.1080/15534510.2024.2431421 Diederik Stapel’s book: http://nick.brown.free.fr/stapel/FakingScience-20161115.pdf Merton, R. K. (1957). Priorities in Scientific Discovery: A Chapter in the Sociology of Science. American Sociological Review, 22(6), 635–659. https://doi.org/10.2307/2089193…

1
Episode 51: Quinquagesimus - II 52:55

9 weeks ago52:55

52:55

In this special two-part celebration, we answer questions submitted by our listeners. Thanks to Don Moore, Leif Nelson, Henry Wyneken, Charlotte Pennington, and Karan Paranganat for the questions featured in this episode. And thank you for joining us for 50 episodes!

1
Episode 50: Quinquagesimus - I 1:04:06

11 weeks ago1:04:06

1:04:06

In this special two-part celebration, we answer questions submitted by our listeners. Thanks to James Steele, Peder Isager, and Simen Leithe Tajet for the questions featured in this episode. And thank you for joining us for 50 episodes! Shownotes Roger Scruton Quote Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2003). The theoretical status of latent variables. Psychological Review, 110 (2), 203–219. https://doi.org/10.1037/0033-295X.110.2.203 Danermark, B., Ekström, M., & Karlsson, J. C. (2019). Explaining Society: Critical Realism in the Social Sciences (2nd ed.). Routledge. https://doi.org/10.4324/9781351017831 Maxwell, J. A., & Mittapalli, K. (2010). Realism as a Stance for Mixed Methods Research. In A. Tashakkori & C. Teddlie, SAGE Handbook of Mixed Methods in Social & Behavioral Research (pp. 145–168). SAGE Publications, Inc. https://doi.org/10.4135/9781506335193.n6 Vincent, S., & O’Mahoney, J. (2017). Critical realism and qualitative research: An introductory overview (G. Grandy, C. Cassell, & A. L. Cunliffe, Eds.; pp. 201–216). SAGE Publications Ltd. https://doi.org/10.4135/9781526430212 Danermark, B. (2019). Applied interdisciplinary research: A critical realist perspective. Journal of Critical Realism , 18 (4), 368–382. https://doi.org/10.1080/14767430.2019.1644983…

1
Episode 49: Valor Scientiae Psychologicae 1:08:50

15 weeks ago1:08:50

1:08:50

You can listen to the podcast More of a Comment Than a Question here: https://moreofacomment.buzzsprout.com/ Our joint episode is a response to the episode ‘Final Final Final Comments’: https://moreofacomment.buzzsprout.com/1207223/episodes/16055645-final-final-final-comments

1
Prologus 49: We Have to Break Up (R. B. Cialdini) 11:44

16 weeks ago11:44

11:44

In preparation for our next episode, a joint recording with our friends from More of a Comment than a Question, we read a paper by Robert Cialdini about the value of social psychology for the general public. Cialdini, R. B. (2009). We Have to Break Up. Perspectives on Psychological Science , 4 (1), 5–6. https://doi.org/10.1111/j.1745-6924.2009.01091.x…

1
Episode 48: Defectum 59:17

17 weeks ago59:17

59:17

How I Fail. Blog by Veronika Cheplygina https://veronikach.com/category/how-i-fail/ Arkin, R. (2011). Most Underappreciated: 50 Prominent Social Psychologists Describe Their Most Unloved Work. Oxford University Press. Kerr, N. L. (1998). HARKing: Hypothesizing After the Results are Known. Personality and Social Psychology Review, 2(3), 196–217. https://doi.org/10.1207/s15327957pspr0203_4 Sharpe, D. (2013). Why the resistance to statistical innovations? Bridging the communication gap. Psychological Methods, 18(4), 572–582. https://doi.org/10.1037/a0034177 Anti-Creativity Letters episode: https://nulliusinverba.podbean.com/e/prologus-23-the-anticreativity-letters-r-e-nisbett Rouder, J. N., Haaf, J. M., & Snyder, H. K. (2019). Minimizing Mistakes in Psychological Science. Advances in Methods and Practices in Psychological Science, 2(1), 3–11. https://doi.org/10.1177/2515245918801915 Firestein, S. (2015). Failure: Why Science Is So Successful (First Edition). Oxford University Press. Sternberg, R. J. (Ed.). (2019). My Biggest Research Mistake: Adventures and Misadventures in Psychological Research (1st edition). SAGE Publications, Inc.…

1
Episode 47: Inductio et Deductio 1:18:06

19 weeks ago1:18:06

1:18:06

In this episode, we delve into induction and deduction and talk further about issues related to generalizability. Shownotes Popper, K. The Logic of Scientific Discovery. (1953). Hutchinson & Co. (Originally published in 1935) Yarkoni, T. (2022). The generalizability crisis. Behavioral and Brain Sciences , 45 , e1. Mook, D. G. (1983). In defense of external invalidity. American psychologist, 38( 4), 379-387. Salmon, W. C. (1981). Rational Prediction. The British Journal for the Philosophy of Science , 32 (2), 115–125. https://doi.org/10.1093/bjps/32.2.115 Reichenbach, H. (1938) [2006], Experience and Prediction: An Analysis of the Foundations and the Structure of Knowledge , Chicago: University of Chicago Press. Senn, S. (2007). Statistical issues in drug development (2nd ed). John Wiley & Sons. Ernst, M. D. (2004). Permutation Methods: A Basis for Exact Inference. Statistical Science , 19 (4), 676–685. Bacon, F. (1620). Instauratio magna [Novum organum] . London: John Bill. Urbach, P. (1982). Francis Bacon as a Precursor to Popper. The British Journal for the Philosophy of Science , 33 (2), 113–132.…

1
Episode 46: Invaliditas Externa 1:03:02

21 weeks ago1:03:02

1:03:02

In this episode, we discuss the paper "In defense of external invalidity" by Douglas Mook. Shownotes Mook, D. G. (1983). In defense of external invalidity. American Psychologist, 38(4), 379–387. Mook, D. G. (1989). The myth of external validity. Everyday cognition in adulthood and late life, 25-43. The case of Phineas Gage was written up: Harlow, J. M. (1848). Passage of an iron rod through the head. The Boston Medical and Surgical Journal (1828-1851), 39(20)…

1
Prologus 46: In Defense of External Invalidity (D. G. Mook) 52:55

22 weeks ago52:55

52:55

A reading of the paper In Defense of External Invalidty by Douglas G. Mook, which will be discussed in the next episode. Mook, D. G. (1983). In defense of external invalidity. American Psychologist, 38(4), 379–387. https://doi.org/10.1037/0003-066X.38.4.379

1
Episode 45: Apprenticiatus 51:15

23 weeks ago51:15

51:15

In this episode, we discuss the role of apprenticeship in training scientists and researchers. What’s the difference between traditional apprenticeship and cognitive apprenticeship? Does graduate training live up to its promise as an apprenticeship model? What can we do to improve the modeling of skills that are to be taught during graduate training? Shownotes Collins, A., Brown, J. S., & Holum, A. (1991). Cognitive apprenticeship: Making thinking visible. American educator , 15 (3), 6-11. Gabrys, B. J., & Beltechi, A. (2012). Cognitive apprenticeship: The making of a scientist. In Reshaping doctoral education (pp. 144-155). Routledge. Casadevall, A., & Fang, F. C. (2016). Rigorous science: a how-to guide. MBio, 7(6), 10-1128. Alvesson, M., Gabriel, Y., & Paulsen, R. (2017). Return to meaning: A social science with something to say. Oxford University Press. Polanyi, M. (1958). Personal Knowledge: Towards a Post-Critical Philosophy (M. J. Nye, Ed.). University of Chicago Press.…

1
Episode 44: Reprehensio Scientiae Aperta 1:00:09

25 weeks ago1:00:09