Podd: Everything Hertz

190: What happens when you pay reviewers?

2 april 2025 | 44 min

189: Crit me baby, one more time

2 mars 2025 | 54 min

188: Double-blind peer review vs. scientific integrity

30 januari 2025 | 55 min

187: What started the replication crisis era?

3 december 2024 | 55 min

186: Evaluating journal quality

13 november 2024 | 43 min

185: The Retraction

4 oktober 2024 | 69 min

184: A race to the bottom

5 september 2024 | 48 min

183: Too beautiful to be true

3 augusti 2024 | 45 min

Dan and James discuss a paper describing a journal editor's efforts to receive data from authors who submitted papers with results that seemed a little too beautiful to be true

Main edisode takeaways (AI generated summary)

This editorial on the reproducibility crisis emphasizes the importance of providing raw data in scientific publications and highlights the need for transparency and accountability in the research process
The lack of oversight and the discrepancy between the amount of data required for scientific statements and what is often provided in academic publishing is a cause for concern.
Ensuring the integrity of scientific research requires the active involvement of editors, reviewers, and researchers in promoting transparency and upholding ethical standards. The scientific publishing process lacks oversight and accountability, leading to potential issues with the accuracy and trustworthiness of published papers.
Journals should prioritize maintaining high standards and ensuring that papers are thoroughly reviewed and validated before publication.
Changing behaviors within the scientific community, such as pledging to publish in open access journals, can promote positive change and improve research integrity.
There is a need for active maintenance and improvement of the systems and parameters of scientific research to prevent potential negative consequences.

Links for papers we mentioned

The Molecular Brain editorial by Miyakawa: https://doi.org/10.1186/s13041-020-0552-2
The STALT preprint: https://osf.io/6hste

Other links

Everything Hertz on social media

Support us on Patreon and get bonus stuff!

$1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
$5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D. S., & Heathers, J. (2024, Aug 3). 183: Too beautiful to be true Everything Hertz [Audio podcast], https://doi.org/10.17605/OSF.IO/JF5MS

Support Everything Hertz

182: What practices should the behavioural sciences borrow (and ignore) from other research fields?

2 juli 2024 | 51 min

181: Down the rabbit hole

3 juni 2024 | 43 min

180: Consortium peer reviews

2 maj 2024 | 50 min

179: Discovery vs. maintenance

3 april 2024 | 49 min

178: Alerting researchers about retractions

29 februari 2024 | 50 min

177: Plagiarism

31 januari 2024 | 43 min

176: Tracking academic workloads

29 december 2023 | 36 min

175: Defending against the scientific dark arts

7 december 2023 | 38 min

174: Smug missionaries with test tubes

1 november 2023 | 53 min

173: How do science journalists evaluate psychology papers?

1 oktober 2023 | 35 min

172: In defence of the discussion section

31 augusti 2023 | 36 min

171: The easiest person to fool is yourself (with Daniel Simons and Christopher Chabris)

20 juli 2023 | 56 min

170: Holy sheet

23 juni 2023 | 51 min

169: Using big data to understand behavior (Live episode with Sandra Matz)

31 maj 2023 | 44 min

168: Meta-meta-science

27 april 2023 | 48 min

167: Diluted effect sizes

16 mars 2023 | 44 min

166: Is science becoming less disruptive over time?

25 januari 2023 | 52 min

165: Self-promotion

30 december 2022 | 41 min

164: The great migration

28 november 2022 | 49 min

163: eLife's new peer review model

7 november 2022 | 55 min

162: Status bias in peer review

17 oktober 2022 | 51 min

161: The memo (with Brian Nosek)

12 september 2022 | 48 min

160: Whistleblowing

31 augusti 2022 | 51 min

159: Peer review isn't working (with Saloni Dattani)

15 augusti 2022 | 52 min

158: Word limits

1 augusti 2022 | 45 min

157: Limitations

11 juli 2022 | 47 min

156: Looking for seeders

21 juni 2022 | 51 min

155: Don't you know who I am?

30 maj 2022 | 46 min

154: When the evidence is constructed around the narrative

9 maj 2022 | 51 min

153: Shame shame shame

18 april 2022 | 47 min

152: Sorry Not Sorry

4 april 2022 | 56 min

151: The dirty dozen

21 mars 2022 | 40 min

150: Why can't you do nothing?

28 februari 2022 | 53 min

149: Medical misinformation (with Rohin Francis)

14 februari 2022 | 57 min

148: Academic reference letters

31 januari 2022 | 52 min

147: The $7000 golden ticket

17 januari 2022 | 54 min

146: Skills pay bills

27 december 2021 | 70 min

We answer a series of questions from a listener on whether to start a PhD, what to ask potential supervisors, the financial perils of being a PhD student, the future of higher education, the importance of skills, what keeps us going, and more.

Here are the specific questions that we answered in this episode (the background to these questions is shared in the episode):

Would you have any advice on how I can even decide whether to commence a PhD?
Are there any questions in particular that you think are important to ask prospective supervisors?
How do people make PhDs work financially? You are supposed to treat the degree like a regular 40 hour/ week job (and students commonly fail to do so). However, what full-time job pays ~$540 per week and expects this?! You are not supposed to work > 8 hours/ week outside of this?! I thought I could at least work 2-3 full days a week if I needed to.
Why do people generally leave academia, or not continue, after their PhD, despite obvious potential?
In what form do you think universities will be around in 5 and 10 years?
Are one-year progress reports from the PhD committee enough to stay on track?
What utility do PhDs hold inside and outside of academia? Apparently, skills matter more than a topic, and you have a better chance of getting a postdoc etc... if you have worked on a hot topic with a well-known supervisor.
How can one start the PhD prepared enough to finish it on time and earlier?
I am wondering what keeps academics going. I may be jaded, but lab environments don’t seem collaborative, and academics seem to be ruled by the admin people and hedge fund managers (or whoever). They also seem to make their money off students (i.e., the customer). I see a reverence for science and people trying to game the system, but not people wanting to seek truth in science. I now wonder how much of academia is motivated by pride, comfort, and not knowing what else to do. In my mind (and I am exaggerating a little), the PhD journey is coming to resemble an abusive relationship between the student and the uni, facilitated by the supervisor who hopefully gets something out of it. I assume it only gets worse from here.
I have been told that the PhD is the only program that offers solid research training and the ability to do your original research (something an industry job does not offer). Even if I accept those premises, I now wonder what it is all for.
Where do you both see yourselves in 5 and 10 years?
What keeps you both going?

We have new merch! Use the discount code 'METAL' to get 20% off (valid until January 31st, 2022).

Everything Hertz on social media

Support us on Patreon and get bonus stuff!

$1 per month: A 20% discount on Everything Hertz merchandise, a monthly newsletter, access to the occasional bonus episode, and the the warm feeling you're supporting the show
$5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Episode citation

Quintana, D.S., Heathers, J.A.J. (Hosts). (2021, December 27) "146: Skills pay bills", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/PUW6N

Support Everything Hertz

145: Our boat is sinking slightly slower

13 december 2021 | 48 min

144: The role of luck in academia

15 november 2021 | 54 min

143: A little less conversation, a little more action

1 november 2021 | 55 min

142: Red flags in academia [Live episode]

18 oktober 2021 | 57 min

141: Why we should diversify study samples (with Sakshi Ghai)

4 oktober 2021 | 57 min

140: You can’t buy cat biscuits with ‘thank you’ emails

20 september 2021 | 61 min

139: Open science from a funder's perspective (with Ashley Farley)

6 september 2021 | 56 min

138: Preprints in the time of coronavirus (with Michele Avissar-Whiting)

16 augusti 2021 | 67 min

137: Ten rules for improving academic work-life balance

2 augusti 2021 | 53 min

136: Who peer-reviews the peer-reviewed journals?

19 juli 2021 | 50 min

135: A loss of confidence

5 juli 2021 | 51 min

134: Paywalled questionnaires

21 juni 2021 | 57 min

133: Manuscript submission fees

7 juni 2021 | 49 min

132: Post-pandemic academia

17 maj 2021 | 51 min

131: Long live the overhead projector!

3 maj 2021 | 63 min

130: Normalizing retractions (with Dorothy Bishop)

19 april 2021 | 60 min

Dan and James chat with Dorothy Bishop (University of Oxford) about the importance of normalizing the retraction of scientific papers, publication ethics, and whether paper mills (companies that make fake papers at scale) are an issue in the psychological sciences

Here are some links and stuff we covered:

Dorothy's thoughts on how the adoption of open science practices has been progressing since we last had her on the show in June 2018
The European Research Council's new open access journal, which is free to publish in if you're ERC funded
Dan's proposal of something similar in a 2019 The Chronicle piece (free to read with email signup)
Dorothy's recent blogpost on publication ethics
The Society for Microbiology piece behind the blogpost
Our episode with Elisabeth Bik
Image manipulation in scientific papers
Why don't publishers have agreements with authors that if they discover something dodgy in papers the author can't sue them?
Retraction notices don't easily discriminate between fraud and mistakes, and this is one reason why that authors are so reluctant to have their work retracted
James' fictional paper mill story (based on real fraud), that he wrote with Otto Kalliokoski
Are paper mills a problem in psychology?
Dorothy's Oxford photo challenge

Other links

[Dan on twitter](www.twitter.com/dsquintana)
[James on twitter](www.twitter.com/jamesheathers)
[Everything Hertz on twitter](www.twitter.com/hertzpodcast)
[Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/)

Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/)

Support us on Patreon and get bonus stuff!

$1 a month: 20% discount on Everything Hertz merchandise, a monthly newsletter, access to the occasional bonus episode, and the the warm feeling you're supporting the show

- $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Episode citation

Quintana, D.S., Heathers, J.A.J. (Hosts). (2021, April 19) "130: Normalizing retractions (with Dorothy Bishop)", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/HRXU2

Special Guest: Dorothy Bishop.

Support Everything Hertz

129: Transparency audits

5 april 2021 | 57 min

128: How do you generate new research ideas?

15 mars 2021 | 71 min

Dan and James chat about how they come up with new ideas, why everyone seems to be trying to monetise their hobbies, and why it's so hard for most labs to have a singular focus of research.

We had some problems with James' mic so the quality of his audio wasn't up our usual standard. To make up for this we've added one of our older bonus episodes at the end of this conventional episode (this begins at 54:18). These bonus episodes are typically only made available for our Professor Fancypants Patreon patrons, but now you'll get to hear one!

- $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Episode citation

Quintana, D.S., Heathers, J.A.J. (Hosts). (2021, March 15) "128: How do you generate new research ideas?", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/U79NW

Support Everything Hertz

127: Speak up or shut up?

1 mars 2021 | 51 min

126: The division of scientific labor (with Saloni Dattani)

15 februari 2021 | 52 min

We have a wide-ranging chat with Saloni Dattani (Kings College London and University of Hong Kong) about the benefits of dividing scientific labor, the magazine she co-founded (Works in Progress) that shares novel ideas and stories of progress, and fighting online misinformation

Here are some links and other stuff we cover

Follow Saloni on Twitter: https://twitter.com/salonium
Why Saloni started the Works in Progress magazine
[Overleaf](overleaf.com), for writing papers in LaTeX
How science will benefit from the division of labour
Public writing vs. scientific writing
Why has behavioral science not been very useful in curbing the pandemic?
A paper suggested a link between digit ratio (2D:4D) and sex differences in COVID fatalities, and another paper debunking this claim
A paper suggesting baldness is a coronavirus risk factor, without controlling for age
Should peer-review be abolished altogether? Paper link
The Japanese mathematician who solved an "impossible" conjecture and posted the papers on his website
Reforms are more likely by work by chipping away at smaller problems, rather trying to fix everyting
Google dataset search https://datasetsearch.research.google.com/
The COVIDfaq.co website

Other links

[Dan on twitter](www.twitter.com/dsquintana)
[James on twitter](www.twitter.com/jamesheathers)
[Everything Hertz on twitter](www.twitter.com/hertzpodcast)
[Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/)

Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/)

Support us on Patreon and get bonus stuff!

$1 a month: 20% discount on Everything Hertz merchandise, a monthly newsletter, access to the occasional bonus episode, and the the warm feeling you're supporting the show

- $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Buy our Merch here: https://everything-hertz-podcast.creator-spring.com/

Episode citation

Quintana, D.S., Heathers, J.A.J. (Hosts). (2021, February 15) "126: The division of scientific labor (with Saloni Dattani)", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/VJA4S

Special Guest: Saloni Dattani.

Support Everything Hertz

125: Upon reasonable request

1 februari 2021 | 47 min

124: From Ptolemy to Takeshi's Castle

18 januari 2021 | 51 min

123: Authenticated anonymity (with Michael Eisen)

4 januari 2021 | 54 min

122: Reoptimizing scientific publishing for the internet age (with Michael Eisen)

21 december 2020 | 40 min

121: Transparent peer review

7 december 2020 | 58 min

120: How false beliefs spread in science (with Cailin O'Connor)

16 november 2020 | 47 min

119: Rules of thumb

2 november 2020 | 57 min

118: Evidence-free gatekeeping

19 oktober 2020 | 65 min

117: How we peer-review papers

5 oktober 2020 | 65 min

116: In my opinion

21 september 2020 | 77 min

115: A modest proposal

7 september 2020 | 60 min

We discuss James' recent proposal that scientists should be paid for performing peer review for journals published by for-profit companies—$450, to be precise. Dan also puts forward three meta-science projects that he thinks are worth funding.

More details

James' tweet proposing peer review should be compensated
Since recording this episode, James has set up the @450Movement twitter account
Also see James' blog post
The Collabra Psychology journal
Did the folks that co-authored the "redefine statistical sigificance" paper actually go on to follow their own recommendations?
Would high financial compensation of people on job search panels lead to better quality hires?
A tool that would automatically scrape the email addresses the of authors of papers you cite would make life easier for asking for feedback and providing review recommendations.
I'm curious as to whether people are reading the show notes. If you are reading this and want a Hertz mug, the first person to send @hertzpodcast a tweet saying they read the show notes will get a free mug
Kristoffer Magnusson's statstics art
Join the Git Gud Science Discord by following this link: https://discord.gg/s8MN3gA

Other links

[Dan on twitter](www.twitter.com/dsquintana)
[James on twitter](www.twitter.com/jamesheathers)
[Everything Hertz on twitter](www.twitter.com/hertzpodcast)
[Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/)

Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/)

Support us on Patreon and get bonus stuff!

$1 a month: 20% discount on Everything Hertz merchandise, a monthly newsletter, access to the occasional bonus episode, and the the warm feeling you're supporting the show
$5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

- Save 16% on either tier if you pay annually!

Episode citation
Quintana, D.S., Heathers, J.A.J. (Hosts). (2020, September 7) "115: A modest proposal", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/4ZQ2E

Support Everything Hertz

114: Diversity in science (with Jess Wade)

17 augusti 2020 | 54 min

113: Citation needed

3 augusti 2020 | 53 min

112: Leaving academia

27 juli 2020 | 51 min

111: The cumulative advantage of academic capital (with Chris Jackson)

6 juli 2020 | 60 min

110: Red flags for errors in papers

15 juni 2020 | 47 min

109: Open scientific publishing [Live episode]

1 juni 2020 | 52 min

108: Requiem for a Screen

18 maj 2020 | 48 min

107: Memes, TikTok, and science communication (with Chelsea Parlett-Pelleriti)

4 maj 2020 | 65 min

106: Science on the run

20 april 2020 | 50 min

105: Tell it like it is (with Marike Schiffer)

6 april 2020 | 58 min

104: Now we'll discover which meetings could've been emails

16 mars 2020 | 63 min

103: Swiping right

2 mars 2020 | 77 min

102: Master of none

17 februari 2020 | 64 min

101: Punishing research misconduct

3 februari 2020 | 59 min

100: Hundredth episode live special (with Daniel Lakens, Amy Orben, and Chris Chambers)

27 januari 2020 | 111 min

99: Science advocacy

6 januari 2020 | 50 min

98: Episode titles are redundant, at best (with Sophia Crüwell)

16 december 2019 | 59 min

97: Slow science

2 december 2019 | 61 min

96: The chaotic state of doctoral research

18 november 2019 | 48 min

95: All good presentations are alike; each bad presentation is bad in its own way

4 november 2019 | 64 min

94: Predicting the replicability of research

21 oktober 2019 | 58 min

93: Double-blind peer review vs. open science

7 oktober 2019 | 55 min

92: Chaos in the brickyard

16 september 2019 | 74 min

Dan and James discuss the role of Google Scholar in citation patterns and whether we should limit academics to only publishing two papers a year.

Links and details:

James has a new Hertz-quarters
The Metascience conference
How is google scholar influencing citation patterns
A slide from @Jevinwest's presentation on Google Scholars
Is this a symptom of lazy citaton practices rather than the algorithm?
What are the alternatives to google scholar?
Should google open up the algorithm?
GS will find your preprint and link it to the paywall link
Why is Google Scholar free?
What would make GS better?
Using the Zotero plugin for to collected citation info in bulk from search results in GS
Top recommended articles in GS are phenomenal
GS is not great for meta-analysis
GS reduces friction
Should we limit academics to only publishing two papers a year, as suggested by former guest, Dorothy Bishop?
Who would stand to lose from this?
Is this idea practical? What if only a few countries or institutions implemented this?
The Japanese math genius who posts papers on his own website
Chaos in the brickyard paper Chaos in the brickyard cartoon Open data isn’t a new concept

Other links

[Dan on twitter](www.twitter.com/dsquintana)
[James on twitter](www.twitter.com/jamesheathers)
[Everything Hertz on twitter](www.twitter.com/hertzpodcast)
[Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/)

Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/)

Support us on Patreon and get bonus stuff!

$1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show
$5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes)

Episode citation and permanent link
Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, September 16) "Chaos in the brickyard", Everything Hertz [Audio podcast] https://osf.io/xfd2p/

Support Everything Hertz

91: Shifting the goalposts in statistics (with Kristin Sainani)

2 september 2019 | 64 min

90: Mo data mo problems

19 augusti 2019 | 58 min

89: Conflicts of interest in psychology (with Tom Chivers)

5 augusti 2019 | 60 min

88: The pomodoro episode

15 juli 2019 | 60 min

87: Improving the scientific poster (with Mike Morrison)

1 juli 2019 | 51 min

86: Should I stay or should I go?

17 juni 2019 | 65 min

85: GWAS big teeth you have, grandmother (with Kevin Mitchell)

3 juni 2019 | 83 min

We chat with Kevin Mitchell (Trinity College Dublin) about what the field of psychology can learn from genetics research, how our research theories tend to be constrained by our research tools, and his new book, "Innate".

Other stuff we cover:

Kevin's book, "Innate: How the Wiring of Our Brains Shapes Who We Are"
The story with link between Serotonin transporter gene 5-HTTLPR variation and it’s link to depression
Gene wide association studies (GWAS) vs. candidate gene studies
Correcting for multiple tests in GWAS
What can psychology learn from genetics when it comes to improving methods?
When did people lose faith in candidate gene studies?
Common vs. rare gene mutations
How our theories are shaped by our technology and resources
The third overlooked area of genetic variation (other than nature and nurture): How the developmental program plays out
Why intelligence is just an index of how well your brain is put together
What happened to identifying new therapeutic targets via genetics?
What Kevin has changed his mind about
A few papers on mice behavior relevant to what Kevin mentioned, Matias et al (2017), Menegas et al (2018), Zalocusky et al (2016), Miyazaki et al (2018), Bercovici et al (2018), Orsini et al (2018)
Robustness and Evolvability in Living Systems (Wagner, A)
The Strategy of the Genes (Waddington, C)
Cybernetics : Or the Control and Communication in the Animal and the Machine (Wiener, N)

Other links

[Dan on twitter](www.twitter.com/dsquintana)
[James on twitter](www.twitter.com/jamesheathers)
[Everything Hertz on twitter](www.twitter.com/hertzpodcast)
[Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/)

Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/)

Support us on Patreon and get bonus stuff!

$1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show
$5 a month or more: All the stuff you get in the $1 tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes)

Episode citation and permanent link
Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, June 3) "GWAS big teeth you have, grandmother (with Kevin Mitchell)", Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/RS4HX

Special Guest: Kevin Mitchell.

Support Everything Hertz

84: A GPS in the Garden of Forking Paths (with Amy Orben)

21 maj 2019 | 52 min

We chat with Amy Orben, who applies "multiverse" methodology to combat and expose analytical flexibility in her research area of the impact of digital technologies on psychological wellbeing. We also discuss ReproducibiliTea, an early career researcher-led journal club initiative she co-founded, which helps young researchers create local open science groups.

Here are some more details and links:

The tweet pointing our Dan's gramatical error in his usual introduction. THANKS DENIS
Is Twitter melting our brains?
The history of "new technology" panic
What's the next panic?
Moral entrepreneurs: profiting from moral panic
Specification curve analysis: a way to run all theoretically defensible analysis options on a given dataset
Amy's Nature Human Behavior paper
Amy's PNAS paper
The longitudincal effect of social media use on life satisfaction
How should scientists speak out against dodgy science?
The story behind Reproducabilitea
The ReproducibiliTea podcast
ReproducibiliTea stickers!
The UK Reproducibility network
Daniel Lakens' Coursera course
A multiverse of multiverses
Press releasing every paper might not be the best idea
Amy's book recommendation: The long way to a small angry planet

Other links

Amy on Twitter
[Dan on twitter](www.twitter.com/dsquintana)
[James on twitter](www.twitter.com/jamesheathers)
[Everything Hertz on twitter](www.twitter.com/hertzpodcast)
[Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/)

Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/)

Support us on Patreon and get bonus stuff!

$1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show
$5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes)

Episode citation and permanent link
Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, May 21) "A GPS in the Garden of Forking Paths (with Amy Orben)", Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/38KPE

Special Guest: Amy Orben.

Support Everything Hertz

83: Back to our dirty unwashed roots

8 maj 2019 | 59 min

82: More janitors and fewer architects

15 april 2019 | 71 min

We answer a listener question on the possible negative consequences of the open science movement—are things moving too quickly?

Links and things we discuss in the episode:

We have a new logo, if you haven't already noticed...
Contact us via our website form!
Considering the potential downsides of open science
Here come dat boi meme explination
The dangers of open access by fiat
The role of commercial entities in open science
The “University of Oslo fancy Norway people-pay-taxes oil money bloody library”
Dropping the success rate of grants to increase the quality of evaluation
Reframing open science reform efforts to a mission of equity and fairness
We don’t know the process behind university sexual harassment/misconduct investigations
Does transparency even matter if people won’t follow up on problems?
James' prediction: If someone starts a journal that ONLY does Registered Report, this will be very successful
The milkshake duck tweet and an explanation
Paul Roos and his “no dickheads” policy
Linking DOIs
We can't let edge case scenarios, which may not even play out, hobble progress

Other links

[Dan on twitter](www.twitter.com/dsquintana)
[James on twitter](www.twitter.com/jamesheathers)
[Everything Hertz on twitter](www.twitter.com/hertzpodcast)
[Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/)

Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/)

Support us on Patreon and get bonus stuff!

$1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show
$5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes)

Episode citation and permanent link
Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, April 15) "More janitors and fewer architects" Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/7ZR9J

Support Everything Hertz

81: Too Young To Know, Too Old To Care

1 april 2019 | 56 min

80: Cites are not endorsements (with Sean Rife)

17 mars 2019 | 52 min

79: Clinical trial reporting (with Henry Drysdale)

3 mars 2019 | 56 min

78: Large-scale collaborative science (with Lisa DeBruine)

17 februari 2019 | 59 min

In this episde, we chat with Lisa DeBruine (University of Glasgow) about her experience with large-scale collaborative science and how her psychology department made the switch from SPSS to R.

Discussion points and links galore:

Deborah Apthorp's tweet on having to teach SPSS, "because that's what students know"
People who are involved with teaching R for psychology at the University of Glasgow: @Eavanmac @dalejbarr @McAleerP @clelandwoods @PatersonHelena @emilynordmann
Why the #psyTeachR started teaching R for reproducible science
Data wrangling vs. statistical analysis
The psyTeachR website
Danielle Navarro, and her R text book that you should read
Lisa's "faux" package for data simulation
Sometimes you can't share data, simulations are a good way around this problem
"synthpop" is the name of the package that Dan mentioned that can simulate census data
Power analysis can be hard once you go beyond the more conventional statistical tests (e.g., t-tests, ANOVAs etc...)
Lisa's OSF page
Dirty code is always better than no code (but the cleaner the better)
Live coding is terrifying but a useful teaching tool. Here's Dan live coding how to build a website in R, typos and all
Using a Slack group for help
The psychological science accelerator
Chris Chartier (Psych Science Accelerator Director) on Twitter
A few of the other (hundreds) of folks involved with the Psych Science Accelerator Director: @PsySciAcc: @CRChartier @Ben_C_J @JkayFlake @hmoshontz
Lisa's Registered Report project on face rating
The challenges associated with collaborating with 100+ labs
Authorship order
Author contributions: CRediT taxonomy
The DARPA-funding project on using AI to determine reproducibility
Interacting Minds workshop in Denmark in March on open science and reproducibility
Lisa shares what Glasgow is like
Lisa has changed her mind about the importance of research metrics (h-index, impact factors etc...)
Lisa thinks you should read this paper on equivalence testing, which includes two former guests, Daniel Lakens, Anne Scheel, and friend of the show Peder Isager.
Here's the latest episode from Psych Soc O'Clock

Other links

[Dan on twitter](www.twitter.com/dsquintana)
[James on twitter](www.twitter.com/jamesheathers)
[Everything Hertz on twitter](www.twitter.com/hertzpodcast)
[Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/)

Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/)

Support us on Patreon and get bonus stuff!

$1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show
$5 a month or more: All the stuff you get in the first tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes)

Episode citation and permanent link
Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, February 18) "Large-scale collaborative science (with Lisa DeBruine)", Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/JDT6F

Special Guest: Lisa DeBruine.

Support Everything Hertz

77: Promiscuous expertise

4 februari 2019 | 55 min

Dan and James discuss how to deal with the problem of scientists who start talking about topics outside their area of expertise. They also discuss what they would do differently if they would do their PhDs again

Here's what they cover...

The podcast will now be permanently archived on Open Science Framework
James did a talk at the Sound Education conference on podcasting for early career researchers. Here's the video if you want to see him squirm uncomfortably in his chair for 20 minutes and/or hear his thoughts our approach to podcasting
The temptation for academics to believe their own press and to have their thoughts reinforced by the praise they get
Keeping a handle on what you know and don't know
Nassim Nicholas Taleb has FANS
The "Pete Evans" effect, James' solution, that we should eat Pete Evans, pesca-pescaterianism, and the spectacularly bad advice that we should stare into the sun
You should follow gynecologist Jennifer Gunter on Twitter
How much money would you pay for 100,000 engaged twitter followers? Here's the tweet James was referring to
Should researchers have something like a Hippocratic Oath? How would we police this?
Researchers are not good at admitting they're wrong, do we need to approach retractions differently?
Would a bounty system, in which journals offer rewards, for finding errors in their papers, work well?
The "Loss of confidence" project, and Rebecca Willen's CV
The "Nobel disease"

Other links

[Dan on twitter](www.twitter.com/dsquintana)
[James on twitter](www.twitter.com/jamesheathers)
[Everything Hertz on twitter](www.twitter.com/hertzpodcast)
[Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/)

Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/)

Support us on Patreon and get bonus stuff!

$1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show
$5 a month or more: All the stuff you get in the $1 tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes)

Episode citation and permanent link
Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, February 4) "Promiscuous expertise", Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/VYCAH

Support Everything Hertz

76: Open peer review

21 januari 2019 | 48 min

75: Overlay journals (with Daniele Marinazzo)

7 januari 2019 | 58 min

We’re joined by Daniele Marinazzo (University of Ghent) to chat about the recently launched overlay journal Neurons, Behavior, Data analysis and Theory (NBDT), for which he on the Editorial Board.

An overlay journal is organised a set of manuscripts that is published and hosted by a seperate entity (in this case, the Arxiv server), a feature that dramatically reduces publication costs. We discuss the unique overlay model, how this can drive article fees to essentially zero, and what it takes to build a good community journal.

Here’s what we cover:

Why launch a new neuroscience journal and how is it different from currently established journals?
The unique way that editor’s decide which papers to send out for review
How does the journal operate when it’s open access and submissions only cost $10?
How do you build a journal that your target community will recognise as a ‘good’ journal?
The process of submitting a manuscript to NBDT
Should journals allow or encourage authors to suggest potential reviewers for their papers?
Using Twitter to find reviewers based on who’s ‘liked’ the preprint
Is posting a preprint on twitter actually useful?
What can neuroscience learn from the field of physics?
Authorship attribution
How can a journal better champion early career researchers?

Links...

NBDT journal: https://nbdt.scholasticahq.com
Danielle on twitter: https://twitter.com/dan_marinazzo
Dan on twitter https://www.twitter.com/dsquintana
James on twitter https://www.twitter.com/jamesheathers
Everything Hertz on twitter https://www.twitter.com/hertzpodcast
Everything Hertz on Facebook https://www.facebook.com/everythinghertzpodcast/

Music credits: Lee Rosevere freemusicarchive.org/music/Lee_Rosevere/

Support us on Patreon and get bonus stuff!

$1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show
$5 a month or more: All the stuff you get in the $1 tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes)

Special Guest: Daniele Marinazzo.

Support Everything Hertz

74: Seeing double (with Elisabeth Bik)

19 december 2018 | 52 min

73: Update your damn syllabus

3 december 2018 | 61 min

72: Anonymity in scientific publishing

16 november 2018 | 58 min

71: Moving for your job

5 november 2018 | 54 min

70: Doubling-blinding dog balls

15 oktober 2018 | 67 min

69: Open science tools (with Brian Nosek)

9 oktober 2018 | 49 min

68: Friends don’t let friends believe in impact factors (with Nathan Hall)

3 september 2018 | 74 min

67: Shit Academics Say (with Nathan Hall)

20 augusti 2018 | 63 min

66: Ideal worlds vs grim truths

6 augusti 2018 | 54 min

65: Blockchain and open science (with Jon Brock)

16 juli 2018 | 55 min

64: Salami slicing

2 juli 2018 | 62 min

63: Science journalism (with Brian Resnick)

18 juni 2018 | 61 min

62: Adopting open science practices (with Dorothy Bishop)

4 juni 2018 | 57 min

61: Performance enhancing thugs (with Greg Nuckols)

21 maj 2018 | 56 min

60: This is more of a comment than a question

8 maj 2018 | 68 min

59: Rethinking the scientific journal (with Rickard Carlsson)

16 april 2018 | 63 min

58: Lessons from podcasting (with Simine Vazire)

2 april 2018 | 61 min

57: Radical Transparency (with Rebecca Willén)

15 mars 2018 | 49 min

56: Registered reports (with Chris Chambers)

2 februari 2018 | 54 min

55: The proposal to redefine clinical trials

18 januari 2018 | 59 min

In this episode, Dan and James discuss the US National Institutes of Health's new definition of a “clinical trial”, which comes into effect on the 25th of January.

Here’s the new definition: “A research study in which one or more human subjects are prospectively assigned to one or more interventions (which may include placebo or other control) to evaluate the effects of those interventions on health-related biomedical or behavioural outcomes”.

Over the course of this episode, they cover the pros and cons of this decision along with the implications for researchers and science in general.

Here are a few things they cover:

The traditional definition of a clinical trial
We go through James’ old work to determine if he’s been a clinical trialist all along
The lack of clarity surrounding the new definition
Why are adopting a clinical trial approach when this approach has obvious weaknesses?
What do you actually have to do when running a clinical trial?
Will institutions also adopt this new definition, thus putting basic research through clinical trial IRBs?
What if this extra red tape actually improves science?
One argument against the proposal is that registering more studies on clinicaltrials.gov will confuse the public. We don’t buy that.
Clinical trial registrations generally miss the many nuances of study design
The new clinical trial definition will eliminate some of the ‘forking paths’ when analysing and reporting data
How this new definition will affect grant applications for early career researchers?
What happens to exploratory research?
NIH case studies of what may constitute a clinical trial

Links

NIH clinical trial definition https://grants.nih.gov/policy/clinical-trials/definition.htm
The NIH “clinical trial decision tree” https://grants.nih.gov/policy/clinical-trials/CT-decision-tree.pdf
NIH case studies of what may constitute a clinical trial https://grants.nih.gov/policy/clinical-trials/case-studies.htm#case1

Support Everything Hertz

54: Cuckoo Science

15 december 2017 | 55 min

53: Skin in the game

17 november 2017 | 67 min

52: Give p's a chance (with Daniel Lakens)

20 oktober 2017 | 63 min

51: Preprints (with Jessica Polka)

6 oktober 2017 | 56 min

50: Special 50th episode (LIVE)

14 september 2017 | 100 min

49: War and p's

31 juli 2017 | 56 min

48: Breaking up with the impact factor (with Jason Hoyt)

21 juli 2017 | 54 min

47: Truth bombs from a methodological freedom fighter (with Anne Scheel)

7 juli 2017 | 69 min

46: Statistical literacy (with Andy Field)

23 juni 2017 | 80 min

In this episode, Dan and James are joined by Andy Field (University of Sussex), author of the “Discovering Statistics” textbook series, to chat about statistical literacy.

Highlights:

The story behind Andy’s new book
SPSS and Bayesian statistics
Andy explains why he thinks the biggest problem in science is statistical illiteracy
Researcher degrees of freedom and p-hacking
The story behind the the first version of ‘Discovering statistics’
How to improve your statistical literacy
Does peer review improve the statistics of papers
Researchers will draw different conclusions on the same dataset
The American Statistical Association’s statement on p-values
How has the teaching of statistics for psychology degrees changed over the years
Andy fact checks his own Wikipedia page
Andy’s thoughts on Bayesian statistics and how he applied it in a recent paper
The peer review of new statistical methods
Andy’s future textbook plans
The rudeness of mailing lists/discussion forums
What is something academia or stats-related that Andy believes that others think is crazy?
The one book that Andy recommends that everyone should read
We learn the crossover in James and Andy’s taste in metal bands

Links

Andy’s books: https://uk.sagepub.com/en-gb/eur/author/andy-field-0
The ‘PENIS of statistics’ lecture from Andy: https://www.youtube.com/watch?v=oe3_DeLC2JE
Daniel Lakens’ Coursera course: https://www.coursera.org/learn/statistical-inferences
The American Statistical Association’s statement on p-values: http://amstat.tandfonline.com/doi/abs/10.1080/00031305.2016.1154108
The refereeing decision paper: https://osf.io/gvm2z/
R stan: https://cran.r-project.org/web/packages/rstan/index.html
Statistical rethinking book: https://www.crcpress.com/Statistical-Rethinking-A-Bayesian-Course-with-Examples-in-R-and-Stan/McElreath/p/book/9781482253443

Music credits: Lee Rosevere freemusicarchive.org/music/Lee_Rosevere/

Special Guest: Andy Field.

Support Everything Hertz

45: Conferences and conspiracy theories

2 juni 2017 | 62 min

44: Who’s afraid of the New Bad People? (with Nick Brown)

19 maj 2017 | 69 min

43: Death, taxes, and publication bias in meta-analysis (with Daniel Lakens)

5 maj 2017 | 63 min

42: Some of my best friends are Bayesians (with Daniel Lakens)

21 april 2017 | 67 min

41: Objecting to published research (with William Gunn)

7 april 2017 | 67 min

40: Meta-research (with Michèle Nuijten)

24 mars 2017 | 49 min

39: Academic hipsters

10 mars 2017 | 55 min

38: Work/life balance - Part 2

24 februari 2017 | 62 min

37: Work/life balance in academia

17 februari 2017 | 57 min

36: Statistical inconsistencies in published research

27 januari 2017 | 51 min

35: A manifesto for reproducible science

20 januari 2017 | 51 min

34: E-health (with Robin Kok)

22 december 2016 | 60 min

33: Zombie theories

16 december 2016 | 44 min

32: Can worrying about getting sick make you sicker?

1 december 2016 | 43 min

31: Discover your psychiatric risk with this one weird trick

16 november 2016 | 55 min

30: Authorship

2 november 2016 | 49 min

29: Learning new skills

16 oktober 2016 | 49 min

28: Positive developments in biomedical science

30 september 2016 | 49 min

27: Complaints and grievances

23 september 2016 | 53 min

26: Interpreting effect sizes

9 september 2016 | 46 min

25: Misunderstanding p-values

27 augusti 2016 | 55 min

24: Incentive structures in science

17 augusti 2016 | 60 min

23: Serious academics

11 augusti 2016 | 53 min

22: Pokemon and public health

3 augusti 2016 | 59 min

21: This is your brain on steroids

22 juli 2016 | 58 min

20: Sample sizes in psychology studies

13 juli 2016 | 62 min

19: Let us spray: oxytocin and spirituality

6 juli 2016 | 47 min

18: Data sharing

29 juni 2016 | 51 min

17: Journals: Do we need them?

22 juni 2016 | 51 min

16: What makes a good psych study?

15 juni 2016 | 49 min

15: Software and coding

8 juni 2016 | 45 min

14: Science communication

2 juni 2016 | 33 min

13: Academic horror stories

26 maj 2016 | 52 min

12: Reporting heart rate variability studies

21 maj 2016 | 60 min

11: The placebo effect

10 maj 2016 | 44 min

10: Failure

4 maj 2016 | 50 min

9: What happens if your research is wrong?

28 april 2016 | 52 min

8: The PhD to Postdoc transition

20 april 2016 | 51 min

Episode 7: 7: The writing process

15 april 2016 | 50 min

6: The research pipeline - getting from idea to publication

7 april 2016 | 54 min

5: Do you even replicate?

30 mars 2016 | 45 min

4: Meta-analysis or mega-silliness?

22 mars 2016 | 40 min

3: Scientific publishing

16 mars 2016 | 49 min

2: Nutrition and Psychiatry

9 mars 2016 | 47 min

1: So you want to measure heart rate variability...

2 mars 2016 | 44 min

Everything Hertz

Methodology, scientific life, and bad language.

Om podden

Avsnitt

190: What happens when you pay reviewers?

189: Crit me baby, one more time

188: Double-blind peer review vs. scientific integrity

187: What started the replication crisis era?

186: Evaluating journal quality

185: The Retraction

184: A race to the bottom

183: Too beautiful to be true

182: What practices should the behavioural sciences borrow (and ignore) from other research fields?

181: Down the rabbit hole

180: Consortium peer reviews

179: Discovery vs. maintenance

178: Alerting researchers about retractions

177: Plagiarism

176: Tracking academic workloads

175: Defending against the scientific dark arts

174: Smug missionaries with test tubes

173: How do science journalists evaluate psychology papers?

172: In defence of the discussion section

171: The easiest person to fool is yourself (with Daniel Simons and Christopher Chabris)

170: Holy sheet

169: Using big data to understand behavior (Live episode with Sandra Matz)

168: Meta-meta-science

167: Diluted effect sizes

166: Is science becoming less disruptive over time?

165: Self-promotion

164: The great migration

163: eLife's new peer review model

162: Status bias in peer review

161: The memo (with Brian Nosek)

160: Whistleblowing

159: Peer review isn't working (with Saloni Dattani)

158: Word limits

157: Limitations

156: Looking for seeders

155: Don't you know who I am?

154: When the evidence is constructed around the narrative

153: Shame shame shame

152: Sorry Not Sorry

151: The dirty dozen

150: Why can't you do nothing?

149: Medical misinformation (with Rohin Francis)

148: Academic reference letters

147: The $7000 golden ticket

146: Skills pay bills

145: Our boat is sinking slightly slower

144: The role of luck in academia

143: A little less conversation, a little more action

142: Red flags in academia [Live episode]

141: Why we should diversify study samples (with Sakshi Ghai)

140: You can’t buy cat biscuits with ‘thank you’ emails

139: Open science from a funder's perspective (with Ashley Farley)

138: Preprints in the time of coronavirus (with Michele Avissar-Whiting)

137: Ten rules for improving academic work-life balance

136: Who peer-reviews the peer-reviewed journals?

135: A loss of confidence

134: Paywalled questionnaires

133: Manuscript submission fees

132: Post-pandemic academia

- $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

131: Long live the overhead projector!

- $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

130: Normalizing retractions (with Dorothy Bishop)

- $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

129: Transparency audits

- $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

128: How do you generate new research ideas?

- $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

127: Speak up or shut up?

- $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

126: The division of scientific labor (with Saloni Dattani)

- $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

125: Upon reasonable request

- $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

124: From Ptolemy to Takeshi's Castle

- $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month