Demonstrations of p-hacking and related readings
(Also known as "fishing", "researcher degrees of freedom", and "the garden of forking paths")
- Music about old age makes you younger: Simmons, J., Nelson, L., & Simonsohn, U. (2011). False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359-1366.
- Countries with shorter names have higher GDPs: Video demo by Neuroskeptic (example begins around 14:48)
- People whose subject ID is odd believe odd things: Blog post by Simonsohn
- Countries with more languages have more traffic accidents: Roberts, S., & Winters, J. (2013). Linguistic diversity and traffic accidents: lessons from statistical studies of cultural traits. PLoS ONE, 8, e70902.
- Brain activation in the nose: Baker, C., Hutchison, T., & Kanwisher, N. (2007). Does the fusiform face area contain subregions highly selective for nonfaces? Nature Neuroscience, 10, 3-4.
- Brain activation in a dead fish: Bennett, C., Baird, A., Miller, M., & Wolford, G. (2009). Neural correlates of interspecies perspective taking in the post-mortem atlantic salmon: an argument for proper multiple comparisons correction. 15th Annual Meeting of the Organization for Human Brain Mapping, San Francisco.
- General review: Gelman, A., & Loken, E. (2013 manuscript). The garden of forking paths: why multiple comparisons can be a problem, even when there is no "fishing expedition" or "p-hacking" and the research hypothesis was posited ahead of time.
- The 9 Circles of Scientific Hell by Neuroskeptic
Readings about pre-registration
- Joe Simmons, Leif Nelson, and Uri Simonsohn. (2017). How to properly preregister a study. Data Colada.
- Wagenmakers, E., Wetzels, R., Borsboom, D., van der Maas, H., & Kievit, R. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7, 632-638.
- Nosek, B., Ebersole, C., DeHaven, A., & Mellor, D. (2017 preprint). The preregistration revolution. doi: 10.17605/OSF.IO/2DXU5
- 2016 blog post by Bergmann
- Sakaluk, J. (2016). Exploring small, confirming big: an alternative system to The New Statistics for advancing cumulative and replicable psychological research. Journal of Experimantal Social Psychology, 66, 47-54.
Places to pre-register
If you care about pre-registration there's also a good chance that you care about open science, so also look at the Peer Reviewers' Openness Initative (paper).
by Stephen Politzer-Ahles. Last modified on 2017-08-24.