In the social sciences it is increasingly common to preregister hypotheses, research designs and analysis plans before a study is conducted (Ferguson et al., 2023). Because the time-stamp on the preregistration documents when the ideas for the study were deposited, a preregistration is an intellectual honesty device. Preregistration of study hypotheses prevents alteration of hypotheses to match empirical results – also known as hypothesizing after results are known (HARKing, Kerr, 1998) because such alterations will be transparent from a comparison of the preregistration and the eventual write-up in manuscripts. Preregistration of an analysis plan reduces researcher degrees of freedom (Simons, Nelson & Simonsohn, 2011) in the garden of forking paths (Gelman & Loken, 2014) because p-hacking and other questionable research practices will be evident from a comparison of the study plans and their execution. Without a preregistration, confirmations of hypotheses and stark findings are less convincing. Results from preregistered experiments are less often confirming the study hypotheses than experiments that were not preregistered (Scheel, Schijen & Lakens, 2021).
Preregistration is less common in research that does not rely on data from experiments (figure 7, Ferguson et al., 2023), and hence it is less likely in sociology than in economics, political science and psychology (in that order). The benefits of preregistration are similar though. If you haven’t created a preregistration before, it will be helpful to see some guidance and examples. Also it is good to know that a preregistration is not a prison cell – you can update it and diverge from it in practice, as long as you motivate and document the discrepancies.
General information about preregistrations is here: https://www.cos.io/initiatives/prereg. The Open Science Framework guidance to create preregistrations is here: https://help.osf.io/article/158-create-a-preregistration
For an online field experiment testing effects of social information on charitable giving to culture through a crowdfunding platform (Van Teunenbroek & Bekkers, 2020) we used AsPredicted. The design of the experiment and the key hypotheses were preregistered in 2016 at https://aspredicted.org/u5w9u.pdf. The publication appeared in 2020. All materials are at the OSF: https://osf.io/epuj6/. A key learning from this project was that it is important to think about procedures to handle outliers, which in the analysis of the data turned out to be influential.
For an analysis of data from an experiment embedded in a survey conducted among a population sample and a sample of high net worth individuals (Bekkers, Whillans, Smeets & Norton, 2019) we created a pre-registration in 2018, posted here: https://osf.io/x69ds. A paper has not yet been published. Materials are here: https://osf.io/bvs6t/
For an analysis of data on prosocial values and behavior a pre-analysis plan is here: https://osf.io/3pnj5. The current version of the paper is here: https://osf.io/d9wte.
For a study analyzing multiple longitudinal panel survey datasets including information on health and volunteer work (De Wit, Qu & Bekkers, 2022) we created an analysis plan before the analyses were conducted. The pre-analysis plan is publicly available at the Open Science Framework page. The code for the analyses, the paper and supplementary materials are also available on the page.
References
Bekkers, R. (2023). Prosocial Values for the Philanthropy Commons. WZB conference Pro-sociality: Corporate Philanthropy and Individual Donations, Berlin, June 20, 2023. https://osf.io/mwkca
Bekkers, R., Whillans, A., Smeets, P.M., & Norton, M. (2019). The Joy of Giving: Evidence from a Matching Experiment with Millionaires and the General Population. Paper presented at the ASSA meeting, January 4, 2019. https://osf.io/r6p32
De Wit, A., Qu, H. & Bekkers, R. (2022). The health advantage of volunteering is larger for older and less healthy volunteers in Europe: a mega-analysis. European Journal of Aging. https://doi.org/10.1007/s10433-022-00691-5
Ferguson, J., Littman, R., Christensen, G., Levy Paluck, E., Swanson, N., Wang, Z., Miguel, E., Birke, D. & Pezzuto, J-H. (2023). Survey of open science practices and attitudes in the social sciences. Nature Communications, 14, 5401. https://doi.org/10.1038/s41467-023-41111-1
Gelman, A., & Loken, E. (2014). The statistical crisis in science data-dependent analysis—a “garden of forking paths”—explains why many statistically significant comparisons don’t hold up. American Scientist, 102(6), 460-465. https://www.jstor.org/stable/43707868
Kerr, N. L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2(3), 196-217. https://doi.org/10.1207/s15327957pspr0203_4
Scheel, A. M., Schijen, M. R., & Lakens, D. (2021). An excess of positive results: Comparing the standard Psychology literature with Registered Reports. Advances in Methods and Practices in Psychological Science, 4(2). https://doi.org/10.1177/25152459211007467
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 1359-1366. https://doi.org/10.1177/0956797611417632
Van Teunenbroek, P.S.C. & Bekkers, R. (2020). Follow the Crowd: Social Information and Crowdfunding Donations in a Large Field Experiment. Journal of Behavioral Public Administration, 3(1): 1-17. https://doi.org/10.30636/jbpa.31.87
Rene Bekkers 