## Justify Everything ### (Or: Why All Norms You Rely on When Doing Research Are Wrong) Science is difficult. To do good science researchers need to know about philosophy of science, learn how to develop theories, become experts in experimental design, study measurement theory, understand the statistics they require to analyze their data, and clearly communicate their results. My personal goal is to become good enough in all these areas such that I will be able to complete a single flawless experiment, just before I retire – but I expect to fail. In the meantime, I often need to rely on social norms when I make choices as I perform research. From the way I phrase my research question, to how I determine the sample size for a study, or my decision for a one or two-sided test, my justifications are typically ‘this is how we do it’. If you ask me ‘why’ I often don’t know the answer. In this talk I will explain that, regrettably, almost all the norms we rely on are wrong. I will provide some suggestions for attempts to justify aspects of the research cycle that I am somewhat knowledgeable in, mainly in the area of statistics and experimental design. I will discuss the (im)possibility of individually accumulating sufficient knowledge to actually be able to justify all important decisions in the research you do, and make some tentative predictions that in half a century most scientific disciplines will have become massively more collaborative, with a stronger task division between scholars working on joint projects.