Contents **I The crisis of confidence in social and life sciences: State of affairs (4 September 2019)** Learning objectives: Become acquainted with the recent developments regarding the so-called “replication crisis”. - Replication crisis: how it all started (this time around). - Medicine, you were supposed to be the best of us! - Consequences of problematic practices. - You’re not alone in misinterpreting p-values. **II From questionable research practices and biased stories, to better evidence and/or decisions (11 September 2019)** Learning objectives: Understand what the research community is doing to improve the quality of published research. Extrapolate to non-academic settings. - Transparency and Openness Promotion (TOP) guidelines to fight bad science. - Transforming publication practices with pre-prints - Disentangling confirmatory and exploratory research. - Tricky rule-of-thumb questions to ask when being presented research (1/2: “null findings”). III **Magnificient mistakes and where to find them (18 September 2019)** Learning objectives: Recognise some particular pitfalls in evidential statements. Understand that decisions in the field do not need to rely on correct predictive statements, let alone scientific evidence. - Tricky rule-of-thumb questions to ask when being presented research (2/2: “statistically significant” findings). - Ways tests can fail: Type I/II mistakes. Type M and Type S mistakes. - The difference between evidence of absence and absence of evidence: Black Swans and the Turkey Problem. - When you don’t need to be right: green lumber, and a first taste of convexity. - Heuristics: Simple rules that make us smart. **IV On interpreting data nudes instead of summary tables (25 September 2019)** Learning objectives: Understand the rationale for visualising data, and what can be hidden when reporting summary statistics only. Learn to spot some common tricks used to visualise data in a favourable way to the presenter. - A crude redux to evidence of absence. - Data Nudes vs. Shitty Tables. - The End of Average. - What gets lost in looking at numbers alone: Uncertainty hidden in the absence of distributions. - Demons with(in) axes: Slaying or summoning effects with presentation tricks. - Dose-response effects masked by averages. **V Complex systems and why they ruin everything straightforward (2 October 2019)** Learning objectives: Become familiar with general features of so-called complex systems. Understand how they can be thought of in the context of practical interventions. - Intro to complexity, and general features of complex systems. - Interaction vs. component dominant systems. - Don’t camp at 1st order effects in dragon season. - Navigating the Four Quadrants **VI Never cross Heraclitus’ river, if it’s on average 1 meter deep: Interventions and their offspring (9 October 2019)** Learning objectives: Understand the rationale behind interventions and experimenting/intervening in complex systems, as well as some limitations of big data. - Change comes in a triad. Sales tricks to counter, use and abuse. - Pathway thinking & complexity thinking in behaviour change science. - Failures and unexpected effects of social interventions. - When is it safe(r) to intervene? **VII Dynamic/idiographic phenomena, and hidden assumptions (16 October 2019)** Learning objectives: Describe the concepts of ergodicity and stationarity. Understand how they can mislead when not taken into account when e.g. assessing risks. - Assumptions, schmassumptions; mind your foundations! - Damned world not sitting still: Ergodicity & stationarity - The idiographic approach to science - The best map fallacy - The precautionary principle for policy and interventions - Frequency vs. consequences of being wrong: What matters more? - Recap on the course: The Fourth Quadrant will find you, so better put your house in order