Contents
**I
The crisis of confidence in social and life sciences: State of affairs (4 September 2019)**
Learning objectives: Become acquainted with the recent developments regarding the so-called “replication crisis”.
- Replication crisis: how it all started (this time around).
- Medicine, you were supposed to be the best of us!
- Consequences of problematic practices.
- You’re not alone in misinterpreting p-values.
**II
From questionable research practices and biased stories, to better evidence and/or decisions (11 September 2019)**
Learning objectives: Understand what the research community is doing to improve the quality of published research. Extrapolate to non-academic settings.
- Transparency and Openness Promotion (TOP) guidelines to fight bad science.
- Transforming publication practices with pre-prints
- Disentangling confirmatory and exploratory research.
- Tricky rule-of-thumb questions to ask when being presented research (1/2: “null findings”).
III
**Magnificient mistakes and where to find them (18 September 2019)**
Learning objectives: Recognise some particular pitfalls in evidential statements. Understand that decisions in the field do not need to rely on correct predictive statements, let alone scientific evidence.
- Tricky rule-of-thumb questions to ask when being presented research (2/2: “statistically significant” findings).
- Ways tests can fail: Type I/II mistakes. Type M and Type S mistakes.
- The difference between evidence of absence and absence of evidence: Black Swans and the Turkey Problem.
- When you don’t need to be right: green lumber, and a first taste of convexity.
- Heuristics: Simple rules that make us smart.
**IV
On interpreting data nudes instead of summary tables (25 September 2019)**
Learning objectives: Understand the rationale for visualising data, and what can be hidden when reporting summary statistics only. Learn to spot some common tricks used to visualise data in a favourable way to the presenter.
- A crude redux to evidence of absence.
- Data Nudes vs. Shitty Tables.
- The End of Average.
- What gets lost in looking at numbers alone: Uncertainty hidden in the absence of distributions.
- Demons with(in) axes: Slaying or summoning effects with presentation tricks.
- Dose-response effects masked by averages.
**V
Complex systems and why they ruin everything straightforward (2 October 2019)**
Learning objectives: Become familiar with general features of so-called complex systems. Understand how they can be thought of in the context of practical interventions.
- Intro to complexity, and general features of complex systems.
- Interaction vs. component dominant systems.
- Don’t camp at 1st order
effects in dragon season.
- Navigating the Four Quadrants
**VI
Never cross Heraclitus’ river, if it’s on average 1 meter deep: Interventions and their offspring (9 October 2019)**
Learning objectives: Understand the rationale behind interventions and experimenting/intervening in complex systems, as well as some limitations of big data.
- Change comes in a triad. Sales tricks to counter, use and abuse.
- Pathway thinking & complexity thinking in behaviour change science.
- Failures and unexpected effects of social interventions.
- When is it
safe(r) to intervene?
**VII
Dynamic/idiographic phenomena, and hidden assumptions (16 October 2019)**
Learning objectives: Describe the concepts of ergodicity and stationarity. Understand how they can mislead when not taken into account when e.g. assessing risks.
- Assumptions, schmassumptions; mind your foundations!
- Damned world not
sitting still: Ergodicity & stationarity
- The idiographic approach to
science
- The best map fallacy
- The precautionary principle for policy
and interventions
- Frequency vs. consequences of being wrong: What
matters more?
- Recap on the course: The Fourth Quadrant will find you,
so better put your house in order