Talk overview
Reproducibility
- Basic principle of scientific method
- Same data, same code, different analyst, same results
- Tools: R Markdown, documentation
Code sharing
- Tied to reproducibility
- For learning and critical review
- Tools: GitHub and Zenodo
Reproducibility: Importance and practice
Patil, 2019; Plessar, 2018; American Statistical Association statement
Reproducibility in biomedical science
Already know replication is a major problem
- e.g. Many Labs Project, OSC Project, Reproducibility Project
Don't know extent of reproducibility
- Few studies share data [1]
- Almost no study provides code [2]
Reproducibility in biomedical science
Already know replication is a major problem
- e.g. Many Labs Project, OSC Project, Reproducibility Project
Don't know extent of reproducibility
- Few studies share data [1]
- Almost no study provides code [2]
Why is it important? 🧐
- Simplest: It's a key pillar of scientific method.
- With modern technology, easy to implement (relative to past)
Why is it important? 🧐
- Simplest: It's a key pillar of scientific method.
- With modern technology, easy to implement (relative to past)
- In biomedical research, poorly implemented or not done at all
- Selected articles: Goldacre, 2019; Munafó, 2017; TOP guidelines; Transparency Checklist; Peng, 2011
Keep in mind: Reproducibility is a spectrum
- Should say "Full reproducibility".
Practical ways of doing reproducibility
- Generic ways:
- Documenting scripts and their order, keep them together (in same folder)
- Reproducible document systems
- Pipeline management
- Can't share data? Make fake dataset of the original one.
Practical ways of doing reproducibility
- Generic ways:
- Documenting scripts and their order, keep them together (in same folder)
- Reproducible document systems
- Pipeline management
- Can't share data? Make fake dataset of the original one.
- specific (easy to hard):
- Documenting R scripts, ordering them, and using R Projects
- R Markdown documents
- Pipeline tools (📦: drake, targets)
Code sharing: Importance and practice
Why is it important? 🧐
- Tightly tied to reproducibility
- Need code to reproduce results
Why is it important? 🧐
- Tightly tied to reproducibility
- Need code to reproduce results
- Critically review a study's exact analysis
Why is it important? 🧐
- Tightly tied to reproducibility
- Need code to reproduce results
Critically review a study's exact analysis
Builds common standards and best practices
Why is it important? 🧐
- Tightly tied to reproducibility
- Need code to reproduce results
Critically review a study's exact analysis
Builds common standards and best practices
Read others code to learn how to write better
- Code is written to be read by yourself and others [1]
[1]: Otherwise we'd all write in Assembly.