Approaches to open, scalable, and reproducible data management and analysis: Training and software

.title[
# Approaches to open, scalable, and reproducible data management and analysis: Training and software
]

---

.footer-right[
Slides: [slides.lwjohnst.com/misc/2022-08-17](https://slides.lwjohnst.com/misc/2022-08-17/)
Licensed under CC-BY <svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#990033;overflow:visible;position:relative;"><path d="M314.9 194.4v101.4h-28.3v120.5h-77.1V295.9h-28.3V194.4c0-4.4 1.6-8.2 4.6-11.3 3.1-3.1 6.9-4.7 11.3-4.7H299c4.1 0 7.8 1.6 11.1 4.7 3.1 3.2 4.8 6.9 4.8 11.3zm-101.5-63.7c0-23.3 11.5-35 34.5-35s34.5 11.7 34.5 35c0 23-11.5 34.5-34.5 34.5s-34.5-11.5-34.5-34.5zM247.6 8C389.4 8 496 118.1 496 256c0 147.1-118.5 248-248.4 248C113.6 504 0 394.5 0 256 0 123.1 104.7 8 247.6 8zm.8 44.7C130.2 52.7 44.7 150.6 44.7 256c0 109.8 91.2 202.8 203.7 202.8 103.2 0 202.8-81.1 202.8-202.8.1-113.8-90.2-203.3-202.8-203.3z"/></svg>
]

---

<div>
<style type="text/css">.xaringan-extra-logo {
width: 60px;
height: 128px;
z-index: 0;
background-image: url(../../common/sdca-logo.png);
background-size: contain;
background-repeat: no-repeat;
position: absolute;
top:1em;right:1em;
}
</style>
<script>(function () {
  let tries = 0
  function addLogo () {
    if (typeof slideshow === 'undefined') {
      tries += 1
      if (tries < 10) {
        setTimeout(addLogo, 100)
      }
    } else {
      document.querySelectorAll('.remark-slide-content:not(.title-slide):not(.inverse):not(.hide_logo)')
        .forEach(function (slide) {
          const logo = document.createElement('a')
          logo.classList = 'xaringan-extra-logo'
          logo.href = 'https://www.stenoaarhus.dk/'
          slide.appendChild(logo)
        })
    }
  }
  document.addEventListener('DOMContentLoaded', addLogo)
})()</script>
</div>

<!-- 
Details:

- Stimulate scientific discussion
- ~30 min + discussion (~45 min total?)

Outline (do in order but don't need to complete):

- DIF Project
- r-cubed

-->

# Outline: My current projects

1. NNF-funded Data Infrastructure Framework (DIF) Project

2. Reproducible Research in R hands-on courses with Danish Diabetes Academy

???

We'll go through this outline in this order, not expecting to cover them all.

---

# Data Infrastructure Framework (DIF) Project

???

**Setting the stage**

Imagine that you are a new professor, just starting getting a group and research
programme going... or solo researcher or a small research group starts a study
to have data for their PhD students, but have limited funds and technical
expertise.

- Or, you are a small startup company trying get investment and build income
quickly... in the research realm so need to follow best practices/requirements
for data management... relies on data collection for business. Needs to get
operational quickly, but doesn't yet have funds to hire technical personnel.

- Or, you are a large, multi-national/center consortium that wants to keep better
track of who's working on what, and how to discover and share data added to the
project... or has an aim of widely disseminating their data for maximal, and
cost-effective, use by their collaborators and others.

All of these could use the framework to abide by the best practices in FAIR data
management.

---

## Aims of the DIF Project

???

We're still working out a better name, but for now we're calling it DIF

These aims may seem vague, but bare with me.

1. **Primary aim**: Create and implement an efficient, scalable, and open source
data infrastructure framework that connects data collectors, researchers,
clinicians, and other stakeholders, with the data, documentation, and findings
(starting within the DD2 study)

.footnote[Check out [DIF Project Website](https://steno-aarhus.github.io/dif-project/) for more details.]

???

Just for some clarification, infrastructure here meaning the computational 
structure of the data and all its support structures, for instance, how the files
and folders are structured, where the data files are saved and what file format,
how to connect to data. In many ways like the roads and buildings of a city,
where data is the people moving about.

"Framework" on the other hand is the bundle or package that contains the
instructions to create an infrastructure, that someone can take and use to
create the infrastructure somewhere else. You can think of this as the blueprint
for building a city.

2. **Secondary aim**: Create this framework so that *other research groups and
companies*, who are unable or can't build something similar, can relatively
easily implement it and modify as needed for their own purposes.

> In short: Make a software product that makes it easier to find, store, and
use data for research projects that abide by best practices, and make it so
that it is easy and free to use for others.

???

Again, these might not be really tangible to grasp what this actually means.

---

## Why is this important? 🤔

**Large trends across science in computing, data quantity, accountability, transparency**

???

Increasing need in science for...

- Computational tools and technologies
- Secure and reliable IT infrastructure
- Greater openness and transparency
- More reproducibility of studies
- Highly technical skills and knowledge
*... especially in relation to data management.*

Questions like:

- How do store your data? In what file format?
- Where do you store your data and how do you name the files?
- How do you keep track of changes to the data?
- (For multi-center studies) Who has which datasets and how do you combine them together?
- How do you or your collaborators find out what variables there are in the data, what do they mean?
- When there are errors or problems in your data, and you've already published
with or analyzed on it, how can you easily determine which publications used the
in correct data and how can you easily update the publications with the correct
data?
- How can you easily share your data with colleagues or reviewers to check your 
findings?

---

## Past and current barriers <svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#da9100;overflow:visible;position:relative;"><path d="M400 224h-24v-72C376 68.2 307.8 0 224 0S72 68.2 72 152v72H48c-26.5 0-48 21.5-48 48v192c0 26.5 21.5 48 48 48h352c26.5 0 48-21.5 48-48V272c0-26.5-21.5-48-48-48zm-104 0H152v-72c0-39.7 32.3-72 72-72s72 32.3 72 72v72z"/></svg>: Lack of funding, awareness, understanding, skill, and knowledge

???

- Funding agencies don't fully recognize these challenges, so don't provide
funding
- Researchers aren't aware of or understand the issues, or don't have skills to tackle them
- People with needed technical skills leave for industry

---

## Recent new funding 💰: NNF Data Science Research Infrastructure 5 year grant

???

Development of new ... methods and technologies within data science, ..., data
engineering, ...

---

## <svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#da9100;overflow:visible;position:relative;"><path d="M225.38 233.37c-12.5 12.5-12.5 32.76 0 45.25 12.49 12.5 32.76 12.5 45.25 0 12.5-12.5 12.5-32.76 0-45.25-12.5-12.49-32.76-12.49-45.25 0zM248 8C111.03 8 0 119.03 0 256s111.03 248 248 248 248-111.03 248-248S384.97 8 248 8zm126.14 148.05L308.17 300.4a31.938 31.938 0 0 1-15.77 15.77l-144.34 65.97c-16.65 7.61-33.81-9.55-26.2-26.2l65.98-144.35a31.938 31.938 0 0 1 15.77-15.77l144.34-65.97c16.65-7.6 33.8 9.55 26.19 26.2z"/></svg> Guiding principles

1. Follow and enable FAIR principles

2. Openly licensed and re-usable (e.g. CC-BY, MIT)

3. State-of-the-art principles and tools in software and UI design

4. Built from software that may be more familiar to researchers/academia

5. Friendly to beginner and non-technical users

???

FAIR = Findable Accessible Interoperable Reusable

---

---

---

---

---

---

---

## Interested in being involved or learning more? 🤓 Let me know! 🙋

---

# Reproducible Research in R (r-cubed) courses

---

## Reproducibility, a core principle of science, is rarely done

- Reproducibility: Same data + same analysis = same results?
- Replication: Same design + different data + same analysis = same results?

???

- *Non-replication* is a known major problem, but extent of non-reproducible
results is unknown. Barriers to addressing the problem include:
    - Lack of incentives to be reproducible
    - Emphasis on novelty and original work

---

## We don't share as much as we should

.footnote[
There are few studies on the extent of code and data availability, and whether study results
can be reproduced. Figure shows results of some of them:
1) [10.1177/2515245920918872](https://doi.org/10.1177/2515245920918872),
2) [10.1007/s11306-017-1299-3](https://link.springer.com/article/10.1007/s11306-017-1299-3),
3) [10.1371/journal.pone.0251194](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0251194).
]

???

- Estimating the reproducibility of scientific studies is currently very difficult 
because of:
    - Nearly non-existent publishing of code/data
    - General lack of awareness of and training in it

---

## We in research need more skills in data analysis 👩🏽‍💻

### ... and for more awareness and training on reproducibility and open science 🤯

???

While I've been teaching these general topics since my Masters, this course
specifically I started during my postdoc because one, there was a need for more
computational skills in my field and two, because the awareness around
reproducibility and open science was very lacking.

---

## Reproducible Research in R (R3 or r-cubed) course/workshop for PhD students and postdocs *doing biomedical research*

- Introduction course: [r-cubed.rostools.org](https://r-cubed.rostools.org)
    - JOSE paper: [10.21105/jose.00122](https://jose.theoj.org/papers/10.21105/jose.00122)
- Intermediate course: [r-cubed-intermediate.rostools.org](https://r-cubed-intermediate.rostools.org/)

???

The course is teaching reproducible research in R to PhD students and postdocs who
do biomedical research, largely diabetes research. 
Participants are working/full-time researchers (including PhD students), not
necessarily in an undergraduate context and related to learning data analysis or
more practical type skills.

This course is 3 full days, composing of 5 code along sessions where the
instructor types and the learners follow along, a few lectures, and a final
group project. For more info on the course, check out the links below.

---

## Key 🔑 features of course

1. Multiple activities to learning *in class* (reading, doing, listening,
discussing, teaching, group, and solo)

1. Openly licensed <svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#da9100;overflow:visible;position:relative;"><path d="M314.9 194.4v101.4h-28.3v120.5h-77.1V295.9h-28.3V194.4c0-4.4 1.6-8.2 4.6-11.3 3.1-3.1 6.9-4.7 11.3-4.7H299c4.1 0 7.8 1.6 11.1 4.7 3.1 3.2 4.8 6.9 4.8 11.3zm-101.5-63.7c0-23.3 11.5-35 34.5-35s34.5 11.7 34.5 35c0 23-11.5 34.5-34.5 34.5s-34.5-11.5-34.5-34.5zM247.6 8C389.4 8 496 118.1 496 256c0 147.1-118.5 248-248.4 248C113.6 504 0 394.5 0 256 0 123.1 104.7 8 247.6 8zm.8 44.7C130.2 52.7 44.7 150.6 44.7 256c0 109.8 91.2 202.8 203.7 202.8 103.2 0 202.8-81.1 202.8-202.8.1-113.8-90.2-203.3-202.8-203.3z"/></svg> and easily accessible <svg aria-hidden="true" role="img" viewBox="0 0 640 512" style="height:1em;width:1.25em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#da9100;overflow:visible;position:relative;"><path d="M634.91 154.88C457.74-8.99 182.19-8.93 5.09 154.88c-6.66 6.16-6.79 16.59-.35 22.98l34.24 33.97c6.14 6.1 16.02 6.23 22.4.38 145.92-133.68 371.3-133.71 517.25 0 6.38 5.85 16.26 5.71 22.4-.38l34.24-33.97c6.43-6.39 6.3-16.82-.36-22.98zM320 352c-35.35 0-64 28.65-64 64s28.65 64 64 64 64-28.65 64-64-28.65-64-64-64zm202.67-83.59c-115.26-101.93-290.21-101.82-405.34 0-6.9 6.1-7.12 16.69-.57 23.15l34.44 33.99c6 5.92 15.66 6.32 22.05.8 83.95-72.57 209.74-72.41 293.49 0 6.39 5.52 16.05 5.13 22.05-.8l34.44-33.99c6.56-6.46 6.33-17.06-.56-23.15z"/></svg> online

1. Written not just for participants but also (future) instructors

1. Largely hands-on (code-along), limit lectures and slides

???

Briefly discuss before showing website.

---

## Try out the material and give us feedback on it! 🤓

???

And to end, please, if you try out the material, lets us know! We'd love more
feedback on it! Thanks for listening!