reproducibility
Table of Contents
- 1. reproducibility
- 1.1. ReScience/ten-years: Ten Years Reproducibility Challenge
- 1.2. Replication crisis - Wikipedia
- 1.3. Is the reproducibility crisis reproducible? - by Ben Recht
- 1.4. Reproducible Data Science and why it matters | by Carl W. Handlin | Rappi Tech
- 1.5. Barriers to reproducibility — The Turing Way
- 1.6. Reproducible Builds — a set of software development practices that create an independently-verifiable path from source to binary code
- 1.7. GoogleCloudPlatform/professional-services-data-validator: Utility to compare data between homogeneous or heterogeneous environments to ensure source and target tables match
- 1.8. diffoscope: in-depth comparison of files, archives, and directories
- 1.9. Outreach and Publications - Research Object Crate (RO-Crate)
- 1.10. Tazro Ohta Verifying workflow reproducibility using RO-Crate - YouTube
1. reproducibility
1.2. Replication crisis - Wikipedia
https://en.wikipedia.org/wiki/Replication_crisis
Cita trabajos de Luhman acerca de porqué pasa, en general por que la gente tiene distintos códigos sobre lo que es importante y lo que no
Social systems theory, as expounded in the work of German sociologist Niklas Luhmann, offers a similar diagnosis.
This theory holds that each system, such as economy, science, religion or media, communicates using its own code: true and false for science, profit and loss for the economy, news and no-news for the media, and so on.
According to some sociologists, science’s mediatization, its commodification and its politicization, as a result of the structural coupling among systems, have led to a confusion of the original system codes.
If science’s code of true and false is substituted with those of the other systems, such as profit and loss or news and no-news, science enters into an internal crisis.
1.5. Barriers to reproducibility — The Turing Way
- Limited incentives to give evidence against yourself (if I share my data people may find mistakes)
- Publication bias towards novel findings
- Held to higher standards than others
- Not considered for promotion
- Big data and complex computational infrastructure
- Being reproducible does not mean the answer is right
- Takes time
1.6. Reproducible Builds — a set of software development practices that create an independently-verifiable path from source to binary code
Sponsorized by google
1.10. Tazro Ohta Verifying workflow reproducibility using RO-Crate - YouTube
1.10.1. sapporo-wes/tonkaz: CLI tool to verify workflow reproducibility
Tonkaz is a CLI tool to verify workflow reproducibility. It compares the RO-Crate of workflow execution results and calculates the reproducibility level of each output file.
- Level3 ⭐⭐⭐ : Files are identical with the same checksum
- Level2 ⭐⭐ : Files are different, but their features (file size, map rate, etc.) are similar (within threshold: 0.05)
- Level1 ⭐ : Files are different, and their features are different (beyond threshold)
- Level0 : File not found
Tonkaz supports ONLY RO-Crate generated by Sapporo-service and Yevis-cli.