The Role of Metadata in Reproducible Computational Research

This is a supplemental resource to Leipzig et al. "The Role of Metadata in Reproducible Computational Research" now published in Cell Patterns https://www.cell.com/patterns/fulltext/S2666-3899(21)00170-7

Contributions are welcome!

Organization

├───data/
│   ├───examples/                  Examples of metadata standards
│   ├───lens/                      Search exports for scimetric journal analysis
│   └───standards.tsv              Raw standards table
├───src/
│   ├───cwl/tools/                 CWL configuration to produce the timeline plot
│   ├───manuscript/                Manuscript revision document
│   ├───secrets/
│   │   └───api.template.py        Replace this with api.py using your NCBI/NCBO keys
│   ├───ontologies/                Scimetric ontology popularity analysis
│   ├───repotutils/                Scripts for automating management of this repository
│   ├───scimetric/                 Scimetric journal meta/rcr frequency analysis in a Jupyter Notebook
│   ├───timeline/                  R Markdown document to produce the RCR case study timeline in the paper, incl. helper files for execution with CWL (wrapper script, Dockerfile)
│   ├───wget2jsonld.py             Helper script to convert wget output to jsonld
│   └───wordcloud/                 R script to produce word cloud from cited abstracts
├───LICENSE                        The LICENSE file
├───README.md                      What you are looking at
├───environment.osx.yaml           OSX pinned Conda depenencies
├───environment.unpinned.yaml      Unpinned Conda depenencies
└───ro-crate-metadata.jsonld       RO Crate config
└───.binder                        Environment configuration files for usage with Binder (mybinder.org)

Examples of RCR metadata standards

In this table we provide links to the authoritative publications and homepages for these metadata standards, as well as examples we have collected. Schema refers the parent structure this standard conforms to, if any. Encoding refers to the markup format used. Note that for schemas such as OWL, which relies on RDF subject–predicate–object triplets, the encoding could be one of at least seven serialization types (RDF/XML, RDF/JSON, JSON-LD, Turtle, N-Triples, N-Quads, N3), so the listed encoding is somewhat arbitrary. For other standards, such as DICOM, the encoding is a custom binary although there are numerous export format and even attempts to serialize JSON within DICOM.

[📚] Publication [🏠] Homepage [📋] Example

Standard	Layer	Domain	Encoding	Schema	Description
CellML 📚 🏠 📋	Input	Biology	XML	RDF	mathematical models for biology
CIF2 📚 🏠	Input	Crystallography	Custom		atomic structure
DATS 📚 🏠	Input	Biomedical	JSON		desc metadata (people, org, repo) for data pubs
DICOM 📚 🏠 📋	Input	Images	Custom	Key-Value	standard for all medical imaging
EML 📚 🏠	Input	Ecology	XML		eco support for geo, species, pubs used in KNB
FAANG  🏠	Input	Specimens	Tabular		sample and experiment metadata for farm animal genomes
GBIF 📚 🏠	Input	Biodiversity	JSON		species occurrence and biodiversity records
GO 📚 🏠	Input	Genes	XML		controlled vocabulary of gene and gene product functions
ISO/TC 276  🏠	Input	Biotechnology			ISO committee defining biotechnology data standards
MIAME 📚 🏠	Input	Microarrays	XML		minimum information to interpret a microarray experiment
NetCDF 📚 🏠	Input	Arrays			self-describing, array-oriented scientific data
OGC  🏠	Input	Geospatial			open standards for geospatial content and services
ThermoML 📚 🏠	Input	Compounds	XML		thermodynamic and thermophysical property data
CRAN  🏠	Tools	R packages			R package DESCRIPTION metadata (dependencies, authors)
Conda  🏠	Tools	Dependencies			package and environment dependency specifications
pip setup.cfg  🏠	Tools	Python modules	CFG	Key-Value	Python cfg files have headers and key-value pairs similar to Windows INI files
EDAM 📚 🏠	Tools	Bfx data			ontology of bioinformatics operations, data, formats, and topics
CodeMeta  🏠	Tools	Source code			schema.org-based crosswalk for software metadata
Biotoolsxsd 📚 🏠	Tools	Bfx software	XML		XML schema behind the bio.tools software registry
DOAP  🏠	Tools	Software	XML		Description of a Project vocabulary for software projects
ontosoft  🏠	Tools	Geo software			ontology for describing and sharing scientific software
SWO 📚 🏠	Tools	Bfx Software			ontology of software, its inputs, outputs, and tasks
OBCS 📚 🏠	Reports	Biostatistics			ontology of biological and clinical statistics
STATO  🏠	Reports	Statistics			ontology of statistical methods and tests
SDMX  🏠	Reports	Statistics	JSON		standard for exchanging statistical data and metadata
DDI  🏠	Reports	Studies	XML		documentation for social, behavioral, and economic data
MEX 📚 🏠	Reports	ML	XML		lightweight vocabulary for interchanging machine learning experiments
MLSchema  🏠	Reports	ML			schema for describing machine learning algorithms and experiments
MLFlow  🏠	Reports	ML			tracking of ML experiments, runs, parameters, and models
Rmd  🏠	Reports	Docs	YAML	Key-Value	YAML front matter for reproducible R Markdown reports
CWL 📚 🏠	Tools, Pipelines		YAML	Schema Salad	Common Workflow Language specifies how to invoke a command line tool or a pipeline of such tools
CWLProv 📚 🏠	Pipelines		YAML, JSON, XML		BagIt of Research Object folder containing manifest (JSON-LD), CWL (YAML), PROV (JSON, XML, RDF)
RO-Crate  🏠	Input, Pipelines, Publication		JSON-LD	RDF, schema.org	RO-Crate is a profile of using schema.org to annotate any collections of research data and their real-life origins
RO  🏠	Pipelines		Turtle, JSON-LD, XML	OWL	bundles data, methods, and provenance into a single research object
WICUS  🏠	Pipelines				ontology for conserving the computational infrastructure of workflows
OPM  🏠	Pipelines				model for representing provenance as a directed graph
PROV-O  🏠	Pipelines			OWL	Several PROV serializations exists; PROV-O is in OWL, which again has many serializations including the RDF syntaxes
ReproZip  🏠	Pipelines				The config for ReproZip is YAML. The actual recorded sessions are more like coredumps.
ProvOne  🏠	Pipelines				extends PROV to capture scientific workflow provenance
WES	Pipelines				GA4GH API for submitting and monitoring workflow runs
BagIt  🏠	Input, Pipelines		Text	Key-Value	For long-term perservation and availability BagIt specifies a fixed folder structure of payload files, their checksums and other metadata tag files. Bags can be archived as zip, tar, etc or remain folders
BCO	Pipelines				BioCompute Object records regulatory bioinformatics pipelines
ERC 📚 🏠	Pipelines	Research Compendia	YAML	Key-Value	Executable Research Compendium packaging code, data, and environment
BEL	Publication				Biological Expression Language for causal biological relationships
DC	Publication				Dublin Core general-purpose metadata element set
JATS  🏠	Publication	Articles	XML	Tags DTD	XML tag set for marking up scholarly journal articles
ONIX	Publication				metadata standard for the book and publishing supply chain
MeSH	Publication				controlled vocabulary for indexing biomedical literature
LCSH	Publication				Library of Congress subject headings for bibliographic records
MP 📚	Publication	Micropublications		OWL	micropublications model linking scientific claims to evidence
Open PHACTS 📚 🏠	Publication	Drugs		RDF	semantic interoperability across drug discovery datasets
SWAN 📚	Publication	Neuromedicine			ontology for scientific discourse in neuromedicine
SPAR  🏠	Publication	Publishing		OWL	suite of ontologies for semantic publishing and referencing
PWO 📚	Publication	Publishing			ontology describing the steps of a publishing workflow
PAV 📚	Publication	Authorship		OWL	provenance, authoring, and versioning ontology
Manubot   📋	Publication	Publishing	YAML		metadata for collaborative, automated manuscript writing
ReScience   📋	Publication	Publishing	YAML		metadata for peer-reviewed replications of computational research
PandocScholar   📋	Publication	Publishing	YAML		scholarly article metadata authored through Pandoc

RDF vs OWL https://stackoverflow.com/questions/1740341/what-is-the-difference-between-rdf-and-owl

How to generate the timeline for this article

Install cwltool

pip install cwltool
cwltool src/cwl/tools/timeline.cwl --reportfile timeline.html

Note that the tools requires Docker for runningthe computing environment, see the file timeline/Dockerfile for the definition of the image used in the .cwl file.

Run on Binder

MyBinder is a tool for creating executable computing environments based on standard and widely used dependency management files. You can easily run important parts of the analysis for the manuscript by clicking on the badges below. Binder will create a container using the environment configuration from the directory .binder/ and provide you with an interactive environment to execute notebooks or scripts.

• Scimetric journal frequency analysis of RCR and metadata terms (opens a Jupyter Notebook)
• Create Figure 2 from the paper (R Markdown notebook, open the file src/timeline/timeline.Rmd manually in RStudio)
• Create word cloud from cited abstracts (run R script src/wordcloud/wordcloud.R)

For development purposes, you can also run repo2docker locally in the directory of the repository.

repo2docker --editable .

License