maDMP to human readable DMP

July 1, 2019 · View on GitHub

How it works

Our solution uses Jinja templates to convert the provided JSON maDMP (via the -d argument) to an HTML string (styled with CSS to match the template) which is subsequently converted to PDF with the pdfkit package.

The text used to make the maDMP more "human-like" is specified directly in the template, which makes tweaking and changing it quite straight-forward. The mapping of maDMP fields to the template structure can also be changed right inside of the Jinja template. This approach makes it easy for non-coders to adjust the template to their needs with very basic HTML and Jinja knowledge needed.

How to run

  1. (Use Pyhton 3.7)
  2. Clone repository
  3. Run pip install -r requirements.txt to install required packages
  4. cd to src directory
  5. Run python main.py -d path/to/dmp.json -t [FWF|HORIZON]

The -d argument specifies the path of the maDMP to convert (e.g. ../dmps/DMP_ML_slim.json), while the -t argument specifies the template to use, which can be either FWF or HORIZON.

The repository containing the full code as well as this README can be found at: https://github.com/datastewardstuw/DataStewards2

The original DMPs can be found in the repository as well inside the original directory, while the maDMP versions are located in the dmps directory. There are several versions of the DMP_ML with different levels of completeness, which were used to test the output our converter generates when fields are missing.

The generated human readable DMPS are located in the dmps directory as well, and are named after the JSON files they originate from, adding either FWF or HORIZON depending on which template was used, and out to be able to discern them from the input files (not only based on the file extension).

An example DMP generated with the FWF template is shown below: fwf-example.png

Comparison

As a first step two Data Management Plans have been created and manually converted into machine-actionable DMPs. In a further step they have been converted back again into a human-readable format. The workflow looks simplified as follows:

human-readable (pdf) → machine-actionable (json) → human-readable (pdf)

In order to compare the original and the converted DMPs we have created a table with the categories for each section, which are part of the relevant DMP templates. They are either FWF or Horizon 2020. As a start we will do a short overview (1) how they differ. For each category we will check where (2) information got lost and (3) which elements are more detailed. To finish we will discuss (4) which parts of the maDMP were how difficult to generate.

Original vs. Generated DMP

The templates of the DMPs have been designed to match the available templates of the FWF and the Horizon 2020 demo reports. Therefore, they have a much clearer appereance compared to the used online editors (like DMPOnline).

Categorical Comparison

FWF Template

As the original Data Management Plan was the basis for the generated one, they should not differ very much, because each information should be the same or at least similar.

The DMPs can be found in the directory "dmps" in the GitHub Repo. For comparison we used

FWForiginal DMPconverted DMP
I Data Characteristics
I.1 Description of the dataMore information about how the data is processed, the programming environments and where it is from.More information about the project team, different deadlines and funding statuses.
II Documentation and Metadata
II.1 Metadata standardsDescribes the metadata information very well.Misses some key facts about Metadata.
II.2 Documentation of dataWell described.Information about how the data has been documented got lost in the transformation.
II.3 Data quality controlWas not part of it.Is also unknown.
III Data Availability and Storage
III.1 Data sharing strategyReadme files and DOI are already included.All information was transformed very well.
III.2 Data storage strategyAll relevant (and available) information is already contained.All available with the bonus of extimated project costs.
IV Legal and Ethical Aspects
IV.1 Legal aspectsLicense is included.License is included with dates.
IV.2 Ethical aspectsAll ethical issues have been addressed and discussed.All information has been taken along and added with preservation statements.

Horizon 2020 Template

The Horizon 2020 Output is almost the same as the FWF template with different changes in the categories. For a better compability we used the same layout, but obviously with different categories and subcategories.

All DMP plans stay the same with exception of the generated DMPs (DMP_BK_HORIZON_out and DMP_ML_full_HORIZON_out)

As the all six main categories use the same attributes or the same information from the machine-actionable DMP, the information remains the same. To have a better overview we will show them here:

  1. Data summary
  2. FAIR Data 2.1 Making data findable (metadata) 2.2 Making data openly accessible 2.3 Making data interoperable 2.4 Increase data re-use (licences)
  3. Allocation of resources
  4. Data security
  5. Ethical aspects
  6. Other

Discussion

As already mentioned both DMP templates have the same information basis and contain mostly the same information. There are some differences but mainly with regard to the design. Overall, it makes sense to use machine-actionable DMPs to generate human-readable Data Management Plans, because the always result in the same design and the are or can be restricted with different rules to ensure a good level of data quality. One hard thing is to explain each part as good as possible, so that the understanding of a clause is almost the same for everyone. This is a tricky part, but can be almost bypassed by meaningful explanations and samples.

By creating the machine-actionable DMPs we encountered a few misleading fields, which required some reading to better understand the meaning of the fields. This could be improved but nevertheless all fields already contain a good definition of the requested information.