Skip to content

Get a catalog record as YAML

The MIDAS Catalog publishes every record as JSON-LD (JSON for linked data), at https://catalog.midasnetwork.us/linkml/{id}.jsonld. LinkML can turn such a record into plain data. This example converts catalog record 350 to YAML, then checks the result as a Dataset.

Set up

You need Python 3.11 or newer (https://www.python.org/downloads/).

Work in a new folder, with a private Python environment (a virtual environment) for the tools:

mkdir record-yaml && cd record-yaml
python3 -m venv venv
source venv/bin/activate
pip install linkml==1.11.1

linkml==1.11.1 is the LinkML version that generates the published files. It includes linkml-convert, the converter this page uses, and linkml-validate, the validator.

Convert the record

Download record 350 and the model's YAML file, and convert the record to YAML:

curl -sSfL -o 350.json https://catalog.midasnetwork.us/linkml/350.jsonld
curl -sSfL -o midas-catalog.yaml https://w3id.org/midas-catalog/linkml/generated/midas-catalog.yaml
linkml-convert -s midas-catalog.yaml -C Dataset -t yaml 350.json

The output is 511 lines long. These are the first 10:

id: https://w3id.org/midas-catalog/350
identifier: '350'
name: 2022-2023 U.S. Map & Case Count
description: The archived dataset (CSV) contains Mpox and Orthopoxvirus confirmed
  cumulative case and death counts in each U.S. state and territory, including cases
  of non-U.S. residents in the U.S from 2022-2023. The data are downloadable and also
  visible on the website on a U.S. map and in a table.
url: https://catalog.midasnetwork.us/collection/350
sameAs: https://archive.cdc.gov/#/details?url=https://www.cdc.gov/poxvirus/mpox/response/2022/us-map.html
conformsTo:
  • The record is saved as 350.json because linkml-convert chooses how to read a file from its extension, and it has no reader for .jsonld. A JSON-LD record is JSON, so the JSON reader can read it.
  • -C Dataset names the record's type. The record's own @type tells you which type to name: here, https://schema.org/Dataset is the model's Dataset. A record's @type can be shared by more than one record type, so name the type you want with -C.
  • The conversion drops @context and @type, which leaves the plain record. It has all 28 of record 350's fields, with the same structure as the example record dataset-350.yaml.

Validate it

linkml-convert checks the record as a Dataset while it converts it. If the record isn't valid, it stops with an error instead of printing YAML. To check the result yourself, save the YAML and validate it with linkml-validate:

linkml-convert -s midas-catalog.yaml -C Dataset -t yaml 350.json > 350.yaml
linkml-validate -s midas-catalog.yaml -C Dataset 350.yaml
No issues found

Each time this site is built, the build converts a saved copy of record 350 with the command above. The build fails if the YAML doesn't validate, or if it differs from the record without its @context and @type.