Skip to content

Build a record in code

Instead of writing a record's JSON by hand, you can generate Python classes from the model and build the record as Python objects. There is one class for each kind of thing the model defines, such as a dataset or an organization. This example builds the example record from the Examples page (dataset-350.yaml) as a Dataset. It then writes the record out as JSON, and checks it as a Dataset with LinkML's validator.

Set up

You need Python 3.11 or newer (https://www.python.org/downloads/).

Work in a new folder, with a private Python environment for the tools:

mkdir midas-record && cd midas-record
python3 -m venv venv
source venv/bin/activate
pip install linkml==1.11.1

linkml==1.11.1 is the LinkML version that generates the published files.

Generate the classes

Download the model's YAML file, and generate Python classes from it with gen-pydantic. The classes use Pydantic, a Python library for data validation:

curl -sSfL -o midas-catalog.yaml https://w3id.org/midas-catalog/linkml/generated/midas-catalog.yaml
gen-pydantic midas-catalog.yaml > midas_catalog.py

midas_catalog.py has a Python class for each class in the model, such as Dataset, Organization, DefinedTerm and Place. Each Python class has one attribute for each of the model class's fields.

Build the record

Save this as build_record.py:

# build_record.py
from midas_catalog import Dataset, DefinedTerm, Organization, Place

record = Dataset(
    id="https://w3id.org/midas-catalog/350",
    name="2022-2023 U.S. Map & Case Count",
    description=(
        "The archived dataset (CSV) contains Mpox and Orthopoxvirus confirmed cumulative case and "
        "death counts in each U.S. state and territory, including cases of non-U.S. residents in the "
        "U.S from 2022-2023. The data are downloadable and also visible on the website on a U.S. map "
        "and in a table."
    ),
    creator=[
        Organization(
            id="https://www.cdc.gov/",
            name="Centers for Disease Control and Prevention of the United States",
        )
    ],
    about=[
        DefinedTerm(
            id="http://purl.obolibrary.org/obo/DOID_3292",
            name="mpox",
            inDefinedTermSet="http://w3id.org/midas-metadata/midas2",
        )
    ],
    places=[Place(id="https://sws.geonames.org/6252001/", name="United States")],
)

with open("my-dataset.json", "w", encoding="utf-8") as f:
    f.write(record.model_dump_json(exclude_none=True, indent=2) + "\n")
print("wrote my-dataset.json")
python build_record.py
wrote my-dataset.json

my-dataset.json holds the same record as dataset-350.json, with its fields in the order the classes define them:

{
  "id": "https://w3id.org/midas-catalog/350",
  "name": "2022-2023 U.S. Map & Case Count",
  "description": "The archived dataset (CSV) contains Mpox and Orthopoxvirus confirmed cumulative case and death counts in each U.S. state and territory, including cases of non-U.S. residents in the U.S from 2022-2023. The data are downloadable and also visible on the website on a U.S. map and in a table.",
  "about": [
    {
      "id": "http://purl.obolibrary.org/obo/DOID_3292",
      "name": "mpox",
      "inDefinedTermSet": "http://w3id.org/midas-metadata/midas2"
    }
  ],
  "creator": [
    {
      "id": "https://www.cdc.gov/",
      "name": "Centers for Disease Control and Prevention of the United States",
      "agentType": "schema:Organization"
    }
  ],
  "places": [
    {
      "id": "https://sws.geonames.org/6252001/",
      "name": "United States"
    }
  ]
}

The script doesn't set the creator's agentType. The Organization class sets it: its agentType is schema:Organization unless you give another value. In the same way, the Person class's agentType is schema:Person.

exclude_none=True leaves out the fields the script didn't set. Without it, each of those fields is written as null, and there are 27 of them on the dataset alone. The schema accepts null, but leaving the fields out keeps the record the same as dataset-350.json.

Validate it

Check the record with LinkML's validator, linkml-validate, as on Check a record. It reads the model's YAML file, which you downloaded to generate the classes:

linkml-validate -s midas-catalog.yaml -C Dataset my-dataset.json
No issues found

The classes catch some mistakes while you build the record. If you leave out description, or add a field the model doesn't have, Pydantic raises a ValidationError. But the classes don't check everything. They don't check which namespaces a subject term may come from. A term's id is an IRI, a web address used as an identifier. A namespace is the start that all IRIs from one vocabulary share. So a DefinedTerm whose id starts with https://purl.obolibrary.org/ builds without an error. linkml-validate does catch it, so validate the record you write.

Each time this site is built, the build runs this example from start to finish, with classes generated from the same model. The build fails if the record it writes doesn't validate, or if it differs from dataset-350.json.