OCSF Schema Compiler

This is a Python library and command-line tool for compiling the Open Cybersecurity Schema Framework (OCSF) schema, specifically the schema at https://github.com/ocsf/ocsf-schema.

This project is published to PyPI: ocsf-schema-compiler · PyPI.

Getting started

There are three ways to use the OCSF Schema Compiler:

1. As a command-line tool, installed from PyPI.
2. As a library, installed from PyPI.
3. As a developer working on this project.

Python version 3.14 or later is required. The core project code does not require any dependencies, though optional developer dependencies are used (see below).

Using ocsf-schema-compiler as a command-line tool

Create a virtual environment then install with pip. For example:

python3 -m venv .venv
source ./.venv/bin/activate
python -m pip install ocsf-schema-compiler

Running from this environment is now a matter of calling ocsf-schema-compiler:

ocsf-schema-compiler -h

The basic usage is passing the base directory of a schema to the compiler and capturing the output to a file.

ocsf-schema-compiler path/to/ocsf-schema > schema.json

Using ocsf-schema-compiler as a library

Create a virtual environment then install with pip. For example:

python3 -m venv .venv
source ./.venv/bin/activate
python -m pip install ocsf-schema-compiler

The compiler is implemented in the SchemaCompiler class. The class constructor accepts the same options as the command-line tool. The class's compile method does the heavy lifting, returning a dict containing the compiled schema. Specifically, compiler returns an ocsf_schema_compiler.jsonish.JObject, which is a type alias for JSON-compatible dict.

from pathlib import Path
from ocsf_schema_compiler.compiler import SchemaCompiler


compiler = SchemaCompiler(Path("path/to/ocsf-schema"))
output = compiler.compile()

The returned compile_version 1 structure is documented in Compiled schema format.

See ocsf_schema_compiler.__main__ for a working example.

Developing ocsf-schema-compiler

The recommended way to work on OCSF projects is to create fork in your own GitHub profile or organization. Create your fork of this repo using the GitHub CLI tool (or, more painfully, manually).

This project requires Python 3.14 or later and uses basedpyright for type checking and Ruff for linting and code formatting. There are no runtime dependencies.

This project intentionally uses unpinned versions of the basedpyright and ruff tool so we use their evolving lint rules. Because of this, it is a best practice to update to the latest versions of these tools and fixing newly identified issues before making other changes. Lint and formatting chances should ideally be placed in their own pull requests.

# Get / update pip dependencies and run the lint tools
make pip-update lint
# If issues are found, create a branch to fix these issues
git checkout -b lint-fixes
# Fix the issues
# Create a PR

NOTE: Please avoid "fixing" lint issue with ignore directives.

These ignore directives should only be used when there is no good way to fix this issue. For example, the tests use # pyright: ignore[reportImplicitRelativeImport] to import from other test modules since I couldn't find another way to do this.

This project includes regression tests in the tests directory, built using the unittest library. These can also be run without a virtual environment so long as python3 refers to Python 3.14 or later. The tests can be run with the Makefile target tests.

make tests

Basedpyright was picked as an alternative to Pylance because I'm using the open-source and telemetry-free VSCodium variation of VS Code. The Microsoft-proprietary Pylance extension (part of the Python extension) does not work in VSCodium by design. Basedpyright also offers other benefits: it is strict by default and includes additional type checking rules. Extensions are available for both VSCodium and VS Code; in both cases look for "BasedPyright" by detachhead. Use in VS Code does, however, take a little more work. I hope Pyright fans — and especially VS Code users — will find this workable, and perhaps consider using the privacy-focused VSCodium themselves.

The choice of Ruff should be less controversial: it is the current favorite for Python linting and code formatting. The formatting is very similar to Black with some minor differences (which I consider improvements).

Use of these tools requires a virtual environment. With the virtual environment activated, the linting and formatting can be run with the Makefile target lint.

Integrating basedpyright and Ruff with your editor is recommended. In both cases, these tools will pick up the configuration in this project's pyproject.toml file.

• Editor integration | Ruff
• IDEs - basedpyright

Note: this project's .gitignore assumes the virtual environment is at .venv.

# A standard Python virtual environment works fine
python3 -m venv .venv
source ./.venv/bin/activate

# Install the tools
# The make pip-update target gets / updates the needed dependencies
make pip-update

# Now the lint target will work
make lint

To run local development changes, use of a virtual environment is recommended through use of the pip tool's editable requirement feature.

# A standard Python virtual environment works fine
python3 -m venv .venv
source ./.venv/bin/activate

# The pip-update make target updates the dev dependencies and
# install the project as and an editable requirement ("develop mode")
make pip-update

# Or only install the project as an editable requirement
python -m pip install -e .

Since this repo has no runtime dependencies, you can also run it directly from a cloned repo's src directory without creating a virtual environment. This is more of a novelty than a best practice as it skips installation of the lint and format tools.

cd path/to/ocsf-schema-compiler
cd src
python3 -m ocsf_schema_compiler ~/path/to/ocsf-schema > ~/path/to/output/schema.json

To ensure the project can be built for distribution, the build-check target in the Makefile can be used. This project is built with Flit, a modern minimal build and publishing tool. It can be run locally to ensure the project remains buildable. This target runs flit build and flit install to build the package and install it locally, and then runs ocsf-schema-compiler -h to verify it works.

# Create virtual environment if it doesn't already exist
python3 -m venv .venv
# Activate it
source ./.venv/bin/activate
# Install Flit
python -m pip install -U flit

# Run the build check
make build-check

Continuous integration

The continuous integration is done via a GitHub action in .github/workflows/ci.yaml. This action uses the Makefile targets test, lint-github, and build-check. (The lint-github target is a minor variation of the lint target with Ruff's GitHub output format option.)

Publishing

Publishing details are covered in docs/publishing.md.

Copyright

Copyright © OCSF a Series of LF Projects, LLC. See NOTICE for details.

License

This project is distributed under the Apache License Version 2.0. See LICENSE for details.