Testing

jsonschema test [schemas-or-directories...]
  [--http/-h] [--verbose/-v] [--debug/-g]
  [--header/-H "<name>: <value>"]
  [--resolve/-r <schemas-or-directories> ...]
  [--extension/-e <extension>] [--ignore/-i <schemas-or-directories>]
  [--default-dialect/-d <uri>] [--json/-j] [--format-assertion/-F]

Schemas are code. As such, you should run an automated unit testing suite against them. Just like popular test frameworks like Jest, GoogleTest, and PyTest, the JSON Schema CLI provides a schema-oriented test runner inspired by the official JSON Schema test suite.

If you want to validate that a schema adheres to its metaschema, use the metaschema command instead.

Pass --json to output results in CTRF (Common Test Report Format), a standardized JSON format for test results that integrates with CI/CD tools and test result dashboards.

Writing tests

To test a schema, you define one or more test suite (i.e. collections of tests) as JSON files that follow a specific format:

• target: The URI of the schema you want to test, which must be imported into the resolution context using the --resolve or --http options.

  If the target is relative, it will be interpreted as a file path relative to the test file location.

  This property may also be set to a non-empty array of URIs, in which case every test case in tests will be run against each of the listed schemas in turn. This is convenient for asserting that a single set of instances is accepted (or rejected) the same way by multiple related schemas.

• tests: An array of tests you want to run.

Every item in the tests array must be an object with the following properties:

• valid: A boolean that determines whether the given instance is expected to be valid or not against the target schema. In other words, if this property is true, the test passes when the instance successfully validates against the schema. Conversely, if this property is false, the test passes when the instance does NOT validate against the schema
• description: An optional string property to make test output more readable

And either of the following properties, but not both:

• data: The instance that you want to test against the target schema.
• dataPath: The instance that you want to test against the target schema, loaded from an external file instead

For example, here is a minimal test suite that expects an object with two properties (foo and bar) to successfully validate against the target schema https://example.com/my-schema-id:

{
  "target": "https://example.com/my-schema-id",
  "tests": [
    {
      "description": "I expect to pass",
      "valid": true,
      "data": {
        "foo": 1,
        "bar": 1
      }
    }
  ]
}

Examples

This is a test suite definition that runs a few test cases against the official JSON Schema Draft 4 meta-schema. The first test asserts that the instance {} is valid. The second test asserts that a schema where the type keyword is set to an integer is invalid. The third test asserts that an instance loaded from a relative path is valid against the schema:

{
  "target": "http://json-schema.org/draft-04/schema#",
  "$comment": "An arbitrary comment! Put whatever you want here",
  "tests": [
    {
      "description": "The empty object is valid",
      "valid": true,
      "data": {}
    },
    {
      "description": "The `type` keyword must be a string",
      "valid": false,
      "data": {
        "type": 1
      }
    },
    {
      "description": "Load from an external file, relative to the test",
      "valid": true,
      "dataPath": "../my-data.json"
    }
  ]
}

Assuming this file is saved as test/draft4.json, you can run it as follows:

jsonschema test test/draft4.json

Run a single test definition

jsonschema test path/to/test.json

Run every .json test definition in a given directory (recursively)

jsonschema test path/to/tests/

Run every .json test definition in the current directory (recursively)

jsonschema test

Run every .json test definition in the current directory while ignoring another

jsonschema test --ignore dist

Run every .test.json test definition in the current directory (recursively)

jsonschema test --extension .test.json

Run a single test definition enabling HTTP resolution

jsonschema test path/to/test.json --http

Run a single test definition with a custom HTTP header

jsonschema test path/to/test.json \
  --http --header "Authorization: Bearer $REGISTRY_TOKEN"

Run a single test definition importing a single local schema

jsonschema test path/to/test.json --resolve path/to/external.json

Run a single test definition importing a directory of .schema.json schemas

jsonschema test path/to/test.json --resolve path/to/schemas --extension schema.json

Run a single test definition forcing every format to assert

The --format-assertion/-F option forces every format keyword in the schema under test to behave as an assertion regardless of dialect or vocabulary, which lets tests cover invalid-format cases without changing the schema.

jsonschema test path/to/test.json --resolve path/to/schema.json --format-assertion