JSON schema

This document outlines the JSON schemas used by the testing library for its ABI entry point and for the --event-stream-output-path command-line argument. For more information about the ABI entry point, see the documentation for ABI.v0.EntryPoint.

Modified Backus-Naur form

This schema is expressed using a modified Backus-Naur syntax. {, }, :, and , represent their corresponding JSON tokens. \n represents an ASCII newline character.

The order of keys in JSON objects is not normative. Whitespace in this schema is not normative; it is present to help the reader understand the content of the various JSON objects in the schema. The event stream is output using the JSON Lines format and does not include newline characters (except one at the end of the <output-record-line> rule.)

Trailing commas in JSON objects and arrays are only to be included where syntactically valid.

Common data types

<string> and <number> are defined as in JSON. <array:T> represents an array (also defined as in JSON) whose elements all follow rule <T>.

<bool> ::= true | false ; as in JSON

<source-location> ::= {
  ["fileID": <string>,] ; the Swift file ID of the file if available, as per
                        ; SE-0274 § "Specification of the #file string format"
  "filePath": <string>, ; the compile-time path to the file
  "line": <number>,
  "column": <number>,
}

<instant> ::= {
  "absolute": <number>, ; floating-point seconds since system-defined epoch
  "since1970": <number>, ; floating-point seconds since 1970-01-01 00:00:00 UT
}

<version> ::= "version": <version-number>
<version-number> ::= 0 | "<version core>" ; as per https://semver.org

Streams

A stream consists of a sequence of values encoded as JSON Lines. A single instance of <output-stream> is defined per test process and can be accessed by passing --event-stream-output-path to the test executable created by swift build --build-tests.

<output-stream> ::= <output-record>\n | <output-record>\n <output-stream>

Records

Records represent the values produced on a stream. Each record is encoded on a single line and can be decoded independently of other lines. If a decoder encounters a record whose "kind" field is unrecognized, the decoder should ignore that line.

<output-record> ::= <test-record> | <event-record>

<test-record> ::= {
  <version>,
  "kind": "test",
  "payload": <test>
}

<event-record> ::= {
  <version>,
  "kind": "event",
  "payload": <event>
}

Tests

Test records represent individual test functions and test suites. Test records are passed through the record stream before most events.

<test> ::= <test-suite> | <test-function>

<test-suite> ::= {
  "kind": "suite",
  "name": <string>, ; the unformatted, unqualified type name
  ["displayName": <string>,] ; the user-supplied custom display name
  "sourceLocation": <source-location>, ; where the test suite is defined
  "id": <test-id>,
}

<test-function> ::= {
  "kind": "function",
  "name": <string>, ; the unformatted function name
  ["displayName": <string>,] ; the user-supplied custom display name
  "sourceLocation": <source-location>, ; where the test is defined
  "id": <test-id>,
  "isParameterized": <bool>, ; is this a parameterized test function or not?
  ["tags": <array:tag>,] ; the tags associated with this test function
  ["bugs": <array:bug>,] ; the bugs associated with this test function
  ["timeLimit": <number>] ; the time limit associated with this test function
}

<test-id> ::= <string> ; an opaque string representing the test case

<tag> ::= <string> ; a string representation of a tag

<bug> ::= {
  ["url": <string>,] ; the bug URL
  ["id": <string>,] ; the bug id
  ["title": <string>] ; the human readable bug title
}

Events

Event records represent things that can happen during testing. They include information about the event such as when it occurred and where in the test source it occurred. They also include a "messages" field that contains sufficient information to display the event in a human-readable format.

<event> ::= {
  "kind": <event-kind>,
  "instant": <instant>, ; when the event occurred
  ["issue": <issue>,] ; the recorded issue (if "kind" is "issueRecorded")
  ["attachment": <attachment>,] ; the attachment (if kind is "valueAttached")
  "messages": <array:message>,
  ["testID": <test-id>,]
  ["iteration": <number>,] ; the iteration number (if the event is recorded
                           ; during test execution)
}

<event-kind> ::= "runStarted" | "testStarted" | "testCaseStarted" |
  "issueRecorded" | "testCaseEnded" | "testEnded" | "testSkipped" |
  "runEnded" | "valueAttached" | "testCancelled" | "testCaseCancelled"
  ; additional event kinds may be added in the future

<issue> ::= {
  "isKnown": <bool>, ; is this a known issue or not?
  "severity": <string>, ; the severity of the issue
  "isFailure": <bool>, ; if the issue is a failing issue
  ["sourceLocation": <source-location>,] ; where the issue occurred, if known
}

<attachment> ::= {
  "path": <string>, ; the absolute path to the attachment on disk
}

<message> ::= {
  "symbol": <message-symbol>,
  "text": <string>, ; the human-readable text of this message
}

<message-symbol> ::= "default" | "skip" | "pass" | "passWithKnownIssue" |
  "fail" | "difference" | "warning" | "details"

See Also

Relevant Swift Evolution proposals

Proposal Number	Summary	Swift Version	Schema Version
ST-0002	Introduced the initial version of this JSON schema.	6.0	0
ST-0009	Added attachments.	6.2	0
ST-0013	Added test issue severity and isFailure.	6.3	"6.3"
ST-0016	Added test cancellation.	6.3	"6.3"
ST-0019	Added tags, bugs, and timeLimit.	6.4	"6.4"
ST-0020	Added filePath.	6.3	"6.3"
ST-0024	Added iteration.	6.4	"6.4"